Docs
Variables

Managing Variables and Secrets in Trainwave

Trainwave provides flexible ways to manage environment variables and secrets for your machine learning jobs, ensuring your training workflows are both configurable and secure.

Job-Specific Variables

You can define environment variables directly within your trainwave.toml configuration file. These variables are then injected into your job's environment, making them easily accessible during training.

To dynamically load values from your local environment, use the ${VAR_NAME} syntax:

Example:

env_vars.WANDB_API_KEY = "${WANDB_API_KEY}"  # Uses your local WANDB_API_KEY value
env_vars.HUGGINGFACE_TOKEN = "${HF_TOKEN}"   # Uses your local HF_TOKEN value
env_vars.SOME_OTHER_VAR = "some_value"       # Sets a fixed value

This allows you to keep sensitive information out of your configuration files and manage it within your local environment.

Securely Storing Secrets

For API keys, tokens, and other sensitive data, Trainwave offers a secure secrets management system. You can store secrets at the organization or project level, ensuring they're readily available to your jobs without compromising security.

All secrets are encrypted at rest and in transit, providing a secure way to manage sensitive information. Secrets are never exposed in logs or job outputs.

Managing Secrets through the Web UI

  1. Navigate to your organization or project in the Trainwave web UI.
  2. Click on the Variables tab.
  3. Click the Add Variable button to add a new secret.

Managing Secrets with the CLI

The Trainwave CLI provides commands for managing secrets:

Set secrets:

wave secrets set SECRET_NAME=SECRET_VALUE SECRET_2=SECRET_2_VALUE

List secrets:

wave secrets list

For more information, see the CLI documentation.

Variable Precedence and Access

Trainwave uses a clear precedence order for resolving variables:

Job-Specific Variable > Project Variable > Organization Variable

This means that if a variable is defined at multiple levels, the most specific definition will be used.

All organization, project, and job-specific variables are injected into your job's environment. You can access them directly within your training scripts like any other environment variable.

ProjectsImages