Sweeps Quickstart

Start from any machine learning model and get a parallel hyperparameter sweep running in minutes.

Want to see a working example? Here's example code and an example dashboard.

Draw insights from large hyperparameter tuning experiments with interactive dashboards.

Trying to quickly generate a sweep based on runs you've already logged? Check out this guide.

1. Set up wandb

Set up your account

  1. Start with a W&B account. Create one now →

  2. Go to your project folder in your terminal and install our library: pip install wandb

  3. Inside your project folder, log in to W&B: wandb login

Set up your Python training script

Trying to run a hyperparameter sweep from a Jupyter notebook? You want these instructions.

  1. Import our library wandb.

  2. Pass the hyperparameter values to wandb.init to populate wandb.config.

  3. Use the values in the config to build your model and execute training.

  4. Log metrics to see them in the live dashboard.

See the code snippets below for several ways to set hyperparameter values so that training scripts can be run stand-alone or as part of a sweep.

Command Line Arguments
In-line Dictionary
config.py File
Command Line Arguments
train.py
import argparse
import wandb
# Build your ArgumentParser however you like
parser = setup_parser()
# Get the hyperparameters
args = parser.parse_args()
# Pass them to wandb.init
wandb.init(config=args)
# Access all hyperparameter values through wandb.config
config = wandb.config
# Set up your model
model = make_model(config)
# Log metrics inside your training loop
for epoch in range(config["epochs"]):
val_acc, val_loss = model.fit()
metrics = {"validation_accuracy": val_acc,
"validation_loss": val_loss}
wandb.log(metrics)
In-line Dictionary
train.py
import wandb
# Set up your default hyperparameters
hyperparameter_defaults = dict(
channels=[16, 32],
batch_size=100,
learning_rate=0.001,
optimizer="adam",
epochs=2,
)
# Pass your defaults to wandb.init
wandb.init(config=hyperparameter_defaults)
# Access all hyperparameter values through wandb.config
config = wandb.config
# Set up your model
model = make_model(config)
# Log metrics inside your training loop
for epoch in range(config["epochs"]):
val_acc, val_loss = model.fit()
metrics = {"validation_accuracy": val_acc,
"validation_loss": val_loss}
wandb.log(metrics)
config.py File
train.py
import wandb
import config # python file with default hyperparameters
# Set up your default hyperparameters
hyperparameters = config.config
# Pass them wandb.init
wandb.init(config=hyperparameters)
# Access all hyperparameter values through wandb.config
config = wandb.config
# Set up your model
model = make_model(config)
# Log metrics inside your training loop
for epoch in range(config["epochs"]):
val_acc, val_loss = model.fit()
metrics = {"validation_accuracy": val_acc,
"validation_loss": val_loss}
wandb.log(metrics)

See a full code example →

2. Configure your sweep

Set up a YAML file to specify the hyperparameters you wish to sweep over, along with the structure of the sweep like the training script to call, the search strategy and stopping criteria to use, etcetera.

Here are some resources on configuring sweeps:

  1. Example YAML files : an example script and several different YAML files

  2. Configuration: full specs to set up your sweep config

  3. Jupyter Notebook: set up your sweep config with a Python dictionary instead of a YAML file

  4. Generate config from UI: take an existing W&B project and generate a config file

  5. Feed in prior runs: take previous runs and add them to a new sweep

Here's an example sweep config file called sweep.yaml:

sweep.yaml
sweep.yaml
program: train.py
method: bayes
metric:
name: validation_loss
goal: minimize
parameters:
learning_rate:
min: 0.0001
max: 0.1
optimizer:
values: ["adam", "sgd"]

If you specify a metric to optimize, make sure you're logging it. In this example, I have validation_loss in my config file, so I have to log that exact metric name in my script:

wandb.log({"validation_loss": val_loss})

This example configuration will use a Bayesian search method to choose sets of hyperparameter values to pass as command line arguments to the train.py script on each step. The hyperparameters are also accessible via wandb.config after wandb.init is called.

If you're using argparse in your script, we recommend that you use underscores in your variable names instead of hyphens.

3. Initialize a sweep

After you've set up a sweep config file at sweep.yaml, run this command to get started:

wandb sweep sweep.yaml

This command will print out a sweep ID, which includes the entity name and project name. Copy that to use in the next step!

4. Launch agent(s)

On each machine or within each process that you'd like to contribute to the sweep, start an "agent". Each agent will poll the central W&B sweep server you launched with wandb sweep for the next set of hyperparameters to run. You'll want to use the same sweep ID for all agents who are participating in the same sweep.

In a shell on your own machine, run the wandb agent command:

wandb agent your-sweep-id

5. Visualize results

Open your project to see your live results in the sweep dashboard. With just a few clicks, construct rich, interactive charts like parallel coordinates plots, parameter importance analyses, and more.

Example dashboard →

6. Stop the agent

From the terminal, hit Ctrl+c to stop the run that the sweep agent is currently running. To kill the agent, hit Ctrl+c again after the run is stopped.