This is the multi-page printable view of this section. Click here to print.
Here are the most commonly asked questions across all categories.
wandb.init do to my training process?If you can’t find what you are looking for, browse through the popular categories below or search through articles based on categories.
Browse articles by category.
The following support questions are tagged with Academic. If you don’t see your question answered, try asking the community, or email support@wandb.com.
Accurately credit all contributors in your report by adding multiple authors.
To add multiple authors, click on the + icon next to the name of the author. This will open a drop-down menu with all the users who have access to the report. Select the users you want to add as authors.
The following support questions are tagged with Administrator. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Alerts. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Anonymous. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Artifacts. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Aws. If you don’t see your question answered, try asking the community, or email support@wandb.com.
Set unique tags with wandb.init(tags='your_tag'). This allows efficient filtering of project runs by selecting the corresponding tag in a Project Page’s Runs Table.
For more information on wandb.int, see the documentation.
The following support questions are tagged with Billing. If you don’t see your question answered, try asking the community, or email support@wandb.com.
Students can apply for an academic plan by following these steps:
Yes, you can also use tags or custom metadata to categorize runs. That can be done using the Group button which is available in the Workspace and Runs views of the project.
By default, W&B does not log dataset examples. By default, W&B logs code and system metrics.
Two methods exist to turn off code logging with environment variables:
WANDB_DISABLE_CODE to true to turn off all code tracking. This action prevents retrieval of the git SHA and the diff patch.WANDB_IGNORE_GLOBS to *.patch to stop syncing the diff patch to the servers, while keeping it available locally for application with wandb restore.As an administrator, you can also turn off code saving for your team in your team’s settings:
https://wandb.ai/<team>/settings. Where <team> is the name of your team.Yes. To overwrite the run name with the run ID, use the following code snippet:
import wandb
wandb.init()
wandb.run.name = wandb.run.id
wandb.run.save()
For example, I want to log training accuracy per batch and validation accuracy per epoch.
Yes, log indices like batch and epoch alongside your metrics. Use wandb.log({'train_accuracy': 0.9, 'batch': 200}) in one step and wandb.log({'val_accuracy': 0.8, 'epoch': 4}) in another. In the UI, set the desired value as the x-axis for each chart. To set a default x-axis for a specific index, use Run.define_metric(). For the example provided, use the following code:
wandb.init()
wandb.define_metric("batch")
wandb.define_metric("epoch")
wandb.define_metric("train_accuracy", step_metric="batch")
wandb.define_metric("val_accuracy", step_metric="epoch")
If a grid search completes but some W&B Runs need re-execution due to crashes, delete the specific W&B Runs to re-run. Then, select the Resume button on the sweep control page. Start new W&B Sweep agents using the new Sweep ID.
W&B Run parameter combinations that completed are not re-executed.
If training occurs on an offline machine, use the following steps to upload results to the servers:
WANDB_MODE=offline to save metrics locally without an internet connection.wandb init in your directory to set the project name.wandb sync YOUR_RUN_DIRECTORY to transfer metrics to the cloud service and access results in the hosted web app.To confirm the run is offline, check run.settings._offline or run.settings.mode after executing wandb.init().
Use wandb.init(mode="disabled") or set WANDB_MODE=disabled to configure W&B as a no-operation (NOOP) for testing purposes.
wandb.init(mode="disabled") does not prevent W&B from saving artifacts to WANDB_CACHE_DIR.Yes. Type “/mark” anywhere in the document and press enter to insert a Markdown block. This allows editing with Markdown as before.
To authenticate W&B, complete the following steps: create a requirements.txt file if using a built-in Amazon SageMaker estimator. For details on authentication and setting up the requirements.txt file, refer to the SageMaker integration guide.
Key engineers and support staff at W&B access logged values for debugging purposes with user permission. All data stores encrypt data at rest, and audit logs record access. For complete data security from W&B employees, license the self-hosted solution to run a W&B server within your own infrastructure.
Use the ${args_no_boolean_flags} macro in the command section of the configuration to pass hyperparameters as boolean flags. This macro automatically includes boolean parameters as flags. If param is True, the command receives --param. If param is False, the flag is omitted.
A run can have multiple tags, so grouping by tags is not supported. Add a value to the config object for these runs and group by this config value instead. This can be accomplished using the API.
To publish the sweep_id so that any W&B Sweep agent can access it, implement a method for these agents to read and execute the sweep_id.
For example, launch an Amazon EC2 instance and execute wandb agent on it. Use an SQS queue to broadcast the sweep_id to multiple EC2 instances. Each instance can then retrieve the sweep_id from the queue and initiate the process.
The following support questions are tagged with Charts. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Connectivity. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Crashing And Hanging Runs. If you don’t see your question answered, try asking the community, or email support@wandb.com.
No. Run Finished alerts (activated with the Run Finished setting in User Settings) operate only with Python scripts and remain turned off in Jupyter Notebook environments to avoid notifications for each cell execution.
Use wandb.alert() in notebook environments instead.
Arguments passed to wandb.init override environment variables. To set a default directory other than the system default when the environment variable isn’t set, use wandb.init(dir=os.getenv("WANDB_DIR", my_default_override)).
Access hyperparameter names and values from the sweep configuration using wandb.config, which acts like a dictionary.
For runs outside a sweep, set wandb.config values by passing a dictionary to the config argument in wandb.init. In a sweep, any configuration supplied to wandb.init serves as a default value, which the sweep can override.
Use config.setdefaults for explicit behavior. The following code snippets illustrate both methods:
# Set default values for hyperparameters
config_defaults = {"lr": 0.1, "batch_size": 256}
# Start a run and provide defaults
# that a sweep can override
with wandb.init(config=config_defaults) as run:
# Add training code here
...
# Set default values for hyperparameters
config_defaults = {"lr": 0.1, "batch_size": 256}
# Start a run
with wandb.init() as run:
# Update any values not set by the sweep
run.config.setdefaults(config_defaults)
# Add training code here
Weights and Biases has a bug bounty program. Access the security portal for details: https://security.wandb.ai/.
“Is the logging function lazy? I don’t want to depend on the network to send results to your servers while executing local operations.”
The wandb.log function writes a line to a local file and does not block network calls. When calling wandb.init, a new process starts on the same machine. This process listens for filesystem changes and communicates with the web service asynchronously, allowing local operations to continue uninterrupted.
The W&B client library supported both Python 2.7 and Python 3 through version 0.10. Support for Python 2.7 discontinued with version 0.11 due to Python 2’s end of life. Running pip install --upgrade wandb on a Python 2.7 system installs only new releases of the 0.10.x series. Support for the 0.10.x series includes critical bug fixes and patches only. The last version of the 0.10.x series that supports Python 2.7 is 0.10.33.
The W&B client library supported Python 3.5 until version 0.11. Support for Python 3.5 ended with version 0.12, which aligns with its end of life. For more details, visit version 0.12 release notes.
The library supports Python 2.7 and later, as well as Python 3.6 and later. The architecture facilitates integration with other programming languages. For monitoring other languages, contact contact@wandb.com.
W&B supports Single Sign-On (SSO) for the Multi-tenant offering through Auth0. SSO integration is compatible with any OIDC-compliant identity provider, such as Okta or Azure AD. To configure an OIDC provider, follow these steps:
grant_type to implicit flow.https://wandb.auth0.com/login/callback.Requirements for W&B
After completing the setup, contact the customer success manager (CSM) with the Client ID and Issuer URL for the application. W&B will establish an Auth0 connection using these details and enable SSO.
Yes, W&B uses the multiprocessing library. An error message like the following indicates a possible issue:
An attempt has been made to start a new process before the current process
has finished its bootstrapping phase.
To resolve this, add an entry point protection with if __name__ == "__main__":. This protection is necessary when running W&B directly from the script.
Pass a SHA or unique identifier to wandb.config.update(...) to associate a dataset with a training run. W&B stores no data unless wandb.save is called with the local file name.
You can share your report by embedding it. Click the Share button at the top right of your report, then copy the embedded code from the bottom of the pop-up window.
The following support questions are tagged with Environment Variables. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Experiments. If you don’t see your question answered, try asking the community, or email support@wandb.com.
multiprocessing library?crashed on the UI but is still running on my machine. What do I do to get my data back?Use the search bar to filter the reports list. Select an unwanted report to delete it individually, or select all reports and click ‘Delete Reports’ to remove them from the project.
The history object tracks metrics logged with wandb.log. Access the history object using the API:
api = wandb.Api()
run = api.run("username/project/run_id")
print(run.history())
To change an account from corporate to academic in Weights & Biases, follow these steps:
Link your academic email:
Apply for an academic plan:
Changing the colors of individual runs within a group is not possible. All runs in the same group share a common color.
Set the logging directory for W&B run data by configuring the environment variable WANDB_DIR. For example:
os.environ["WANDB_DIR"] = os.path.abspath("your/directory")
Expand the image panel and use the step slider to navigate through images from different steps. This process facilitates comparison of a model’s output changes during training.
At the beginning of the training script, call wandb.init with an experiment name. For example: wandb.init(name="my_awesome_run").
If an artifact reference is logged with W&B and versioning is enabled on the buckets, the version IDs appear in the Amazon S3 UI. To retrieve these version IDs and ETags in W&B, fetch the artifact and access the corresponding manifest entries. For example:
artifact = run.use_artifact("my_table:latest")
for entry in artifact.manifest.entries.values():
versionID = entry.extra.get("versionID")
etag = entry.extra.get("etag")
W&B tracks artifacts logged by each run and those used by each run to construct an artifact graph. This graph is a bipartite, directed, acyclic graph with nodes representing runs and artifacts. An example can be viewed here (click “Explode” to expand the graph).
Use the Public API to navigate the graph programmatically, starting from either an artifact or a run.
api = wandb.Api()
artifact = api.artifact("project/artifact:alias")
# Walk up the graph from an artifact:
producer_run = artifact.logged_by()
# Walk down the graph from an artifact:
consumer_runs = artifact.used_by()
# Walk down the graph from a run:
next_artifacts = consumer_runs[0].logged_artifacts()
# Walk up the graph from a run:
previous_artifacts = producer_run.used_artifacts()
api = wandb.Api()
run = api.run("entity/project/run_id")
# Walk down the graph from a run:
produced_artifacts = run.logged_artifacts()
# Walk up the graph from a run:
consumed_artifacts = run.used_artifacts()
# Walk up the graph from an artifact:
earlier_run = consumed_artifacts[0].logged_by()
# Walk down the graph from an artifact:
consumer_runs = produced_artifacts[0].used_by()
Using wandb.log({'final_accuracy': 0.9}) updates the final accuracy correctly. By default, wandb.log({'final_accuracy': <value>}) updates wandb.settings['final_accuracy'], which reflects the value in the runs table.
There are several ways to manage experiments.
For complex workflows, use multiple runs and set the group parameters in wandb.init to a unique value for all processes within a single experiment. The Runs tab will group the table by group ID, ensuring that visualizations function properly. This approach enables concurrent experiments and training runs while logging results in one location.
For simpler workflows, call wandb.init with resume=True and id=UNIQUE_ID, then call wandb.init again with the same id=UNIQUE_ID. Log normally with wandb.log or wandb.summary, and the run values will update accordingly.
To make a project public, follow these steps:
If the “Public” option is not available due to restrictions, consider the following options:
The / character separates logged panels in the W&B UI. By default, the segment of the logged item’s name before the / defines a group of panels known as a “Panel Section.”
wandb.log({"val/loss": 1.1, "val/acc": 0.3})
wandb.log({"train/loss": 0.1, "train/acc": 0.94})
In the Workspace settings, adjust the grouping of panels based on either the first segment or all segments separated by /.
To recover deleted runs, complete the following steps:
Notes:
To regain access to an account when unable to receive a password reset email:
To remove projects from a team space without admin privileges, follow these options:
If unable to contact the admin, reach out to a billing admin or another authorized user in your organization for assistance.
To resolve login issues, follow these steps:
To resolve the “Filestream rate limit exceeded” error in Weights & Biases (W&B), follow these steps:
Optimize logging:
Check for outages:
Contact support:
To resume a sweep, pass the sweep_id to the wandb.agent() function.
import wandb
sweep_id = "your_sweep_id"
def train():
# Training code here
pass
wandb.agent(sweep_id=sweep_id, function=train)
Personal and service account keys can be rotated or revoked. Create a new API key or service account user, then reconfigure scripts to use the new key. After reconfiguration, remove the old API key from your profile or team.
When wandb.init is invoked, the system automatically collects git information, including the remote repository link and the SHA of the latest commit. This information appears on the run page. Ensure the current working directory when executing the script is within a git-managed folder to view this information.
The git commit and the command used to run the experiment remain visible to the user but are hidden from external users. In public projects, these details remain private.
View the bytes stored, bytes tracked, and tracked hours for your organization within organization settings:
https://wandb.ai/account-settings/<organization-name>/settings.Ensure to replace values enclosed in <> with your organization’s name.
To receive W&B alerts in Teams, follow these steps:
If a training program uses multiple processes, structure the program to avoid making wandb method calls from processes without wandb.init().
Manage multiprocess training using these approaches:
wandb.init in all processes and use the group keyword argument to create a shared group. Each process will have its own wandb run, and the UI will group the training processes together.wandb.init from only one process and pass data to log through multiprocessing queues.Direct integration of Plotly or Bokeh figures into tables is not supported. Instead, export the figures to HTML and include the HTML in the table. Below are examples demonstrating this with interactive Plotly and Bokeh charts.
import wandb
import plotly.express as px
# Initialize a new run
run = wandb.init(project="log-plotly-fig-tables", name="plotly_html")
# Create a table
table = wandb.Table(columns=["plotly_figure"])
# Define path for Plotly figure
path_to_plotly_html = "./plotly_figure.html"
# Create a Plotly figure
fig = px.scatter(x=[0, 1, 2, 3, 4], y=[0, 1, 4, 9, 16])
# Export Plotly figure to HTML
# Setting auto_play to False prevents animated Plotly charts from playing automatically
fig.write_html(path_to_plotly_html, auto_play=False)
# Add Plotly figure as HTML file to the table
table.add_data(wandb.Html(path_to_plotly_html))
# Log Table
run.log({"test_table": table})
wandb.finish()
from scipy.signal import spectrogram
import holoviews as hv
import panel as pn
from scipy.io import wavfile
import numpy as np
from bokeh.resources import INLINE
hv.extension("bokeh", logo=False)
import wandb
def save_audio_with_bokeh_plot_to_html(audio_path, html_file_name):
sr, wav_data = wavfile.read(audio_path)
duration = len(wav_data) / sr
f, t, sxx = spectrogram(wav_data, sr)
spec_gram = hv.Image((t, f, np.log10(sxx)), ["Time (s)", "Frequency (Hz)"]).opts(
width=500, height=150, labelled=[]
)
audio = pn.pane.Audio(wav_data, sample_rate=sr, name="Audio", throttle=500)
slider = pn.widgets.FloatSlider(end=duration, visible=False)
line = hv.VLine(0).opts(color="white")
slider.jslink(audio, value="time", bidirectional=True)
slider.jslink(line, value="glyph.location")
combined = pn.Row(audio, spec_gram * line, slider).save(html_file_name)
html_file_name = "audio_with_plot.html"
audio_path = "hello.wav"
save_audio_with_bokeh_plot_to_html(audio_path, html_file_name)
wandb_html = wandb.Html(html_file_name)
run = wandb.init(project="audio_test")
my_table = wandb.Table(columns=["audio_with_plot"], data=[[wandb_html], [wandb_html]])
run.log({"audio_table": my_table})
run.finish()
One effective approach for logging models in a sweep involves creating a model artifact for the sweep. Each version represents a different run from the sweep. Implement it as follows:
wandb.Artifact(name="sweep_name", type="model")
To change the billing address, contact the support team (support@wandb.com).
If you encounter SSL or network errors, such as wandb: Network error (ConnectionError), entering retry loop, use the following solutions:
update-ca-certificates. A valid SSL certificate is essential for syncing training logs to mitigate security risks.WANDB_MODE to offline, and sync files later from a device with Internet access.For the SSL CERTIFICATE_VERIFY_FAILED error, this issue might stem from a company firewall. Configure local CAs and execute:
export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
Select the panel grid and press delete or backspace. Click the drag handle in the top-right corner to select the panel grid.
To delete a team from an account:
To delete an organization account, follow these steps, contact the support team (support@wandb.com).
To downgrade a subscription plan, contact the support team at support@wandb.com with your current plan details and the desired plan.
To enable code logging for sweeps, add wandb.log_code() after initializing the W&B Run. This action is necessary even when code logging is enabled in the W&B profile settings. For advanced code logging, refer to the docs for wandb.log_code() here.
To export a list of users from a W&B organization, an admin uses the SCIM API with the following code:
import base64
import requests
def encode_base64(username, key):
auth_string = f'{username}:{key}'
return base64.b64encode(auth_string.encode('utf-8')).decode('utf-8')
username = '' # Organization admin username
key = '' # API key
scim_base_url = 'https://api.wandb.ai/scim/v2'
users_endpoint = f'{scim_base_url}/Users'
headers = {
'Authorization': f'Basic {encode_base64(username, key)}',
'Content-Type': 'application/scim+json'
}
response = requests.get(users_endpoint, headers=headers)
users = []
for user in response.json()['Resources']:
users.append([user['userName'], user['emails']['Value']])
Modify the script to save the output as needed.
To retrieve artifacts from the best performing run in a sweep, use the following code:
api = wandb.Api()
sweep = api.sweep("entity/project/sweep_id")
runs = sorted(sweep.runs, key=lambda run: run.summary.get("val_acc", 0), reverse=True)
best_run = runs[0]
for artifact in best_run.logged_artifacts():
artifact_path = artifact.download()
print(artifact_path)
To find your API key for Weights and Biases (W&B):
To resolve this error, add ?workspace=clear to the end of the URL and press Enter. This action directs you to a cleared version of the project page workspace.
To join a team, follow these steps:
Call wandb.run.save() to save the current run. Retrieve the name using wandb.run.name.
If you encounter the error message "Failed to query for notebook name, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable," resolve it by setting the environment variable. Multiple methods accomplish this:
%env "WANDB_NOTEBOOK_NAME" "notebook name here"
import os
os.environ["WANDB_NOTEBOOK_NAME"] = "notebook name here"
Tables remain the only feature from Markdown without a direct WYSIWYG equivalent. To add a table, insert a Markdown block and create the table inside it.
If an error occurs when installing wandb that states:
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
Install psutil directly from a pre-built wheel. Determine your Python version and operating system at https://pywharf.github.io/pywharf-pkg-repo/psutil.
For example, to install psutil on Python 3.8 in Linux:
WHEEL_URL=https://github.com/pywharf/pywharf-pkg-repo/releases/download/psutil-5.7.0-cp38-cp38-manylinux2010_x86_64.whl#sha256=adc36dabdff0b9a4c84821ef5ce45848f30b8a01a1d5806316e068b5fd669c6d
pip install $WHEEL_URL
After installing psutil, run pip install wandb to complete the installation of wandb.
Press Ctrl+D on the keyboard to stop a script instrumented with W&B.
Use wandb.init and run.finish() to log multiple runs within a single script:
run = wandb.init(reinit=True) to allow reinitialization of runs.run.finish() at the end of each run to complete logging.import wandb
for x in range(10):
run = wandb.init(reinit=True)
for y in range(100):
wandb.log({"metric": x + y})
run.finish()
Alternatively, utilize a Python context manager to automatically finish logging:
import wandb
for x in range(10):
run = wandb.init(reinit=True)
with run:
for y in range(100):
run.log({"metric": x + y})
These examples show logging losses a couple of different ways using wandb.log().
wandb.log({f"losses/loss-{ii}": loss for ii,
loss in enumerate(losses)})
# Converts losses to a histogram
wandb.log({"losses": wandb.Histogram(losses)})
For more, see the documentation on logging.
Occasionally, it is necessary to mark an artifact as the output of a previously logged run. In this case, reinitialize the old run and log new artifacts as follows:
with wandb.init(id="existing_run_id", resume="allow") as run:
artifact = wandb.Artifact("artifact_name", "artifact_type")
artifact.add_file("my_data/file.txt")
run.log_artifact(artifact)
To launch automated tests or internal tools that log to W&B, create a Service Account on the team settings page. This action allows the use of a service API key for automated jobs, including those running through continuous integration. To attribute service account jobs to a specific user, set the WANDB_USERNAME or WANDB_USER_EMAIL environment variables.
When using a shared machine, ensure that runs log to the correct WandB account by setting the WANDB_API_KEY environment variable for authentication. If sourced in the environment, this variable provides the correct credentials upon login. Alternatively, set the environment variable directly in the script.
Execute the command export WANDB_API_KEY=X, replacing X with your API key. Logged-in users can find their API key at wandb.ai/authorize.
Create a multi-line custom chart with wandb.plot.line_series(). Navigate to the project page to view the line chart. To add a legend, include the keys argument in wandb.plot.line_series(). For example:
wandb.log(
{
"my_plot": wandb.plot.line_series(
xs=x_data, ys=y_data, keys=["metric_A", "metric_B"]
)
}
)
Refer to additional details about multi-line plots here under the Multi-line tab.
The .name attribute of a wandb.Run is accessible as follows:
import wandb
wandb.init()
run_name = wandb.run.name
To rename a project:
Note:
model-registry, it cannot be renamed. Contact support for assistance with protected names.To renew an expired license, contact the support team at support@wandb.com for assistance with the renewal process and to receive a new license key.
To delete a W&B account, navigate to the Settings page, scroll to the bottom, and click the Delete Account button.
To resolve a run initialization timeout error, follow these steps:
WANDB_INIT_TIMEOUT environment variable:
import os
os.environ['WANDB_INIT_TIMEOUT'] = '600'
WANDB_DEBUG=true and WANDB_CORE_DEBUG=true for detailed logs.debug.log, debug-internal.log, debug-core.log, and output.log for errors.To resolve permission errors when logging a run to a W&B entity, follow these steps:
import wandb
run = wandb.init(entity="your_entity", project="your_project")
run.log({'example_metric': 1})
run.finish()
WANDB_API_KEY environment variable:
export WANDB_API_KEY='your_api_key'
wandb login --relogin --host=<host-url>
export WANDB_BASE_URL=<host-url>
Use save_code=True in wandb.init to save the main script or notebook that launches the run. To save all code for a run, version the code with Artifacts. The following example demonstrates this process:
code_artifact = wandb.Artifact(type="code")
code_artifact.add_file("./train.py")
wandb.log_artifact(code_artifact)
To manage artifacts that contain sensitive data or to schedule the deletion of artifact versions, set a TTL (time-to-live) policy. For detailed instructions, refer to the TTL guide.
To suppress log messages in your notebook such as this:
INFO SenderThread:11484 [sender.py:finish():979]
Set the log level to logging.ERROR to only show errors, suppressing output of info-level log output.
import logging
logger = logging.getLogger("wandb")
logger.setLevel(logging.ERROR)
To turn off log output completely, set the WANDB_SILENT environment variable. This must occur in a notebook cell before running wandb.login:
%env WANDB_SILENT=True
import os
os.environ["WANDB_SILENT"] = "True"
Set the environment variable WANDB_SILENT to true.
os.environ["WANDB_SILENT"] = "true"
%env WANDB_SILENT=true
WANDB_SILENT=true
To manage two W&B accounts from the same machine, store both API keys in a file. Use the following code in your repositories to switch between keys securely, preventing secret keys from being checked into source control.
if os.path.exists("~/keys.json"):
os.environ["WANDB_API_KEY"] = json.loads("~/keys.json")["work_account"]
The command wandb offline sets the environment variable WANDB_MODE=offline, preventing data from syncing to the remote W&B server. This action affects all projects, stopping the logging of data to W&B servers.
To suppress warning messages, use the following code:
import logging
logger = logging.getLogger("wandb")
logger.setLevel(logging.WARNING)
You can use W&B Sweeps with custom CLI commands if training configuration passes command-line arguments.
In the example below, the code snippet illustrates a bash terminal where a user trains a Python script named train.py, providing values that the script parses:
/usr/bin/env python train.py -b \
your-training-config \
--batchsize 8 \
--lr 0.00001
To implement custom commands, modify the command key in the YAML file. Based on the previous example, the configuration appears as follows:
program:
train.py
method: grid
parameters:
batch_size:
value: 8
lr:
value: 0.0001
command:
- ${env}
- python
- ${program}
- "-b"
- your-training-config
- ${args}
The ${args} key expands to all parameters in the sweep configuration, formatted for argparse as --param1 value1 --param2 value2.
For additional arguments outside of argparse, implement the following:
parser = argparse.ArgumentParser()
args, unknown = parser.parse_known_args()
Depending on the environment, python might refer to Python 2. To ensure invocation of Python 3, use python3 in the command configuration:
program:
script.py
command:
- ${env}
- python3
- ${program}
- ${args}
To use the resume parameter in W&B , set the resume argument in wandb.init() with entity, project, and id specified. The resume argument accepts values of "must" or "allow".
run = wandb.init(entity="your-entity", project="your-project", id="your-run-id", resume="must")
To update your payment method, follow these steps:
Note: To manage billing, you must be assigned as the billing admin for your organization.
Access the custom chart editor. Click on the currently selected chart type to open a menu displaying all presets. Hover over the preset to delete, then click the Trash icon.
Enable this option on the “Other settings” page of the custom chart editor. Changing the query to use a historyTable instead of a summaryTable provides the option to “Show step selector” in the custom chart editor. This feature includes a slider for selecting the step.
If someone runs the script with anonymous="allow":
When a W&B user finds and runs the script, their results log correctly to their account, just like a normal run.
W&B queues events in memory and writes them to disk asynchronously to manage failures and support the WANDB_MODE=offline configuration, allowing synchronization after logging.
In the terminal, observe the path to the local run directory. This directory includes a .wandb file, which serves as the datastore. For image logging, W&B stores images in the media/images subdirectory before uploading them to cloud storage.
W&B integrates with TensorBoard and improves experiment tracking tools. The founders created W&B to address common frustrations faced by TensorBoard users. Key improvements include:
Model Reproducibility: W&B facilitates experimentation, exploration, and model reproduction. It captures metrics, hyperparameters, code versions, and saves model checkpoints to ensure reproducibility.
Automatic Organization: W&B streamlines project handoffs and vacations by providing an overview of all attempted models, which saves time by preventing the re-execution of old experiments.
Quick Integration: Integrate W&B into your project in five minutes. Install the free open-source Python package and add a few lines of code. Logged metrics and records appear with each model run.
Centralized Dashboard: Access a consistent dashboard regardless of where training occurs—locally, on lab clusters, or cloud spot instances. Eliminate the need to manage TensorBoard files across different machines.
Robust Filtering Table: Search, filter, sort, and group results from various models efficiently. Easily identify the best-performing models for different tasks, an area where TensorBoard often struggles with larger projects.
Collaboration Tools: W&B enhances collaboration for complex machine learning projects. Share project links and utilize private teams for result sharing. Create reports with interactive visualizations and markdown descriptions for work logs or presentations.
Limit each project to approximately 10,000 runs for optimal performance.
Only files that change between two artifact versions incur storage costs.
Consider an image artifact named animals that contains two image files, cat.png and dog.png:
images
|-- cat.png (2MB) # Added in `v0`
|-- dog.png (1MB) # Added in `v0`
This artifact receives version v0.
When adding a new image, rat.png, a new artifact version, v1, is created with the following contents:
images
|-- cat.png (2MB) # Added in `v0`
|-- dog.png (1MB) # Added in `v0`
|-- rat.png (3MB) # Added in `v1`
Version v1 tracks a total of 6MB, but occupies only 3MB of space since it shares the remaining 3MB with v0. Deleting v1 reclaims the 3MB of storage associated with rat.png. Deleting v0 transfers the storage costs of cat.png and dog.png to v1, increasing its storage size to 6MB.
Metrics collect by default every 10 seconds. For higher resolution metrics, email contact@wandb.com.
When using sweeps with the SLURM scheduling system, run wandb agent --count 1 SWEEP_ID in each scheduled job. This command executes a single training job and then exits, facilitating runtime predictions for resource requests while leveraging the parallelism of hyperparameter searches.
With W&B Reports, follow these steps:
The following support questions are tagged with Hyperparameter. If you don’t see your question answered, try asking the community, or email support@wandb.com.
If the report conversion occurred through the message at the top, click the red “Revert” button to restore the prior state. Note that any changes made after conversion will be lost.
If a single Markdown block was converted, use cmd+z to undo.
If options to revert are unavailable because of a closed session, consider discarding the draft or editing from the last saved version. If neither works, contact W&B Support.
If a run is not explicitly named, W&B assigns a random name to identify it in your project. Examples of random names are pleasant-flower-4 and `misunderstood-glade-2.
If you are the admin for your instance, review the User Management section for instructions on adding users and creating teams.
It is critical to avoid interference with training runs. W&B operates in a separate process, ensuring that training continues even if W&B experiences a crash. In the event of an internet outage, W&B continually retries sending data to wandb.ai.
LaTeX integrates seamlessly into reports. To add LaTeX, create a new report and begin typing in the rich text area to write notes and save custom visualizations and tables.
On a new line, press / and navigate to the inline equations tab to insert LaTeX content.
This error indicates that the library encounters an issue launching the process that synchronizes data to the server.
The following workarounds resolve the issue in specific environments:
wandb.init(settings=wandb.Settings(start_method="fork"))
For versions prior to 0.13.0, use:
wandb.init(settings=wandb.Settings(start_method="thread"))
A service account cannot be added to multiple teams in W&B. Each service account is tied to a specific team.
You can change the group assigned to a completed run using the API. This feature does not appear in the web UI. Use the following code to update the group:
import wandb
api = wandb.Api()
run = api.run("<ENTITY>/<PROJECT>/<RUN_ID>")
run.group = "NEW-GROUP-NAME"
run.update()
Changing the username after account creation is not possible. Create a new account with the desired username instead.
A new account can use an email previously associated with a deleted account.
You can move a run from one project to another by following these steps:
W&B supports moving runs through the UI, but does not support copying runs. Artifacts logged with the runs do not transfer to the new project.
Create a scatter plot of the metric. Open the Edit menu and select Annotations. From there, plot the running maximum of the values.
When deleting a run, a prompt asks whether to delete the associated artifacts. Choosing this option permanently removes the artifacts, making recovery impossible, even if the run itself is restored later.
By default, wandb.init starts a process that syncs metrics in real time to the cloud. For offline use, set two environment variables to enable offline mode and sync later.
Set the following environment variables:
WANDB_API_KEY=$KEY, where $KEY is the API Key from your settings page.WANDB_MODE="offline".Here is an example of implementing this in a script:
import wandb
import os
os.environ["WANDB_API_KEY"] = "YOUR_KEY_HERE"
os.environ["WANDB_MODE"] = "offline"
config = {
"dataset": "CIFAR10",
"machine": "offline cluster",
"model": "CNN",
"learning_rate": 0.01,
"batch_size": 128,
}
wandb.init(project="offline-demo")
for i in range(100):
wandb.log({"accuracy": i})
Sample terminal output is shown below:
After completing work, run the following command to sync data to the cloud:
wandb sync wandb/dryrun-folder-name
To enable dark mode:
The Teams plan does not offer a monthly subscription option. This subscription is billed annually.
Check if the W&B multi-tenant cloud at wandb.ai is experiencing an outage by visiting the W&B status page at https://status.wandb.com.
Once a W&B Sweep starts, you cannot change the Sweep configuration. However, you can navigate to any table view, select runs using the checkboxes, and then choose the Create sweep menu option to generate a new Sweep configuration based on previous runs.
To add more seats to an account, follow these steps:
There is an anaconda package that is installable using either pip or conda. For conda, obtain the package from the conda-forge channel.
# Create a conda environment
conda create -n wandb-env python=3.8 anaconda
# Activate the environment
conda activate wandb-env
# Install wandb using pip
pip install wandb
conda activate myenv
conda install wandb --channel conda-forge
For installation issues, refer to this Anaconda documentation on managing packages for assistance.
The following support questions are tagged with Logs. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Metrics. If you don’t see your question answered, try asking the community, or email support@wandb.com.
Performance issues may arise on older hardware or with very large reports. To mitigate this, collapse sections of the report that are not currently in use.
The goal is to maintain the original appearance after transitioning to WYSIWYG, but the conversion process is not flawless. If significant discrepancies arise, report them for evaluation. Users can revert to the previous state until the editing session concludes.
You likely lost connection to your machine during training. Recover data by running wandb sync [PATH_TO_RUN]. The path to your run is a folder in your wandb directory that matches the Run ID of the ongoing run.
The following support questions are tagged with Notebooks. If you don’t see your question answered, try asking the community, or email support@wandb.com.
Check the Debug Bundle. An admin can retrieve it from the /system-admin page by selecting the W&B icon in the top right corner and then choosing Debug Bundle.
To optimize multiple metrics in a single run, use a weighted sum of the individual metrics.
metric_combined = 0.3 * metric_a + 0.2 * metric_b + ... + 1.5 * metric_n
wandb.log({"metric_combined": metric_combined})
Log the new combined metric and set it as the optimization objective:
metric:
name: metric_combined
goal: minimize
The following support questions are tagged with Outage. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Privacy. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Projects. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Python. If you don’t see your question answered, try asking the community, or email support@wandb.com.
Workspaces automatically load updated data. Auto-refresh does not apply to reports. Reload the page to refresh report data.
The following support questions are tagged with Reports. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Resuming. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Runs. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Security. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Storage. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Sweeps. If you don’t see your question answered, try asking the community, or email support@wandb.com.
Est. Runs column?
The following support questions are tagged with Tables. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Team Management. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Tensorboard. If you don’t see your question answered, try asking the community, or email support@wandb.com.
To upload a CSV to a report, use the wandb.Table format. Load the CSV in your Python script and log it as a wandb.Table object. This action renders the data as a table in the report.
Press / on a new line, scroll to the Image option, and drag and drop an image into the report.
The following support questions are tagged with User Management. If you don’t see your question answered, try asking the community, or email support@wandb.com.
There are various methods to version a model. Artifacts provide a tool for model versioning tailored to specific needs. A common approach for projects that explore multiple model architectures involves separating artifacts by architecture. Consider the following steps:
metadata attribute of artifacts to provide detailed descriptions of the architecture, similar to the use of config for a run.log_artifact. W&B builds a history of these checkpoints, labeling the most recent one with the latest alias. Refer to the latest checkpoint for any model architecture using architecture-name:latest.
No artifact logging: A warning appears on the command line when attempting to log an artifact to an anonymous run:
wandb: WARNING Artifacts logged anonymously cannot be claimed and expire after 7 days.
No profile or settings pages: The UI does not include certain pages, as they are only useful for real accounts.
When wandb.init() runs in a training script, an API call creates a run object on the servers. A new process starts to stream and collect metrics, allowing the primary process to function normally. The script writes to local files while the separate process streams data to the servers, including system metrics. To turn off streaming, run wandb off from the training directory or set the WANDB_MODE environment variable to offline.
The exponential moving average formula aligns with the one used in TensorBoard.
Refer to this explanation on Stack OverFlow for details on the equivalent Python implementation. The source code to TensorBoard’s smoothing algorithm (as of this writing) can be found here.
Avoid passing class attributes into wandb.log(). Attributes may change before the network call executes. When storing metrics as class attributes, use a deep copy to ensure the logged metric matches the attribute’s value at the time of the wandb.log() call.
If the library cannot connect to the internet, it enters a retry loop and continues to attempt to stream metrics until the network is restored. The program continues to run during this time.
To run on a machine without internet, set WANDB_MODE=offline. This configuration stores metrics locally on the hard drive. Later, call wandb sync DIRECTORY to stream the data to the server.
The number of points sent affects the loading time of graphs in the UI. For lines exceeding 1,000 points, the backend samples the data down to 1,000 points before sending it to the browser. This sampling is nondeterministic, resulting in different sampled points upon page refresh.
Log fewer than 10,000 points per metric. Logging over 1 million points in a line significantly increases page load time. Explore strategies to minimize logging footprint without sacrificing accuracy in this Colab. With more than 500 columns of config and summary metrics, only 500 display in the table.
W&B supports projects that log only scalars by allowing explicit specification of files or data for upload. Refer to this example in PyTorch that demonstrates logging without using images.
To log specific metrics in each batch and standardize plots, log the desired x-axis values alongside the metrics. In the custom plots, click edit and select a custom x-axis.
wandb.log({"batch": batch_idx, "loss": 0.3})
wandb.log({"epoch": epoch, "val_acc": 0.94})
A service account (Enterprise-only feature) represents a non-human or machine user, which can automate common tasks across teams and projects or ones that are not specific to a particular human user. You can create a service account within a team and use its API key to read from and write to projects within that team.
Among other things, service accounts are useful for tracking automated jobs logged to wandb, like periodic retraining, nightly builds, and so on. If you’d like, you can associate a username with one of these machine-launched runs with the environment variables WANDB_USERNAME or WANDB_USER_EMAIL.
Refer to Team Service Account Behavior for more information.
You can get the API key for a service account in your team at <WANDB_HOST_URL>/<your-team-name>/service-accounts. Alternatively you can go to the Team settings for your team and then refer to the Service Accounts tab.
To create a new service account for your team:
For additional information about teams, visit the teams section.
W&B provides an estimated number of Runs generated when creating a W&B Sweep with a discrete search space. This total reflects the cartesian product of the search space.
For instance, consider the following search space:
In this case, the Cartesian product equals 9. W&B displays this value in the App UI as the estimated run count (Est. Runs):
To retrieve the estimated Run count programmatically, use the expected_run_count attribute of the Sweep object within the W&B SDK:
sweep_id = wandb.sweep(
sweep_configs, project="your_project_name", entity="your_entity_name"
)
api = wandb.Api()
sweep = api.sweep(f"your_entity_name/your_project_name/sweeps/{sweep_id}")
print(f"EXPECTED RUN COUNT = {sweep.expected_run_count}")
The summary displays in the table, while the log saves all values for future plotting.
For instance, call wandb.log whenever accuracy changes. By default, wandb.log() updates the summary value unless set manually for that metric.
The scatterplot and parallel coordinate plots use the summary value, while the line plot shows all values recorded by .log.
Some users prefer to set the summary manually to reflect the optimal accuracy instead of the most recent accuracy logged.
A team serves as a collaborative workspace for users working on the same projects. An entity represents either a username or a team name. When logging runs in W&B, set the entity to a personal or team account using wandb.init(entity="example-team").
A team serves as a collaborative workspace for users working on the same projects. An organization functions as a higher-level entity that can include multiple teams, often related to billing and account management.
These modes are available:
online (default): The client sends data to the wandb server.offline: The client stores data locally on the machine instead of sending it to the wandb server. Use the wandb sync command to synchronize the data later.disabled: The client simulates operation by returning mocked objects and prevents any network communication. All logging is turned off, but all API method stubs remain callable. This mode is typically used for testing.Some functionalities are hidden under a feature flag in the Beta Features section of a team’s settings.
Visit this page for an overview of the available roles and permissions.
Personal Entities are unavailable for accounts created after May 21, 2024. W&B encourages all users to log new projects to a Team to enable sharing of results.
For the affected run, check debug.log and debug-internal.log in wandb/run-<date>_<time>-<run-id>/logs in the directory where your code is running.
Refer to the link for details on roles and permissions: Team Roles and Permissions.
Reports created within an individual’s private project remain visible only to that user. The user can share their project with a team or the public.
In team projects, the administrator or the member who created the report can toggle permissions between edit and view access for other team members. Team members can share reports.
To share a report, select the Share button in the upper right corner. Provide an email address or copy the magic link. Users invited by email must log into W&B to view the report, while users with a magic link do not need to log in.
Shared reports maintain view-only access.
Artifacts inherit access permissions from their parent project:
This section outlines workflows for managing and editing Artifacts. Many workflows utilize the W&B API, a component of the client library that provides access to W&B-stored data.
When visualizing metrics against an X-axis other than Step, expect to see fewer data points. Metrics must log at the same Step to remain synchronized. Only metrics logged at the same Step are sampled while interpolating between samples.
Guidelines
Bundle metrics into a single log() call. For example, instead of:
wandb.log({"Precision": precision})
...
wandb.log({"Recall": recall})
Use:
wandb.log({"Precision": precision, "Recall": recall})
For manual control over the step parameter, synchronize metrics in the code as follows:
wandb.log({"Precision": precision}, step=step)
...
wandb.log({"Recall": recall}, step=step)
Ensure the step value remains the same in both log() calls for the metrics to log under the same step and sample together. The step value must increase monotonically in each call. Otherwise, the step value is ignored.
This indicates a connection problem. If the server loses internet access and data stops syncing to W&B, the system marks the run as crashed after a brief retry period.
If the message “No visualization data logged yet” appears, the script has not executed the first wandb.log call. This situation may occur if the run takes a long time to complete a step. To expedite data logging, log multiple times per epoch instead of only at the end.
When logging various data types under the same key, split them in the database. This results in multiple entries of the same metric name in the UI dropdown. The data types grouped are number, string, bool, other (primarily arrays), and any wandb data type such as Histogram or Image. Send only one type per key to prevent this issue.
Metric names are case-insensitive. Avoid using names that differ only by case, such as "My-Metric" and "my-metric".
W&B has a minimal impact on training performance under normal usage conditions. Normal use includes logging at a rate of less than once per second and limiting data to a few megabytes per step. W&B operates in a separate process with non-blocking function calls, ensuring that brief network outages or intermittent disk read/write issues do not disrupt performance. Excessive logging of large amounts of data may lead to disk I/O issues. For further inquiries, contact support.
The following support questions are tagged with Workspaces. If you don’t see your question answered, try asking the community, or email support@wandb.com.
The following support questions are tagged with Wysiwyg. If you don’t see your question answered, try asking the community, or email support@wandb.com.