Track Jupyter Notebooks
Use Weights & Biases with Jupyter to get interactive visualizations without leaving your notebook. Combine custom analysis, experiments, and prototypes, all fully logged!
Use Cases for W&B with Jupyter notebooks
- Iterative experimentation: Run and re-run experiments, tweaking parameters, and have all the runs you do saved automatically to W&B without having to take manual notes along the way.
- Code saving: When reproducing a model, it's hard to know which cells in a notebook ran, and in which order. Turn on code saving on your settings page to save a record of cell execution for each experiment.
- Custom analysis: Once runs are logged to W&B, it's easy to get a dataframe from the API and do custom analysis, then log those results to W&B to save and share in reports.
Getting started in a notebook
Start your notebook with the following code to install W&B and link your account:
!pip install wandb -qqq
Next, set up your experiment and save hyperparameters:
wandb.init() , start a new cell with
%%wandb to see live graphs in the notebook. If you run this cell multiple times, data will be appended to the run.
# Your training loop here
Try it for yourself in this quick example notebook →
Rendering live W&B interfaces directly in your notebooks
You can also display any existing dashboards, sweeps or reports directly in your notebook using the
# Display a project workspace
# Display a single run
# Display a sweep
# Display a report
# Specify the height of embedded iframe
%wandb USERNAME/PROJECT -h 2048
As an alternative to the
%wandb magics, after running
wandb.init() you can end any cell with
wandb.run to show in-line graphs, or call
ipython.display(...) on any report, sweep, or run object returned from our apis.
# Initialize wandb.run first
# If cell outputs wandb.run, you'll see live graphs
Want to know more about what you can do with W&B? Check out our guide to logging data and media, learn how to integrate us with your favorite ML toolkits, or just dive straight into the reference docs or our repo of examples.
Additional Jupyter features in W&B
- Easy authentication in Colab: When you call
wandb.initfor the first time in a Colab, we automatically authenticate your runtime if you're currently logged in to W&B in your browser. On the overview tab of your run page, you'll see a link to the Colab.
- Jupyter Magic: Display dashboards, sweeps and reports directly in your notebooks. The
%wandbmagic accepts a path to your project, sweeps or reports and will render the W&B interface directly in the notebook.
- Launch dockerized Jupyter: Call
wandb docker --jupyterto launch a docker container, mount your code in it, ensure Jupyter is installed, and launch on port 8888.
- Run cells in arbitrary order without fear: By default, we wait until the next time
wandb.initis called to mark a run as "finished". That allows you to run multiple cells (say, one to set up data, one to train, one to test) in whatever order you like and have them all log to the same run. If you turn on code saving in settings, you'll also log the cells that were executed, in order and in the state in which they were run, enabling you to reproduce even the most non-linear of pipelines. To mark a run as complete manually in a Jupyter notebook, call
run = wandb.init()
# training script and logging goes here
How do I silence W&B info messages?
To disable standard wandb logging and info messages (e.g. project info at the start of a run), run the following in a notebook cell before running
- Jupyter Magic
os.environ["WANDB_SILENT"] = "True"
If you see log messages like
INFO SenderThread:11484 [sender.py:finish():979] in your notebook, you can disable those with the following:
logger = logging.getLogger("wandb")
How do I set the
If you're seeing the error message
"Failed to query for notebook name, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable," you can resolve it by setting the environment variable. There's multiple ways to do so:
- Jupyter Magic
%env "WANDB_NOTEBOOK_NAME" "notebook name here"
os.environ["WANDB_NOTEBOOK_NAME"] = "notebook name here"