Run Weights and Biases on your own machines using Docker
The W&B Server is a single Docker image that is simple to deploy. Your W&B data is saved on a persistent volume or an external database so data can be preserved across container versions. The server requires an instance with at least 4 cores and 8GB memory.
Run the W&B Server locally on your machine or host it in a private cloud. For serious work, we encourage you to set up and manage a scalable file system. We provide extensive technical support and frequent installation updates for privately hosted instances for enterprise customers.
Running wandb on localhost is great for initial testing, but to leverage the collaborative features of wandb/local you should host the service on a central server. Instructions for setting up a centralized server on various platforms can be found in the Production Setup section.
pip install wandb
wandb server start
If this is your first time logging in then you will need to create your local W&B Server account and authorize your API key.
If you're running
wandbon multiple machines or switching between a private instance and the wandb cloud, there are several ways to control where your runs are logged. If you want to send metrics to the shared private instance and you've configured DNS, you can
- set the host flag to the address of the private instance whenever you login:
wandb login --host=http://wandb.your-shared-local-host.com
- set the environment variable
WANDB_BASE_URLto the address of the local instance:
export WANDB_BASE_URL = "http://wandb.your-shared-local-host.com"
To switch to logging to the public cloud instance of wandb, set the host to
wandb login --cloud
export WANDB_BASE_URL = "https://api.wandb.ai"
We are pushing new versions of wandb/local to DockerHub regularly. To upgrade you can run:
$ wandb server start --upgrade
To upgrade your instance manually you can run the following
$ docker pull wandb/local
$ docker stop wandb-local
$ docker run --rm -d -v wandb:/vol -p 8080:8080 --name wandb-local wandb/local
You need a license to complete your configuration of a W&B Server. Open the Deploy Manager to generate a free license. If you do not already have a cloud account then you will need to create one to generate your free license. We offer two options:
- 1.Copy your license from your Deployment and navigate back to your W&B Server's localhost:
- 2.Add it to your local settings by pasting it into the
/system-adminpage of your localhost:
- All metadata and files sent to W&B Server are stored in the
/voldirectory. If you do not mount a persistent volume at this location all data will be lost when the docker process dies.
- You can store metadata in an external MySQL database and files in an external storage bucket.
- The underlying file store should be resizable. Alerts should be put in place to let you know once minimum storage thresholds are crossed to resize the underlying file system.
- For enterprise trials, we recommend at least 100GB free space in the underlying volume for non-image/video/audio heavy workloads.
When a Kubernetes instance is stopped, the wandb application bundles all the user account data into a tarball and uploads it to the S3 object store. On restarting the instance and providing the
BUCKETenvironment variable, wandb pulls that previously uploaded tarball and loads the user account info into the newly started Kubernetes deployment.
Wandb persists instance settings in the external bucket when it's configured. We also persist certificates, and secrets in the bucket but should be moving those into proper secret stores or at least adding a layer of encryption. When an external object store is enabled, strong access controls should be enforced as it will contain all users data