W&B integrates with Amazon SageMaker, automatically reading hyperparameters, grouping distributed runs, and resuming runs from checkpoints.
W&B looks for a file named secrets.env
relative to the training script and loads them into the environment when wandb.init()
is called. You can generate a secrets.env
file by calling wandb.sagemaker_auth(path="source_dir")
in the script you use to launch your experiments. Be sure to add this file to your .gitignore
!
If you're using one of SageMakers preconfigured estimators you need to add a requirements.txt
to your source directory that includes wandb
wandb
If you're using an estimator that's running Python 2, you'll need to install psutil directly from a wheel before installing wandb:
https://wheels.galaxyproject.org/packages/psutil-5.4.8-cp27-cp27mu-manylinux1_x86_64.whlwandb
A complete example is available on GitHub and you can read more on our blog.