Hydra is an open-source Python framework that simplifies the development of research and other complex applications. The key feature is the ability to dynamically create a hierarchical configuration by composition and override it through config files and the command line.
You can continue to use Hydra for configuration management while taking advantage of the power of W&B.
Track your metrics as normal with
wandb.log . Here,
wandb.project are defined within a hydra configuration file.
run = wandb.init(entity=cfg.wandb.entity, project=cfg.wandb.project)
Hydra uses omegaconf as the default way to interface with configuration dictionaries.
OmegaConf's dictionary are not a subclass of primitive dictionaries so directly passing Hydra's
wandb.config leads to unexpected results on the dashboard. It's necessary to convert
omegaconf.DictConfig to the primitive
dict type before passing to
wandb.config = omegaconf.OmegaConf.to_container(
cfg, resolve=True, throw_on_missing=True
model = Model(**wandb.config.model.configs)
If your process hangs when started, this may be caused by this known issue. To solve this, try to changing wandb's multiprocessing protocol either by adding an extra settings parameter to `wandb.init` as:
or by setting a global environment variable from your shell:
$ export WANDB_START_METHOD=thread
W&B Sweeps is a highly scalable hyperparameter search platform, which provides interesting insights and visualization about W&B experiments with minimal requirements code real-estate. Sweeps integrates seamlessly with Hydra projects with no-coding requirements. The only thing needed is a configuration file describing the various parameters to sweep over as normal.
A simple example
sweep.yaml file would be:
values: [mnist, cifar10]
Invoke the sweep with:
wandb sweep sweep.yaml\
Once you call this, W&B automatically creates a sweep inside your project and returns a
wandb agent command for you to run on each machine you want to run your sweep.
Passing parameters not present in Hydra defaults
Hydra supports passing extra parameters through the command line which aren't present in the default configuration file, by using a
+ before command. For example, you can pass an extra parameter with some value by simply calling:
$ python program.py +experiment=some_experiment
You cannot sweep over such
+ configurations similar to what one does while configuring Hydra Experiments. To work around this, you can initialize the experiment parameter with a default empty file and use W&B Sweep to override those empty configs on each call. For more information, read this W&B Report.