Skip to main content



Metaflow is a framework created by Netflix for creating and running ML workflows.

This integration lets users apply decorators to Metaflow steps and flows to automatically log parameters and artifacts to W&B.

  • Decorating a step will enable or disable logging for certain types within that step.
  • Decorating the flow will enable or disable logging for every step in the flow.


Install W&B and login

!pip install -Uqqq metaflow fastcore wandb

import wandb

Decorate your flows and steps

Decorating a step will enable or disable logging for certain types within that Step.

In this example, all datasets and models in start will be logged

from wandb.integration.metaflow import wandb_log

class WandbExampleFlow(FlowSpec):
@wandb_log(datasets=True, models=True, settings=wandb.Settings(...))
def start(self):
self.raw_df = pd.read_csv(...). # pd.DataFrame -> upload as dataset
self.model_file = torch.load(...) # nn.Module -> upload as model

Where is my data? Can I access it programmatically?

You can access the information we've captured in three ways: inside the original Python process being logged using the wandb client library, via the web app UI, or programmatically using our Public API. Parameters are saved to W&B's config and can be found in the Overview tab. datasets, models, and others are saved to W&B Artifacts and can be found in the Artifacts tab. Base python types are saved to W&B's summary dict and can be found in the Overview tab. See our guide to the Public API for details on using the API to get this information programmatically from outside .

Here's a cheatsheet:

DataClient libraryUI
Parameter(...)wandb.configOverview tab, Config
datasets, models, otherswandb.use_artifact("{var_name}:latest")Artifacts tab
Base Python types (dict, list, str, etc.)wandb.summaryOverview tab, Summary

wandb_log kwargs

  • True: Log instance variables that are a dataset
  • False
  • True: Log instance variables that are a model
  • False
  • True: Log anything else that is serializable as a pickle
  • False
  • wandb.Settings(...): Specify your own wandb settings for this step or flow
  • None: Equivalent to passing wandb.Settings()

By default, if:

  • settings.run_group is None, it will be set to {flow_name}/{run_id}
  • settings.run_job_type is None, it will be set to {run_job_type}/{step_name}

Frequently Asked Questions

What exactly do you log? Do you log all instance and local variables?

wandb_log only logs instance variables. Local variables are NEVER logged. This is useful to avoid logging unnecessary data.

Which data types get logged?

We currently support these types:

Logging SettingType
default (always on)
  • dict, list, set, str, int, float, bool
  • pd.DataFrame
  • pathlib.Path
  • nn.Module
  • sklearn.base.BaseEstimator

Examples of logging behavior

Kind of VariablebehaviorExampleData Type
InstanceLogged if datasets=Trueself.dfpd.DataFrame
InstanceNot logged if datasets=Falseself.dfpd.DataFrame
LocalNever loggedaccuracyfloat
LocalNever loggeddfpd.DataFrame

Does this track artifact lineage?

Yes! If you have an artifact that is an output of step A and an input to step B, we automatically construct the lineage DAG for you.

For an example of this behavior, please see this notebook and its corresponding W&B Artifacts page

Was this page helpful?👍👎