W&B Core

• 1: Artifacts
• 1.1: Create an artifact
• 1.2: Download and use artifacts
• 1.3: Update an artifact
• 1.4: Create an artifact alias
• 1.5: Create an artifact version
• 1.6: Track external files
• 1.7: Manage data
• 1.7.1: Delete an artifact
• 1.7.2: Manage artifact data retention
• 1.7.3: Manage artifact storage and memory allocation
• 1.8: Explore artifact graphs
• 1.9: Artifact data privacy and compliance
• 1.10: Tutorial: Create, track, and use a dataset artifact
• 2: Secrets
• 3: Registry
• 3.1: Registry types
• 3.2: Create a custom registry
• 3.3: Configure registry access
• 3.4: Create a collection
• 3.5: Link an artifact version to a registry
• 3.6: Download an artifact from a registry
• 3.7: Find registry items
• 3.8: Organize versions with tags
• 3.9: Annotate collections
• 3.10: Create and view lineage maps
• 3.11: Migrate from legacy Model Registry
• 3.12: Model registry
• 3.12.1: Tutorial: Use W&B for model management
• 3.12.2: Model Registry Terms and Concepts
• 3.12.3: Track a model
• 3.12.4: Create a registered model
• 3.12.5: Link a model version
• 3.12.6: Organize models
• 3.12.7: Create model lineage map
• 3.12.8: Document machine learning model
• 3.12.9: Download a model version
• 3.12.10: Create alerts and notifications
• 3.12.11: Manage data governance and access control
• 4: Reports
• 4.1: Create a report
• 4.2: Edit a report
• 4.3: Collaborate on reports
• 4.4: Clone and export reports
• 4.5: Embed a report
• 4.6: Compare runs across projects
• 4.7: Example reports
• 5: Automations
• 5.1: Create an automation
• 5.1.1: Create a Slack automation
• 5.1.2: Create a webhook automation
• 5.2: Automation events and scopes

W&B Core is the foundational framework supporting W&B Models and W&B Weave, and is itself supported by the W&B Platform.

W&B Core provides capabilities across the entire ML lifecycle. With W&B Core, you can:

• Version and manage ML pipelines with full lineage tracing for easy auditing and reproducibility.
• Explore and evaluate data and metrics using interactive, configurable visualizations.
• Document and share insights across the entire organization by generating live reports in digestible, visual formats that are easily understood by non-technical stakeholders.
• Query and create visualizations of your data that serve your custom needs.
• Protect sensitive strings using secrets.
• Configure automations that trigger key workflows for model CI/CD.

1 - Artifacts

Use W&B Artifacts to track and version data as the inputs and outputs of your W&B Runs. For example, a model training run might take in a dataset as input and produce a trained model as output. You can log hyperparameters, metadata, and metrics to a run, and you can use an artifact to log, track, and version the dataset used to train the model as input and another artifact for the resulting model checkpoints as output.

Use cases

You can use artifacts throughout your entire ML workflow as inputs and outputs of runs. You can use datasets, models, or even other artifacts as inputs for processing.

Create an artifact

Create an artifact with four lines of code:

1. Create a W&B run.
2. Create an artifact object with the wandb.Artifact API.
3. Add one or more files, such as a model file or dataset, to your artifact object.
4. Log your artifact to W&B.

For example, the proceeding code snippet shows how to log a file called dataset.h5 to an artifact called example_artifact:

import wandb

run = wandb.init(project="artifacts-example", job_type="add-dataset")
artifact = wandb.Artifact(name="example_artifact", type="dataset")
artifact.add_file(local_path="./dataset.h5", name="training_dataset")
artifact.save()

# Logs the artifact version "my_data" as a dataset with data from dataset.h5

Download an artifact

Indicate the artifact you want to mark as input to your run with the use_artifact method.

Following the preceding code snippet, this next code block shows how to use the training_dataset artifact:

artifact = run.use_artifact(
    "training_dataset:latest"
)  # returns a run object using the "my_data" artifact

This returns an artifact object.

Next, use the returned object to download all contents of the artifact:

datadir = (
    artifact.download()
)  # downloads the full `my_data` artifact to the default directory.

Next steps

• Learn how to version and update artifacts.
• Learn how to trigger downstream workflows or notify a Slack channel in response to changes to your artifacts with automations.
• Learn about the registry, a space that houses trained models.
• Explore the Python SDK and CLI reference guides.

1.1 - Create an artifact

Use the W&B Python SDK to construct artifacts from W&B Runs. You can add files, directories, URIs, and files from parallel runs to artifacts. After you add a file to an artifact, save the artifact to the W&B Server or your own private server.

For information on how to track external files, such as files stored in Amazon S3, see the Track external files page.

How to construct an artifact

Construct a W&B Artifact in three steps:

1. Create an artifact Python object with wandb.Artifact()

Initialize the wandb.Artifact() class to create an artifact object. Specify the following parameters:

• Name: Specify a name for your artifact. The name should be unique, descriptive, and easy to remember. Use an artifacts name to both: identify the artifact in the W&B App UI and when you want to use that artifact.
• Type: Provide a type. The type should be simple, descriptive and correspond to a single step of your machine learning pipeline. Common artifact types include 'dataset' or 'model'.

You can optionally provide a description and metadata when you initialize an artifact object. For more information on available attributes and parameters, see wandb.Artifact Class definition in the Python SDK Reference Guide.

The proceeding example demonstrates how to create a dataset artifact:

import wandb

artifact = wandb.Artifact(name="<replace>", type="<replace>")

Replace the string arguments in the preceding code snippet with your own name and type.

2. Add one more files to the artifact

Add files, directories, external URI references (such as Amazon S3) and more with artifact methods. For example, to add a single text file, use the add_file method:

artifact.add_file(local_path="hello_world.txt", name="optional-name")

You can also add multiple files with the add_dir method. For more information on how to add files, see Update an artifact.

3. Save your artifact to the W&B server

Finally, save your artifact to the W&B server. Artifacts are associated with a run. Therefore, use a run objects log_artifact() method to save the artifact.

# Create a W&B Run. Replace 'job-type'.
run = wandb.init(project="artifacts-example", job_type="job-type")

run.log_artifact(artifact)

You can optionally construct an artifact outside of a W&B run. For more information, see Track external files.

for i in range(10):
    a = wandb.Artifact(
        "race",
        type="dataset",
        metadata={
            "index": i,
        },
    )
    # ... add files to artifact a ...
    run.log_artifact(a)

Add files to an artifact

The following sections demonstrate how to construct artifacts with different file types and from parallel runs.

For the following examples, assume you have a project directory with multiple files and a directory structure:

project-directory
|-- images
|   |-- cat.png
|   +-- dog.png
|-- checkpoints
|   +-- model.h5
+-- model.h5

Add a single file

The proceeding code snippet demonstrates how to add a single, local file to your artifact:

# Add a single file
artifact.add_file(local_path="path/file.format")

For example, suppose you had a file called 'file.txt' in your working local directory.

artifact.add_file("path/file.txt")  # Added as `file.txt'

The artifact now has the following content:

file.txt

Optionally, pass the desired path within the artifact for the name parameter.

artifact.add_file(local_path="path/file.format", name="new/path/file.format")

The artifact is stored as:

new/path/file.txt

Add multiple files

The proceeding code snippet demonstrates how to add an entire, local directory to your artifact:

# Recursively add a directory
artifact.add_dir(local_path="path/file.format", name="optional-prefix")

The proceeding API calls produce the proceeding artifact content:

Add a URI reference

Artifacts track checksums and other information for reproducibility if the URI has a scheme that W&B library knows how to handle.

Add an external URI reference to an artifact with the add_reference method. Replace the 'uri' string with your own URI. Optionally pass the desired path within the artifact for the name parameter.

# Add a URI reference
artifact.add_reference(uri="uri", name="optional-name")

Artifacts currently support the following URI schemes:

• http(s)://: A path to a file accessible over HTTP. The artifact will track checksums in the form of etags and size metadata if the HTTP server supports the ETag and Content-Length response headers.
• s3://: A path to an object or object prefix in S3. The artifact will track checksums and versioning information (if the bucket has object versioning enabled) for the referenced objects. Object prefixes are expanded to include the objects under the prefix, up to a maximum of 10,000 objects.
• gs://: A path to an object or object prefix in GCS. The artifact will track checksums and versioning information (if the bucket has object versioning enabled) for the referenced objects. Object prefixes are expanded to include the objects under the prefix, up to a maximum of 10,000 objects.

The proceeding API calls will produce the proceeding artifacts:

Add files to artifacts from parallel runs

For large datasets or distributed training, multiple parallel runs might need to contribute to a single artifact.

import wandb
import time

# We will use ray to launch our runs in parallel
# for demonstration purposes. You can orchestrate
# your parallel runs however you want.
import ray

ray.init()

artifact_type = "dataset"
artifact_name = "parallel-artifact"
table_name = "distributed_table"
parts_path = "parts"
num_parallel = 5

# Each batch of parallel writers should have its own
# unique group name.
group_name = "writer-group-{}".format(round(time.time()))


@ray.remote
def train(i):
    """
    Our writer job. Each writer will add one image to the artifact.
    """
    with wandb.init(group=group_name) as run:
        artifact = wandb.Artifact(name=artifact_name, type=artifact_type)

        # Add data to a wandb table. In this case we use example data
        table = wandb.Table(columns=["a", "b", "c"], data=[[i, i * 2, 2**i]])

        # Add the table to folder in the artifact
        artifact.add(table, "{}/table_{}".format(parts_path, i))

        # Upserting the artifact creates or appends data to the artifact
        run.upsert_artifact(artifact)


# Launch your runs in parallel
result_ids = [train.remote(i) for i in range(num_parallel)]

# Join on all the writers to make sure their files have
# been added before finishing the artifact.
ray.get(result_ids)

# Once all the writers are finished, finish the artifact
# to mark it ready.
with wandb.init(group=group_name) as run:
    artifact = wandb.Artifact(artifact_name, type=artifact_type)

    # Create a "PartitionTable" pointing to the folder of tables
    # and add it to the artifact.
    artifact.add(wandb.data_types.PartitionedTable(parts_path), table_name)

    # Finish artifact finalizes the artifact, disallowing future "upserts"
    # to this version.
    run.finish_artifact(artifact)

1.2 - Download and use artifacts

Download and use an artifact that is already stored on the W&B server or construct an artifact object and pass it in to for de-duplication as necessary.

Download and use an artifact stored on W&B

Download and use an artifact stored in W&B either inside or outside of a W&B Run. Use the Public API (wandb.Api) to export (or update data) already saved in W&B. For more information, see the W&B Public API Reference guide.

• During a run
• Outside of a run
• W&B CLI

import wandb

run = wandb.init(project="<example>", job_type="<job-type>")

artifact = run.use_artifact("bike-dataset:latest")

datadir = artifact.download()

path = artifact.get_path(name)

import wandb

api = wandb.Api()
artifact = api.artifact("entity/project/artifact:alias")

artifact.download()

$ wandb artifact get project/artifact:alias --root mnist/

Partially download an artifact

You can optionally download part of an artifact based on a prefix. Using the path_prefix parameter, you can download a single file or the content of a sub-folder.

artifact = run.use_artifact("bike-dataset:latest")

artifact.download(path_prefix="bike.png") # downloads only bike.png

Alternatively, you can download files from a certain directory:

artifact.download(path_prefix="images/bikes/") # downloads files in the images/bikes directory

Use an artifact from a different project

Specify the name of artifact along with its project name to reference an artifact. You can also reference artifacts across entities by specifying the name of the artifact with its entity name.

The following code example demonstrates how to query an artifact from another project as input to the current W&B run.

import wandb

run = wandb.init(project="<example>", job_type="<job-type>")
# Query W&B for an artifact from another project and mark it
# as an input to this run.
artifact = run.use_artifact("my-project/artifact:alias")

# Use an artifact from another entity and mark it as an input
# to this run.
artifact = run.use_artifact("my-entity/my-project/artifact:alias")

Construct and use an artifact simultaneously

Simultaneously construct and use an artifact. Create an artifact object and pass it to use_artifact. This creates an artifact in W&B if it does not exist yet. The use_artifact API is idempotent, so you can call it as many times as you like.

import wandb

artifact = wandb.Artifact("reference model")
artifact.add_file("model.h5")
run.use_artifact(artifact)

For more information about constructing an artifact, see Construct an artifact.

1.3 - Update an artifact

Pass desired values to update the description, metadata, and alias of an artifact. Call the save() method to update the artifact on the W&B servers. You can update an artifact during a W&B Run or outside of a Run.

Use the W&B Public API (wandb.Api) to update an artifact outside of a run. Use the Artifact API (wandb.Artifact) to update an artifact during a run.

• During a run
• Outside of a run
• With collections

import wandb

run = wandb.init(project="<example>")
artifact = run.use_artifact("<artifact-name>:<alias>")
artifact.description = "<description>"
artifact.save()

import wandb

api = wandb.Api()

artifact = api.artifact("entity/project/artifact:alias")

# Update the description
artifact.description = "My new description"

# Selectively update metadata keys
artifact.metadata["oldKey"] = "new value"

# Replace the metadata entirely
artifact.metadata = {"newKey": "new value"}

# Add an alias
artifact.aliases.append("best")

# Remove an alias
artifact.aliases.remove("latest")

# Completely replace the aliases
artifact.aliases = ["replaced"]

# Persist all artifact modifications
artifact.save()

import wandb
run = wandb.init(project="<example>")
api = wandb.Api()
artifact = api.artifact_collection(type="<type-name>", collection="<collection-name>")
artifact.name = "<new-collection-name>"
artifact.description = "<This is where you'd describe the purpose of your collection.>"
artifact.save()

1.4 - Create an artifact alias

Use aliases as pointers to specific versions. By default, Run.log_artifact adds the latest alias to the logged version.

An artifact version v0 is created and attached to your artifact when you log an artifact for the first time. W&B checksums the contents when you log again to the same artifact. If the artifact changed, W&B saves a new version v1.

For example, if you want your training script to pull the most recent version of a dataset, specify latest when you use that artifact. The proceeding code example demonstrates how to download a recent dataset artifact named bike-dataset that has an alias, latest:

import wandb

run = wandb.init(project="<example-project>")

artifact = run.use_artifact("bike-dataset:latest")

artifact.download()

You can also apply a custom alias to an artifact version. For example, if you want to mark that model checkpoint is the best on the metric AP-50, you could add the string 'best-ap50' as an alias when you log the model artifact.

artifact = wandb.Artifact("run-3nq3ctyy-bike-model", type="model")
artifact.add_file("model.h5")
run.log_artifact(artifact, aliases=["latest", "best-ap50"])

1.5 - Create an artifact version

Create a new artifact version with a single run or collaboratively with distributed runs. You can optionally create a new artifact version from a previous version known as an incremental artifact.

Create new artifact versions from scratch

There are two ways to create a new artifact version: from a single run and from distributed runs. They are defined as follows:

• Single run: A single run provides all the data for a new version. This is the most common case and is best suited when the run fully recreates the needed data. For example: outputting saved models or model predictions in a table for analysis.
• Distributed runs: A set of runs collectively provides all the data for a new version. This is best suited for distributed jobs which have multiple runs generating data, often in parallel. For example: evaluating a model in a distributed manner, and outputting the predictions.

W&B will create a new artifact and assign it a v0 alias if you pass a name to the wandb.Artifact API that does not exist in your project. W&B checksums the contents when you log again to the same artifact. If the artifact changed, W&B saves a new version v1.

W&B will retrieve an existing artifact if you pass a name and artifact type to the wandb.Artifact API that matches an existing artifact in your project. The retrieved artifact will have a version greater than 1.

Single run

Log a new version of an Artifact with a single run that produces all the files in the artifact. This case occurs when a single run produces all the files in the artifact.

Based on your use case, select one of the tabs below to create a new artifact version inside or outside of a run:

• Inside a run
• Outside of a run

with wandb.init() as run:
    artifact = wandb.Artifact("artifact_name", "artifact_type")

    # Add Files and Assets to the artifact using
    # `.add`, `.add_file`, `.add_dir`, and `.add_reference`
    artifact.add_file("image1.png")
    run.log_artifact(artifact)

artifact = wandb.Artifact("artifact_name", "artifact_type")
# Add Files and Assets to the artifact using
# `.add`, `.add_file`, `.add_dir`, and `.add_reference`
artifact.add_file("image1.png")
artifact.save()

Distributed runs

Allow a collection of runs to collaborate on a version before committing it. This is in contrast to single run mode described above where one run provides all the data for a new version.

Consider the following example. Different runs (labelled below as Run 1, Run 2, and Run 3) add a different image file to the same artifact with upsert_artifact.

Run 1:

with wandb.init() as run:
    artifact = wandb.Artifact("artifact_name", "artifact_type")
    # Add Files and Assets to the artifact using
    # `.add`, `.add_file`, `.add_dir`, and `.add_reference`
    artifact.add_file("image1.png")
    run.upsert_artifact(artifact, distributed_id="my_dist_artifact")

Run 2:

with wandb.init() as run:
    artifact = wandb.Artifact("artifact_name", "artifact_type")
    # Add Files and Assets to the artifact using
    # `.add`, `.add_file`, `.add_dir`, and `.add_reference`
    artifact.add_file("image2.png")
    run.upsert_artifact(artifact, distributed_id="my_dist_artifact")

Run 3

Must run after Run 1 and Run 2 complete. The Run that calls finish_artifact can include files in the artifact, but does not need to.

with wandb.init() as run:
    artifact = wandb.Artifact("artifact_name", "artifact_type")
    # Add Files and Assets to the artifact
    # `.add`, `.add_file`, `.add_dir`, and `.add_reference`
    artifact.add_file("image3.png")
    run.finish_artifact(artifact, distributed_id="my_dist_artifact")

Create a new artifact version from an existing version

Add, modify, or remove a subset of files from a previous artifact version without the need to re-index the files that didn’t change. Adding, modifying, or removing a subset of files from a previous artifact version creates a new artifact version known as an incremental artifact.

Here are some scenarios for each type of incremental change you might encounter:

• add: you periodically add a new subset of files to a dataset after collecting a new batch.
• remove: you discovered several duplicate files and want to remove them from your artifact.
• update: you corrected annotations for a subset of files and want to replace the old files with the correct ones.

You could create an artifact from scratch to perform the same function as an incremental artifact. However, when you create an artifact from scratch, you will need to have all the contents of your artifact on your local disk. When making an incremental change, you can add, remove, or modify a single file without changing the files from a previous artifact version.

Follow the procedure below to incrementally change an artifact:

1. Obtain the artifact version you want to perform an incremental change on:

• Inside a run
• Outside of a run

saved_artifact = run.use_artifact("my_artifact:latest")

client = wandb.Api()
saved_artifact = client.artifact("my_artifact:latest")

2. Create a draft with:

draft_artifact = saved_artifact.new_draft()

3. Perform any incremental changes you want to see in the next version. You can either add, remove, or modify an existing entry.

Select one of the tabs for an example on how to perform each of these changes:

• Add
• Remove
• Modify

draft_artifact.add_file("file_to_add.txt")

draft_artifact.remove("file_to_remove.txt")

draft_artifact.remove("modified_file.txt")
draft_artifact.add_file("modified_file.txt")

4. Lastly, log or save your changes. The following tabs show you how to save your changes inside and outside of a W&B run. Select the tab that is appropriate for your use case:

• Inside a run
• Outside of a run

run.log_artifact(draft_artifact)

draft_artifact.save()

Putting it all together, the code examples above look like:

• Inside a run
• Outside of a run

with wandb.init(job_type="modify dataset") as run:
    saved_artifact = run.use_artifact(
        "my_artifact:latest"
    )  # fetch artifact and input it into your run
    draft_artifact = saved_artifact.new_draft()  # create a draft version

    # modify a subset of files in the draft version
    draft_artifact.add_file("file_to_add.txt")
    draft_artifact.remove("dir_to_remove/")
    run.log_artifact(
        artifact
    )  # log your changes to create a new version and mark it as output to your run

client = wandb.Api()
saved_artifact = client.artifact("my_artifact:latest")  # load your artifact
draft_artifact = saved_artifact.new_draft()  # create a draft version

# modify a subset of files in the draft version
draft_artifact.remove("deleted_file.txt")
draft_artifact.add_file("modified_file.txt")
draft_artifact.save()  # commit changes to the draft

1.6 - Track external files

Use reference artifacts to track files saved outside the W&B system, for example in an Amazon S3 bucket, GCS bucket, Azure blob, HTTP file server, or even an NFS share. Log artifacts outside of a W&B Run with the W&B CLI.

Log artifacts outside of runs

W&B creates a run when you log an artifact outside of a run. Each artifact belongs to a run, which in turn belongs to a project. An artifact (version) also belongs to a collection, and has a type.

Use the wandb artifact put command to upload an artifact to the W&B server outside of a W&B run. Provide the name of the project you want the artifact to belong to along with the name of the artifact (project/artifact_name).Optionally provide the type (TYPE). Replace PATH in the code snippet below with the file path of the artifact you want to upload.

$ wandb artifact put --name project/artifact_name --type TYPE PATH

W&B will create a new project if a the project you specify does not exist. For information on how to download an artifact, see Download and use artifacts.

Track artifacts outside of W&B

Use W&B Artifacts for dataset versioning and model lineage, and use reference artifacts to track files saved outside the W&B server. In this mode an artifact only stores metadata about the files, such as URLs, size, and checksums. The underlying data never leaves your system. See the Quick start for information on how to save files and directories to W&B servers instead.

The following describes how to construct reference artifacts and how to best incorporate them into your workflows.

Amazon S3 / GCS / Azure Blob Storage References

Use W&B Artifacts for dataset and model versioning to track references in cloud storage buckets. With artifact references, seamlessly layer tracking on top of your buckets with no modifications to your existing storage layout.

Artifacts abstract away the underlying cloud storage vendor (such AWS, GCP or Azure). Information described in the proceeding section apply uniformly to Amazon S3, Google Cloud Storage and Azure Blob Storage.

Assume we have a bucket with the following structure:

s3://my-bucket
+-- datasets/
|		+-- mnist/
+-- models/
		+-- cnn/

Under mnist/ we have our dataset, a collection of images. Lets track it with an artifact:

import wandb

run = wandb.init()
artifact = wandb.Artifact("mnist", type="dataset")
artifact.add_reference("s3://my-bucket/datasets/mnist")
run.log_artifact(artifact)

Our new reference artifact mnist:latest looks and behaves similarly to a regular artifact. The only difference is that the artifact only consists of metadata about the S3/GCS/Azure object such as its ETag, size, and version ID (if object versioning is enabled on the bucket).

W&B will use the default mechanism to look for credentials based on the cloud provider you use. Read the documentation from your cloud provider to learn more about the credentials used:

For AWS, if the bucket is not located in the configured user’s default region, you must set the AWS_REGION environment variable to match the bucket region.

Interact with this artifact similarly to a normal artifact. In the App UI, you can look through the contents of the reference artifact using the file browser, explore the full dependency graph, and scan through the versioned history of your artifact.

Download a reference artifact

import wandb

run = wandb.init()
artifact = run.use_artifact("mnist:latest", type="dataset")
artifact_dir = artifact.download()

W&B will use the metadata recorded when the artifact was logged to retrieve the files from the underlying bucket when it downloads a reference artifact. If your bucket has object versioning enabled, W&B will retrieve the object version corresponding to the state of the file at the time an artifact was logged. This means that as you evolve the contents of your bucket, you can still point to the exact iteration of your data a given model was trained on since the artifact serves as a snapshot of your bucket at the time of training.

Tying it together

The following code example demonstrates a simple workflow you can use to track a dataset in Amazon S3, GCS, or Azure that feeds into a training job:

import wandb

run = wandb.init()

artifact = wandb.Artifact("mnist", type="dataset")
artifact.add_reference("s3://my-bucket/datasets/mnist")

# Track the artifact and mark it as an input to
# this run in one swoop. A new artifact version
# is only logged if the files in the bucket changed.
run.use_artifact(artifact)

artifact_dir = artifact.download()

# Perform training here...

To track models, we can log the model artifact after the training script uploads the model files to the bucket:

import boto3
import wandb

run = wandb.init()

# Training here...

s3_client = boto3.client("s3")
s3_client.upload_file("my_model.h5", "my-bucket", "models/cnn/my_model.h5")

model_artifact = wandb.Artifact("cnn", type="model")
model_artifact.add_reference("s3://my-bucket/models/cnn/")
run.log_artifact(model_artifact)

Filesystem References

Another common pattern for fast access to datasets is to expose an NFS mount point to a remote filesystem on all machines running training jobs. This can be an even simpler solution than a cloud storage bucket because from the perspective of the training script, the files look just like they are sitting on your local filesystem. Luckily, that ease of use extends into using Artifacts to track references to file systems, whether they are mounted or not.

Assume we have a filesystem mounted at /mount with the following structure:

mount
+-- datasets/
|		+-- mnist/
+-- models/
		+-- cnn/

Under mnist/ we have our dataset, a collection of images. Let’s track it with an artifact:

import wandb

run = wandb.init()
artifact = wandb.Artifact("mnist", type="dataset")
artifact.add_reference("file:///mount/datasets/mnist/")
run.log_artifact(artifact)

By default, W&B imposes a 10,000 file limit when adding a reference to a directory. You can adjust this limit by specifying max_objects= in calls to add_reference.

Note the triple slash in the URL. The first component is the file:// prefix that denotes the use of filesystem references. The second is the path to our dataset, /mount/datasets/mnist/.

The resulting artifact mnist:latest looks and acts just like a regular artifact. The only difference is that the artifact only consists of metadata about the files, such as their sizes and MD5 checksums. The files themselves never leave your system.

You can interact with this artifact just as you would a normal artifact. In the UI, you can browse the contents of the reference artifact using the file browser, explore the full dependency graph, and scan through the versioned history of your artifact. However, the UI will not be able to render rich media such as images, audio, etc. as the data itself is not contained within the artifact.

Downloading a reference artifact is simple:

import wandb

run = wandb.init()
artifact = run.use_artifact("entity/project/mnist:latest", type="dataset")
artifact_dir = artifact.download()

For filesystem references, a download() operation copies the files from the referenced paths to construct the artifact directory. In the above example, the contents of /mount/datasets/mnist will be copied into the directory artifacts/mnist:v0/. If an artifact contains a reference to a file that was overwritten, then download() will throw an error as the artifact can no longer be reconstructed.

Putting everything together, here’s a simple workflow you can use to track a dataset under a mounted filesystem that feeds into a training job:

import wandb

run = wandb.init()

artifact = wandb.Artifact("mnist", type="dataset")
artifact.add_reference("file:///mount/datasets/mnist/")

# Track the artifact and mark it as an input to
# this run in one swoop. A new artifact version
# is only logged if the files under the directory
# changed.
run.use_artifact(artifact)

artifact_dir = artifact.download()

# Perform training here...

To track models, we can log the model artifact after the training script writes the model files to the mount point:

import wandb

run = wandb.init()

# Training here...

# Write model to disk

model_artifact = wandb.Artifact("cnn", type="model")
model_artifact.add_reference("file:///mount/cnn/my_model.h5")
run.log_artifact(model_artifact)

1.7 - Manage data

1.7.1 - Delete an artifact

Delete artifacts interactively with the App UI or programmatically with the W&B SDK. When you delete an artifact, W&B marks that artifact as a soft-delete. In other words, the artifact is marked for deletion but files are not immediately deleted from storage.

The contents of the artifact remain as a soft-delete, or pending deletion state, until a regularly run garbage collection process reviews all artifacts marked for deletion. The garbage collection process deletes associated files from storage if the artifact and its associated files are not used by a previous or subsequent artifact versions.

The sections in this page describe how to delete specific artifact versions, how to delete an artifact collection, how to delete artifacts with and without aliases, and more. You can schedule when artifacts are deleted from W&B with TTL policies. For more information, see Manage data retention with Artifact TTL policy.

Delete an artifact version

To delete an artifact version:

1. Select the name of the artifact. This will expand the artifact view and list all the artifact versions associated with that artifact.
2. From the list of artifacts, select the artifact version you want to delete.
3. On the right hand side of the workspace, select the kebab dropdown.
4. Choose Delete.

An artifact version can also be deleted programatically via the delete() method. See the examples below.

Delete multiple artifact versions with aliases

The following code example demonstrates how to delete artifacts that have aliases associated with them. Provide the entity, project name, and run ID that created the artifacts.

import wandb

run = api.run("entity/project/run_id")

for artifact in run.logged_artifacts():
    artifact.delete()

Set the delete_aliases parameter to the boolean value, True to delete aliases if the artifact has one or more aliases.

import wandb

run = api.run("entity/project/run_id")

for artifact in run.logged_artifacts():
    # Set delete_aliases=True in order to delete
    # artifacts with one more aliases
    artifact.delete(delete_aliases=True)

Delete multiple artifact versions with a specific alias

The proceeding code demonstrates how to delete multiple artifact versions that have a specific alias. Provide the entity, project name, and run ID that created the artifacts. Replace the deletion logic with your own:

import wandb

runs = api.run("entity/project_name/run_id")

# Delete artifact ith alias 'v3' and 'v4
for artifact_version in runs.logged_artifacts():
    # Replace with your own deletion logic.
    if artifact_version.name[-2:] == "v3" or artifact_version.name[-2:] == "v4":
        artifact.delete(delete_aliases=True)

Delete all versions of an artifact that do not have an alias

The following code snippet demonstrates how to delete all versions of an artifact that do not have an alias. Provide the name of the project and entity for the project and entity keys in wandb.Api, respectively. Replace the <> with the name of your artifact:

import wandb

# Provide your entity and a project name when you
# use wandb.Api methods.
api = wandb.Api(overrides={"project": "project", "entity": "entity"})

artifact_type, artifact_name = "<>"  # provide type and name
for v in api.artifact_versions(artifact_type, artifact_name):
    # Clean up versions that don't have an alias such as 'latest'.
    # NOTE: You can put whatever deletion logic you want here.
    if len(v.aliases) == 0:
        v.delete()

Delete an artifact collection

To delete an artifact collection:

1. Navigate to the artifact collection you want to delete and hover over it.
2. Select the kebab dropdown next to the artifact collection name.
3. Choose Delete.

You can also delete artifact collection programmatically with the delete() method. Provide the name of the project and entity for the project and entity keys in wandb.Api, respectively:

import wandb

# Provide your entity and a project name when you
# use wandb.Api methods.
api = wandb.Api(overrides={"project": "project", "entity": "entity"})
collection = api.artifact_collection(
    "<artifact_type>", "entity/project/artifact_collection_name"
)
collection.delete()

How to enable garbage collection based on how W&B is hosted

Garbage collection is enabled by default if you use W&B’s shared cloud. Based on how you host W&B, you might need to take additional steps to enable garbage collection, this includes:

• Set the GORILLA_ARTIFACT_GC_ENABLED environment variable to true: GORILLA_ARTIFACT_GC_ENABLED=true
• Enable bucket versioning if you use AWS, GCP or any other storage provider such as Minio. If you use Azure, enable soft deletion.
  Soft deletion in Azure is equivalent to bucket versioning in other storage providers.

The following table describes how to satisfy requirements to enable garbage collection based on your deployment type.

The X indicates you must satisfy the requirement:

1.7.2 - Manage artifact data retention

Schedule when artifacts are deleted from W&B with W&B Artifact time-to-live (TTL) policy. When you delete an artifact, W&B marks that artifact as a soft-delete. In other words, the artifact is marked for deletion but files are not immediately deleted from storage. For more information on how W&B deletes artifacts, see the Delete artifacts page.

Check out this video tutorial to learn how to manage data retention with Artifacts TTL in the W&B App.

Auto-generated Artifacts

Only user-generated artifacts can use TTL policies. Artifacts auto-generated by W&B cannot have TTL policies set for them.

The following Artifact types indicate an auto-generated Artifact:

• run_table
• code
• job
• Any Artifact type starting with: wandb-*

You can check an Artifact’s type on the W&B platform or programmatically:

import wandb

run = wandb.init(project="<my-project-name>")
artifact = run.use_artifact(artifact_or_name="<my-artifact-name>")
print(artifact.type)

Replace the values enclosed with <> with your own.

Define who can edit and set TTL policies

Define who can set and edit TTL policies within a team. You can either grant TTL permissions only to team admins, or you can grant both team admins and team members TTL permissions.

1. Navigate to your team’s profile page.
2. Select the Settings tab.
3. Navigate to the Artifacts time-to-live (TTL) section.
4. From the TTL permissions dropdown, select who can set and edit TTL policies.
5. Click on Review and save settings.
6. Confirm the changes and select Save settings.

Create a TTL policy

Set a TTL policy for an artifact either when you create the artifact or retroactively after the artifact is created.

For all the code snippets below, replace the content wrapped in <> with your information to use the code snippet.

Set a TTL policy when you create an artifact

Use the W&B Python SDK to define a TTL policy when you create an artifact. TTL policies are typically defined in days.

The steps are as follows:

1. Create an artifact.
2. Add content to the artifact such as files, a directory, or a reference.
3. Define a TTL time limit with the datetime.timedelta data type that is part of Python’s standard library.
4. Log the artifact.

The following code snippet demonstrates how to create an artifact and set a TTL policy.

import wandb
from datetime import timedelta

run = wandb.init(project="<my-project-name>", entity="<my-entity>")
artifact = wandb.Artifact(name="<artifact-name>", type="<type>")
artifact.add_file("<my_file>")

artifact.ttl = timedelta(days=30)  # Set TTL policy
run.log_artifact(artifact)

The preceding code snippet sets the TTL policy for the artifact to 30 days. In other words, W&B deletes the artifact after 30 days.

Set or edit a TTL policy after you create an artifact

Use the W&B App UI or the W&B Python SDK to define a TTL policy for an artifact that already exists.

• Python SDK
• W&B App

1. Fetch your artifact.
2. Pass in a time delta to the artifact’s ttl attribute.
3. Update the artifact with the save method.

The following code snippet shows how to set a TTL policy for an artifact:

import wandb
from datetime import timedelta

artifact = run.use_artifact("<my-entity/my-project/my-artifact:alias>")
artifact.ttl = timedelta(days=365 * 2)  # Delete in two years
artifact.save()

The preceding code example sets the TTL policy to two years.

1. Navigate to your W&B project in the W&B App UI.
2. Select the artifact icon on the left panel.
3. From the list of artifacts, expand the artifact type you
4. Select on the artifact version you want to edit the TTL policy for.
5. Click on the Version tab.
6. From the dropdown, select Edit TTL policy.
7. Within the modal that appears, select Custom from the TTL policy dropdown.
8. Within the TTL duration field, set the TTL policy in units of days.
9. Select the Update TTL button to save your changes.

Set default TTL policies for a team

Set a default TTL policy for your team. Default TTL policies apply to all existing and future artifacts based on their respective creation dates. Artifacts with existing version-level TTL policies are not affected by the team’s default TTL.

1. Navigate to your team’s profile page.
2. Select the Settings tab.
3. Navigate to the Artifacts time-to-live (TTL) section.
4. Click on the Set team’s default TTL policy.
5. Within the Duration field, set the TTL policy in units of days.
6. Click on Review and save settings. 7/ Confirm the changes and then select Save settings.

Set a TTL policy outside of a run

Use the public API to retrieve an artifact without fetching a run, and set the TTL policy. TTL policies are typically defined in days.

The following code sample shows how to fetch an artifact using the public API and set the TTL policy.

api = wandb.Api()

artifact = api.artifact("entity/project/artifact:alias")

artifact.ttl = timedelta(days=365)  # Delete in one year

artifact.save()

Deactivate a TTL policy

Use the W&B Python SDK or W&B App UI to deactivate a TTL policy for a specific artifact version.

• Python SDK
• W&B App

1. Fetch your artifact.
2. Set the artifact’s ttl attribute to None.
3. Update the artifact with the save method.

The following code snippet shows how to turn off a TTL policy for an artifact:

artifact = run.use_artifact("<my-entity/my-project/my-artifact:alias>")
artifact.ttl = None
artifact.save()

1. Navigate to your W&B project in the W&B App UI.
2. Select the artifact icon on the left panel.
3. From the list of artifacts, expand the artifact type you
4. Select on the artifact version you want to edit the TTL policy for.
5. Click on the Version tab.
6. Click on the meatball UI icon next to the Link to registry button.
7. From the dropdown, select Edit TTL policy.
8. Within the modal that appears, select Deactivate from the TTL policy dropdown.
9. Select the Update TTL button to save your changes.

View TTL policies

View TTL policies for artifacts with the Python SDK or with the W&B App UI.

• Python SDK
• W&B App

Use a print statement to view an artifact’s TTL policy. The following example shows how to retrieve an artifact and view its TTL policy:

artifact = run.use_artifact("<my-entity/my-project/my-artifact:alias>")
print(artifact.ttl)

View a TTL policy for an artifact with the W&B App UI.

1. Navigate to the W&B App at https://wandb.ai.
2. Go to your W&B Project.
3. Within your project, select the Artifacts tab in the left sidebar.
4. Click on a collection.

Within the collection view you can see all of the artifacts in the selected collection. Within the Time to Live column you will see the TTL policy assigned to that artifact.

1.7.3 - Manage artifact storage and memory allocation

W&B stores artifact files in a private Google Cloud Storage bucket located in the United States by default. All files are encrypted at rest and in transit.

For sensitive files, we recommend you set up Private Hosting or use reference artifacts.

During training, W&B locally saves logs, artifacts, and configuration files in the following local directories:

For a complete guide to using environment variables to configure W&B, see the environment variables reference.

Clean up local artifact cache

W&B caches artifact files to speed up downloads across versions that share files in common. Over time this cache directory can become large. Run the wandb artifact cache cleanup command to prune the cache and to remove any files that have not been used recently.

The proceeding code snippet demonstrates how to limit the size of the cache to 1GB. Copy and paste the code snippet into your terminal:

$ wandb artifact cache cleanup 1GB

1.8 - Explore artifact graphs

W&B automatically tracks the artifacts a given run logged as well as the artifacts a given run uses. These artifacts can include datasets, models, evaluation results, or more. You can explore an artifact’s lineage to track and manage the various artifacts produced throughout the machine learning lifecycle.

Lineage

Tracking an artifact’s lineage has several key benefits:

• Reproducibility: By tracking the lineage of all artifacts, teams can reproduce experiments, models, and results, which is essential for debugging, experimentation, and validating machine learning models.

• Version Control: Artifact lineage involves versioning artifacts and tracking their changes over time. This allows teams to roll back to previous versions of data or models if needed.

• Auditing: Having a detailed history of the artifacts and their transformations enables organizations to comply with regulatory and governance requirements.

• Collaboration and Knowledge Sharing: Artifact lineage facilitates better collaboration among team members by providing a clear record of attempts as well as what worked, and what didn’t. This helps in avoiding duplication of efforts and accelerates the development process.

Finding an artifact’s lineage

When selecting an artifact in the Artifacts tab, you can see your artifact’s lineage. This graph view shows a general overview of your pipeline.

To view an artifact graph:

1. Navigate to your project in the W&B App UI
2. Choose the artifact icon on the left panel.
3. Select Lineage.

Navigating the lineage graph

The artifact or job type you provide appears in front of its name, with artifacts represented by blue icons and runs represented by green icons. Arrows detail the input and output of a run or artifact on the graph.

For a more detailed view, click any individual artifact or run to get more information on a particular object.

Artifact clusters

When a level of the graph has five or more runs or artifacts, it creates a cluster. A cluster has a search bar to find specific versions of runs or artifacts and pulls an individual node from a cluster to continue investigating the lineage of a node inside a cluster.

Clicking on a node opens a preview with an overview of the node. Clicking on the arrow extracts the individual run or artifact so you can examine the lineage of the extracted node.

Use the API to track lineage

You can also navigate a graph using the W&B API.

Create an artifact. First, create a run with wandb.init. Then,create a new artifact or retrieve an existing one with wandb.Artifact. Next, add files to the artifact with .add_file. Finally, log the artifact to the run with .log_artifact. The finished code looks something like this:

with wandb.init() as run:
    artifact = wandb.Artifact("artifact_name", "artifact_type")

    # Add Files and Assets to the artifact using
    # `.add`, `.add_file`, `.add_dir`, and `.add_reference`
    artifact.add_file("image1.png")
    run.log_artifact(artifact)

Use the artifact object’s logged_by and used_by methods to walk the graph from the artifact:

# Walk up and down the graph from an artifact:
producer_run = artifact.logged_by()
consumer_runs = artifact.used_by()

Next steps

• Explore artifacts in more detail
• Manage artifact storage
• Explore an artifacts project

1.9 - Artifact data privacy and compliance

Files are uploaded to Google Cloud bucket managed by W&B when you log artifacts. The contents of the bucket are encrypted both at rest and in transit. Artifact files are only visible to users who have access to the corresponding project.

When you delete a version of an artifact, it is marked for soft deletion in our database and removed from your storage cost. When you delete an entire artifact, it is queued for permanently deletion and all of its contents are removed from the W&B bucket. If you have specific needs around file deletion please reach out to Customer Support.

For sensitive datasets that cannot reside in a multi-tenant environment, you can use either a private W&B server connected to your cloud bucket or reference artifacts. Reference artifacts track references to private buckets without sending file contents to W&B. Reference artifacts maintain links to files on your buckets or servers. In other words, W&B only keeps track of the metadata associated with the files and not the files themselves.

Create a reference artifact similar to how you create a non reference artifact:

import wandb

run = wandb.init()
artifact = wandb.Artifact("animals", type="dataset")
artifact.add_reference("s3://my-bucket/animals")

For alternatives, contact us at contact@wandb.com to talk about private cloud and on-premises installations.

1.10 - Tutorial: Create, track, and use a dataset artifact

This walkthrough demonstrates how to create, track, and use a dataset artifact from W&B Runs.

1. Log into W&B

Import the W&B library and log in to W&B. You will need to sign up for a free W&B account if you have not done so already.

import wandb

wandb.login()

2. Initialize a run

Use the wandb.init() API to generate a background process to sync and log data as a W&B Run. Provide a project name and a job type:

# Create a W&B Run. Here we specify 'dataset' as the job type since this example
# shows how to create a dataset artifact.
run = wandb.init(project="artifacts-example", job_type="upload-dataset")

3. Create an artifact object

Create an artifact object with the wandb.Artifact() API. Provide a name for the artifact and a description of the file type for the name and type parameters, respectively.

For example, the following code snippet demonstrates how to create an artifact called ‘bicycle-dataset’ with a ‘dataset’ label:

artifact = wandb.Artifact(name="bicycle-dataset", type="dataset")

For more information about how to construct an artifact, see Construct artifacts.

Add the dataset to the artifact

Add a file to the artifact. Common file types include models and datasets. The following example adds a dataset named dataset.h5 that is saved locally on our machine to the artifact:

# Add a file to the artifact's contents
artifact.add_file(local_path="dataset.h5")

Replace the filename dataset.h5 in the preceding code snippet with the path to the file you want to add to the artifact.

4. Log the dataset

Use the W&B run objects log_artifact() method to both save your artifact version and declare the artifact as an output of the run.

# Save the artifact version to W&B and mark it
# as the output of this run
run.log_artifact(artifact)

A 'latest' alias is created by default when you log an artifact. For more information about artifact aliases and versions, see Create a custom alias and Create new artifact versions, respectively.

5. Download and use the artifact

The following code example demonstrates the steps you can take to use an artifact you have logged and saved to the W&B servers.

1. First, initialize a new run object with wandb.init().
2. Second, use the run objects use_artifact() method to tell W&B what artifact to use. This returns an artifact object.
3. Third, use the artifacts download() method to download the contents of the artifact.

# Create a W&B Run. Here we specify 'training' for 'type'
# because we will use this run to track training.
run = wandb.init(project="artifacts-example", job_type="training")

# Query W&B for an artifact and mark it as input to this run
artifact = run.use_artifact("bicycle-dataset:latest")

# Download the artifact's contents
artifact_dir = artifact.download()

Alternatively, you can use the Public API (wandb.Api) to export (or update data) data already saved in a W&B outside of a Run. See Track external files for more information.

2 - Secrets

W&B Secret Manager allows you to securely and centrally store, manage, and inject secrets, which are sensitive strings such as access tokens, bearer tokens, API keys, or passwords. W&B Secret Manager removes the need to add sensitive strings directly to your code or when configuring a webhook’s header or payload.

Secrets are stored and managed in each team’s Secret Manager, in the Team secrets section of the team settings.

Add a secret

To add a secret:

1. If the receiving service requires it to authenticate incoming webhooks, generate the required token or API key. If necessary, save the sensitive string securely, such as in a password manager.
2. Log in to W&B and go to the team’s Settings page.
3. In the Team Secrets section, click New secret.
4. Using letters, numbers, and underscores (_), provide a name for the secret.
5. Paste the sensitive string into the Secret field.
6. Click Add secret.

Specify the secrets you want to use for your webhook automation when you configure the webhook. See the Configure a webhook section for more information.

Rotate a secret

To rotate a secret and update its value:

1. Click the pencil icon in the secret’s row to open the secret’s details.
2. Set Secret to the new value. Optionally click Reveal secret to verify the new value.
3. Click Add secret. The secret’s value updates and no longer resolves to the previous value.

Delete a secret

To delete a secret:

1. Click the trash icon in the secret’s row.
2. Read the confirmation dialog, then click Delete. The secret is deleted immediately and permanently.

Manage access to secrets

A team’s automations can use the team’s secrets. Before you remove a secret, update or remove automations that use it so they don’t stop working.

3 - Registry

W&B Registry is a curated central repository of artifact versions within your organization. Users who have permission within your organization can download, share, and collaboratively manage the lifecycle of all artifacts, regardless of the team that user belongs to.

You can use the Registry to track artifact versions, audit the history of an artifact’s usage and changes, ensure governance and compliance of your artifacts, and automate downstream processes such as model CI/CD.

In summary, use W&B Registry to:

• Promote artifact versions that satisfy a machine learning task to other users in your organization.
• Organize artifacts with tags so that you can find or reference specific artifacts.
• Track an artifact’s lineage and audit the history of changes.
• Automate downstream processes such as model CI/CD.
• Limit who in your organization can access artifacts in each registry.

The preceding image shows the Registry App with “Model” and “Dataset” core registries along with custom registries.

Learn the basics

Each organization initially contains two registries that you can use to organize your model and dataset artifacts called Models and Datasets, respectively. You can create additional registries to organize other artifact types based on your organization’s needs.

Each registry consists of one or more collections. Each collection represents a distinct task or use case.

To add an artifact to a registry, you first log a specific artifact version to W&B. Each time you log an artifact, W&B automatically assigns a version to that artifact. Artifact versions use 0 indexing, so the first version is v0, the second version is v1, and so on.

Once you log an artifact to W&B, you can then link that specific artifact version to a collection in the registry.

As an example, the proceeding code example shows how to log and link a model artifact called “my_model.txt” to a collection named “first-collection” in the core registry:

1. Initialize a W&B run.
2. Log the artifact to W&B.
3. Specify the name of the collection and registry to link your artifact version to.
4. Link the artifact to the collection.

Save this Python code to a script and run it. W&B Python SDK version 0.18.6 or newer is required.

import wandb
import random

# Initialize a W&B run to track the artifact
run = wandb.init(project="registry_quickstart") 

# Create a simulated model file so that you can log it
with open("my_model.txt", "w") as f:
   f.write("Model: " + str(random.random()))

# Log the artifact to W&B
logged_artifact = run.log_artifact(
    artifact_or_path="./my_model.txt", 
    name="gemma-finetuned", 
    type="model" # Specifies artifact type
)

# Specify the name of the collection and registry
# you want to publish the artifact to
COLLECTION_NAME = "first-collection"
REGISTRY_NAME = "model"

# Link the artifact to the registry
run.link_artifact(
    artifact=logged_artifact, 
    target_path=f"wandb-registry-{REGISTRY_NAME}/{COLLECTION_NAME}"
)

W&B automatically creates a collection for you if the collection you specify in the returned run object’s link_artifact(target_path = "") method does not exist within the registry you specify.

Navigate to the Registry App to view artifact versions that you and other members of your organization publish. To do so, first navigate to W&B. Select Registry in the left sidebar below Applications. Select the “Model” registry. Within the registry, you should see the “first-collection” collection with your linked artifact version.

Once you link an artifact version to a collection within a registry, members of your organization can view, download, and manage your artifact versions, create downstream automations, and more if they have the proper permissions.

Enable W&B Registry

Based on your deployment type, satisfy the following conditions to enable W&B Registry:

Resources to get started

Depending on your use case, explore the following resources to get started with the W&B Registry:

• Check out the tutorial video:
  • Getting started with Registry from Weights & Biases
• Take the W&B Model CI/CD course and learn how to:
  • Use W&B Registry to manage and version your artifacts, track lineage, and promote models through different lifecycle stages.
  • Automate your model management workflows using webhooks.
  • Integrate the registry with external ML systems and tools for model evaluation, monitoring, and deployment.

Migrate from the legacy Model Registry to W&B Registry

The legacy Model Registry is scheduled for deprecation with the exact date not yet decided. Before deprecating the legacy Model Registry, W&B will migrate the contents of the legacy Model Registry to the W&B Registry.

See Migrating from legacy Model Registry for more information about the migration process from the legacy Model Registry to W&B Registry.

Until the migration occurs, W&B supports both the legacy Model Registry and the new Registry.

To view the legacy Model Registry, navigate to the Model Registry in the W&B App. A banner appears at the top of the page that enables you to use the legacy Model Registry App UI.

Reach out to support@wandb.com with any questions or to speak to the W&B Product Team about any concerns about the migration.

3.1 - Registry types

W&B supports two types of registries: Core registries and Custom registries.

Core registry

A core registry is a template for specific use cases: Models and Datasets.

By default, the Models registry is configured to accept "model" artifact types and the Dataset registry is configured to accept "dataset" artifact types. An admin can add additional accepted artifact types.

The preceding image shows the Models and the Dataset core registry along with a custom registry called Fine_Tuned_Models in the W&B Registry App UI.

A core registry has organization visibility. A registry admin can not change the visibility of a core registry.

Custom registry

Custom registries are not restricted to "model" artifact types or "dataset" artifact types.

You can create a custom registry for each step in your machine learning pipeline, from initial data collection to final model deployment.

For example, you might create a registry called “Benchmark_Datasets” for organizing curated datasets to evaluate the performance of trained models. Within this registry, you might have a collection called “User_Query_Insurance_Answer_Test_Data” that contains a set of user questions and corresponding expert-validated answers that the model has never seen during training.

A custom registry can have either organization or restricted visibility. A registry admin can change the visibility of a custom registry from organization to restricted. However, the registry admin can not change a custom registry’s visibility from restricted to organizational visibility.

For information on how to create a custom registry, see Create a custom registry.

Summary

The proceeding table summarizes the differences between core and custom registries:

3.2 - Create a custom registry

A custom registry offers flexibility and control over the artifact types that you can use, allows you to restrict the registry’s visibility, and more.

Create a custom registry

To create a custom registry:

1. Navigate to the Registry App at https://wandb.ai/registry/.
2. Within Custom registry, click on the Create registry button.
3. Provide a name for your registry in the Name field.
4. Optionally provide a description about the registry.
5. Select who can view the registry from the Registry visibility dropdown. See Registry visibility types for more information on registry visibility options.
6. Select either All types or Specify types from the Accepted artifacts type dropdown.
7. (If you select Specify types) Add one or more artifact types that your registry accepts.
8. Click on the Create registry button.

For example, the proceeding image shows a custom registry called Fine_Tuned_Models that a user is about to create. The registry is Restricted to only members that are manually added to the registry.

Visibility types

The visibility of a registry determines who can access that registry. Restricting the visibility of a custom registry helps ensure that only specified members can access that registry.

There are two type registry visibility options for a custom registry:

A team administrator or registry administrator can set the visibility of a custom registry.

The user who creates a custom registry with Restricted visibility is added to the registry automatically as its registry admin.

Configure the visibility of a custom registry

A team administrator or registry administrator can assign the visibility of a custom registry during or after the creation of a custom registry.

To restrict the visibility of an existing custom registry:

1. Navigate to the Registry App at https://wandb.ai/registry/.
2. Select a registry.
3. Click on the gear icon on the upper right hand corner.
4. From the Registry visibility dropdown, select the desired registry visibility.
5. if you select Restricted visibility:
   1. Add members of your organization that you want to have access to this registry. Scroll to the Registry members and roles section and click on the Add member button.
   2. Within the Member field, add the email or username of the member you want to add.
   3. Click Add new member.

See Create a custom registry for more information on how assign the visibility of a custom registry when a team administrator creates it.

3.3 - Configure registry access

A registry admin can configure registry roles, add users, or remove users from a registry by configuring the registry’s settings.

Manage users

Add a user or a team

Registry admins can add individual users or entire teams to a registry. To add a user or team to a registry:

1. Navigate to the Registry at https://wandb.ai/registry/.
2. Select the registry you want to add a user or team to.
3. Click on the gear icon on the upper right hand corner to access the registry settings.
4. In the Registry access section, click Add access.
5. Specify one or more user names, emails, or the team names to the Include users and teams field.
6. Click Add access.

Learn more about configuring user roles in a registry, or Registry role permissions .

Remove a user or team

A registry admin can remove individual users or entire teams from a registry. To remove a user or team from a registry:

1. Navigate to the Registry at https://wandb.ai/registry/.
2. Select the registry you want to remove a user from.
3. Click on the gear icon on the upper right hand corner to access the registry settings.
4. Navigate to the Registry access section and type in the username, email, or team you want to remove.
5. Click the Delete button.

Registry roles

Each user in a registry has a registry role, which determines what they can do in that registry.

W&B automatically assigns a default registry role to a user or team when they are added to a registry.

A registry admin can assign or modify roles for users and teams in a registry. See Configure user roles in a registry for more information.

The proceeding table lists the different roles a user can have and their permissions:

Inherited permissions

A user’s permission in a registry depends on the highest level of privilege assigned to that user, whether individually or by team membership.

For example, suppose a registry admin adds a user called Nico to Registry A and assigns them a Viewer registry role. A registry admin then adds a team called Foundation Model Team to Registry A and assigns Foundation Model Team a Member registry role.

Nico is a member of the Foundation Model Team, which is a Member of the Registry. Because Member has more permission than Viewer, W&B grants Nico the Member role.

The proceeding table demonstrates the highest level of permission in the event of a conflict between a user’s individual registry role and the registry role of a team they are a member of:

If there is a conflict, W&B displays the highest level of permissions next to the name of the user.

For example, in the proceeding image Alex inherits Member role privileges because they are a member of the smle-reg-team-1 team.

Configure registry roles

1. Navigate to the Registry at https://wandb.ai/registry/.
2. Select the registry you want to configure.
3. Click the gear icon on the upper right hand corner.
4. Scroll to the Registry members and roles section.
5. Within the Member field, search for the user or team you want to edit permissions for.
6. In the Registry role column, click the user’s role.
7. From the dropdown, select the role you want to assign to the user.

3.4 - Create a collection

A collection is a set of linked artifact versions within a registry. Each collection represents a distinct task or use case.

For example, within the core Dataset registry you might have multiple collections. Each collection contains a different dataset such as MNIST, CIFAR-10, or ImageNet.

As another example, you might have a registry called “chatbot” that contains a collection for model artifacts, another collection for dataset artifacts, and another collection for fine-tuned model artifacts.

How you organize a registry and their collections is up to you.

Collection types

Each collection accepts one, and only one, type of artifact. The type you specify restricts what sort of artifacts you, and other members of your organization, can link to that collection.

For example, suppose you create a collection that accepts “dataset” artifact types. This means that you can only link future artifact versions that have the type “dataset” to this collection. Similarly, you can only link artifacts of type “model” to a collection that accepts only model artifact types.

import wandb

# Initialize a run
run = wandb.init(
  entity = "<team_entity>",
  project = "<project>"
  )

# Create an artifact object
artifact = wandb.Artifact(
    name="<artifact_name>", 
    type="<artifact_type>"
    )

When you create a collection, you can select from a list of predefined artifact types. The artifact types available to you depend on the registry that the collection belongs to. .

Before you link an artifact to a collection or create a new collection, investigate the types of artifacts that collection accepts.

Check the types of artifact that a collection accepts

Before you link to a collection, inspect the artifact type that the collection accepts. You can inspect the artifact types that collection accepts programmatically with the W&B Python SDK or interactively with the W&B App

• W&B App
• Python SDK (Beta)

You can find the accepted artifact types on the registry card on the homepage or within a registry’s settings page.

For both methods, first navigate to your W&B Registry App.

Within the homepage of the Registry App, you can view the accepted artifact types by scrolling to the registry card of that registry. The gray horizontal ovals within the registry card lists the artifact types that registry accepts.

For example, the preceding image shows multiple registry cards on the Registry App homepage. Within the Model registry card, you can see two artifact types: model and model-new.

To view accepted artifact types within a registry’s settings page:

1. Click on the registry card you want to view the settings for.
2. Click on the gear icon in the upper right corner.
3. Scroll to the Accepted artifact types field.

Programmatically view the artifact types that a registry accepts with the W&B Python SDK:

import wandb

registry_name = "<registry_name>"
artifact_types = wandb.Api().project(name=f"wandb-registry-{registry_name}").artifact_types()
print(artifact_type.name for artifact_type in artifact_types)

Note that you do not initialize a run with the proceeding code snippet. This is because it is unnecessary to create a run if you are only querying the W&B API and not tracking an experiment, artifact and so on.

Once you know what type of artifact a collection accepts, you can create a collection.

Create a collection

Interactively or programmatically create a collection within a registry. You can not change the type of artifact that a collection accepts after you create it.

Programmatically create a collection

Use the wandb.init.link_artifact() method to link an artifact to a collection. Specify both the collection and the registry to the target_path field as a path that takes the form of:

f"wandb-registry-{registry_name}/{collection_name}"

Where registry_name is the name of the registry and collection_name is the name of the collection. Ensure to append the prefix wandb-registry- to the registry name.

The proceeding code snippet shows how to programmatically create a collection. Ensure to replace other the values enclosed in <> with your own:

import wandb

# Initialize a run
run = wandb.init(entity = "<team_entity>", project = "<project>")

# Create an artifact object
artifact = wandb.Artifact(
  name = "<artifact_name>",
  type = "<artifact_type>"
  )

registry_name = "<registry_name>"
collection_name = "<collection_name>"
target_path = f"wandb-registry-{registry_name}/{collection_name}"

# Link the artifact to a collection
run.link_artifact(artifact = artifact, target_path = target_path)

run.finish()

Interactively create a collection

The following steps describe how to create a collection within a registry using the W&B Registry App UI:

1. Navigate to the Registry App in the W&B App UI.
2. Select a registry.
3. Click on the Create collection button in the upper right hand corner.
4. Provide a name for your collection in the Name field.
5. Select a type from the Type dropdown. Or, if the registry enables custom artifact types, provide one or more artifact types that this collection accepts.
6. Optionally provide a description of your collection in the Description field.
7. Optionally add one or more tags in the Tags field.
8. Click Link version.
9. From the Project dropdown, select the project where your artifact is stored.
10. From the Artifact collection dropdown, select your artifact.
11. From the Version dropdown, select the artifact version you want to link to your collection.
12. Click on the Create collection button.

3.5 - Link an artifact version to a registry

Link artifact versions to a collection to make them available to other members in your organization.

When you link an artifact to a registry, this “publishes” that artifact to that registry. Any user that has access to that registry can access the linked artifact versions in the collection.

In other words, linking an artifact to a registry collection brings that artifact version from a private, project-level scope, to a shared organization level scope.

Link an artifact to a collection

Link an artifact version to a collection interactively or programmatically.

Based on your use case, follow the instructions described in the tabs below to link an artifact version.

• Python SDK
• Registry App
• Artifact browser

wandb-registry-{REGISTRY_NAME}/{COLLECTION_NAME}

import wandb

# Initialize a run
run = wandb.init(
  entity = "<team_entity>",
  project = "<project_name>"
)

# Create an artifact object
# The type parameter specifies both the type of the 
# artifact object and the collection type
artifact = wandb.Artifact(name = "<name>", type = "<type>")

# Add the file to the artifact object. 
# Specify the path to the file on your local machine.
artifact.add_file(local_path = "<local_path_to_artifact>")

# Specify the collection and registry to link the artifact to
REGISTRY_NAME = "<registry_name>"  
COLLECTION_NAME = "<collection_name>"
target_path=f"wandb-registry-{REGISTRY_NAME}/{COLLECTION_NAME}"

# Link the artifact to the collection
run.link_artifact(artifact = artifact, target_path = target_path)

View a linked artifact’s metadata, version data, usage, lineage information and more in the Registry App.

View linked artifacts in a registry

View information about linked artifacts such as metadata, lineage, and usage information in the Registry App.

1. Navigate to the Registry App.
2. Select the name of the registry that you linked the artifact to.
3. Select the name of the collection.
4. If the collection’s artifacts log metrics, compare metrics across versions by clicking Show metrics.
5. From the list of artifact versions, select the version you want to access. Version numbers are incrementally assigned to each linked artifact version starting with v0.
6. To view details about an artifact version, click the version. From the tabs in this page, you can view that version’s metadata (including logged metrics), lineage, and usage information.

Make note of the Full Name field within the Version tab. The full name of a linked artifact consists of the registry, collection name, and the alias or index of the artifact version.

wandb-registry-{REGISTRY_NAME}/{COLLECTION_NAME}:v{INTEGER}

You need the full name of a linked artifact to access the artifact version programmatically.

Troubleshooting

Below are some common things to double check if you are not able to link an artifact.

Logging artifacts from a personal account

Artifacts logged to W&B with a personal entity can not be linked to the registry. Make sure that you log artifacts using a team entity within your organization. Only artifacts logged within an organization’s team can be linked to the organization’s registry.

Find your team entity

W&B uses the name of your team as the team’s entity. For example, if your team is called team-awesome, your team entity is team-awesome.

You can confirm the name of your team by:

1. Navigate to your team’s W&B profile page.
2. Copy the site’s URL. It has the form of https://wandb.ai/<team>. Where <team> is the both the name of your team and the team’s entity.

Log from a team entity

1. Specify the team as the entity when you initialize a run with wandb.init(). If you do not specify the entity when you initialize a run, the run uses your default entity which may or may not be your team entity.

import wandb   

run = wandb.init(
  entity='<team_entity>', 
  project='<project_name>'
  )

2. Log the artifact to the run either with run.log_artifact or by creating an Artifact object and then adding files to it with :

   artifact = wandb.Artifact(name="<artifact_name>", type="<type>")
   

   For more information on how to log artifacts, see Construct artifacts.

3. If an artifact is logged to your personal entity, you will need to re-log it to an entity within your organization.

Confirm the path of a registry in the W&B App UI

There are two ways to confirm the path of a registry with the UI: create an empty collection and view the collection details or copy and paste the autogenerated code on the collection’s home page.

Copy and paste autogenerated code

1. Navigate to the Registry app at https://wandb.ai/registry/.
2. Click the registry you want to link an artifact to.
3. At the top of the page, you will see an autogenerated code block.
4. Copy and paste this into your code, ensure to replace the last part of the path with the name of your collection.

Create an empty collection

1. Navigate to the Registry app at https://wandb.ai/registry/.
2. Click the registry you want to link an artifact to.
3. Click on the empty collection. If an empty collection does not exist, create a new collection.
4. Within the code snippet that appears, identify the target_path field within .link_artifact().
5. (Optional) Delete the collection.

For example, after completing the steps outlined, you find the code block with the target_path parameter:

target_path = 
      "smle-registries-bug-bash/wandb-registry-Golden Datasets/raw_images"

Breaking this down into its components, you can see what you will need to use to create the path to link your artifact programmatically:

ORG_ENTITY_NAME = "smle-registries-bug-bash"
REGISTRY_NAME = "Golden Datasets"
COLLECTION_NAME = "raw_images"

3.6 - Download an artifact from a registry

Use the W&B Python SDK to download an artifact linked to a registry. To download and use an artifact, you need to know the name of the registry, the name of the collection, and the alias or index of the artifact version you want to download.

Once you know the properties of the artifact, you can construct the path to the linked artifact and download the artifact. Alternatively, you can copy and paste a pre-generated code snippet from the W&B App UI to download an artifact linked to a registry.

Construct path to linked artifact

To download an artifact linked to a registry, you must know the path of that linked artifact. The path consists of the registry name, collection name, and the alias or index of the artifact version you want to access.

Once you have the registry, collection, and alias or index of the artifact version, you can construct the path to the linked artifact using the proceeding string template:

# Artifact name with version index specified
f"wandb-registry-{REGISTRY}/{COLLECTION}:v{INDEX}"

# Artifact name with alias specified
f"wandb-registry-{REGISTRY}/{COLLECTION}:{ALIAS}"

Replace the values within the curly braces {} with the name of the registry, collection, and the alias or index of the artifact version you want to access.

Use the wandb.init.use_artifact method to access the artifact and download its contents once you have the path of the linked artifact. The proceeding code snippet shows how to use and download an artifact linked to the W&B Registry. Ensure to replace values within <> with your own:

import wandb

REGISTRY = '<registry_name>'
COLLECTION = '<collection_name>'
ALIAS = '<artifact_alias>'

run = wandb.init(
   entity = '<team_name>',
   project = '<project_name>'
   )  

artifact_name = f"wandb-registry-{REGISTRY}/{COLLECTION}:{ALIAS}"
# artifact_name = '<artifact_name>' # Copy and paste Full name specified on the Registry App
fetched_artifact = run.use_artifact(artifact_or_name = artifact_name)  
download_path = fetched_artifact.download()  

The .use_artifact() method both creates a run and marks the artifact you download as the input to that run. Marking an artifact as the input to a run enables W&B to track the lineage of that artifact.

If you do not want to create a run, you can use the wandb.Api() object to access the artifact:

import wandb

REGISTRY = "<registry_name>"
COLLECTION = "<collection_name>"
VERSION = "<version>"

api = wandb.Api()
artifact_name = f"wandb-registry-{REGISTRY}/{COLLECTION}:{VERSION}"
artifact = api.artifact(name = artifact_name)

import wandb

TEAM_ENTITY = "product-team-applications"
PROJECT_NAME = "user-stories"

REGISTRY = "Fine-tuned Models"
COLLECTION = "phi3-finetuned"
ALIAS = 'production'

# Initialize a run inside the specified team and project
run = wandb.init(entity=TEAM_ENTITY, project = PROJECT_NAME)

artifact_name = f"wandb-registry-{REGISTRY}/{COLLECTION}:{ALIAS}"

# Access an artifact and mark it as input to your run for lineage tracking
fetched_artifact = run.use_artifact(artifact_or_name = name)  

# Download artifact. Returns path to downloaded contents
downloaded_path = fetched_artifact.download()  

See use_artifact and Artifact.download() in the API Reference guide for more information on possible parameters and return type.

import wandb

REGISTRY = "<registry_name>"
COLLECTION = "<collection_name>"
VERSION = "<version>"

# Ensure you are using your team entity to instantiate the API
api = wandb.Api(overrides={"entity": "<team-entity>"})
artifact_name = f"wandb-registry-{REGISTRY}/{COLLECTION}:{VERSION}"
artifact = api.artifact(name = artifact_name)

# Use org display name or org entity in the path
api = wandb.Api()
artifact_name = f"{ORG_NAME}/wandb-registry-{REGISTRY}/{COLLECTION}:{VERSION}"
artifact = api.artifact(name = artifact_name)

Copy and paste pre-generated code snippet

W&B creates a code snippet that you can copy and paste into your Python script, notebook, or terminal to download an artifact linked to a registry.

1. Navigate to the Registry App.
2. Select the name of the registry that contains your artifact.
3. Select the name of the collection.
4. From the list of artifact versions, select the version you want to access.
5. Select the Usage tab.
6. Copy the code snippet shown in the Usage API section.
7. Paste the code snippet into your Python script, notebook, or terminal.

3.7 - Find registry items

Use the global search bar in the W&B Registry App to find a registry, collection, artifact version tag, collection tag, or alias. You can use MongoDB-style queries to filter registries, collections, and artifact versions based on specific criteria using the W&B Python SDK.

Only items that you have permission to view appear in the search results.

Search for registry items

To search for a registry item:

1. Navigate to the W&B Registry App.
2. Specify the search term in the search bar at the top of the page. Press Enter to search.

Search results appear below the search bar if the term you specify matches an existing registry, collection name, artifact version tag, collection tag, or alias.

Query registry items with MongoDB-style queries

Use the wandb.Api().registries() and query predicates to filter registries, collections, and artifact versions based on one or more MongoDB-style queries.

The following table lists query names you can use based on the type of item you want to filter:

The proceeding code examples demonstrate some common search scenarios.

To use the wandb.Api().registries() method, first import the W&B Python SDK (wandb) library:

import wandb

# (Optional) Create an instance of the wandb.Api() class for readability
api = wandb.Api()

Filter all registries that contain the string model:

# Filter all registries that contain the string `model`
registry_filters = {
    "name": {"$regex": "model"}
}

# Returns an iterable of all registries that match the filters
registries = api.registries(filter=registry_filters)

Filter all collections, independent of registry, that contains the string yolo in the collection name:

# Filter all collections, independent of registry, that 
# contains the string `yolo` in the collection name
collection_filters = {
    "name": {"$regex": "yolo"}
}

# Returns an iterable of all collections that match the filters
collections = api.registries().collections(filter=collection_filters)

Filter all collections, independent of registry, that contains the string yolo in the collection name and possesses cnn as a tag:

# Filter all collections, independent of registry, that contains the
# string `yolo` in the collection name and possesses `cnn` as a tag
collection_filters = {
    "name": {"$regex": "yolo"},
    "tag": "cnn"
}

# Returns an iterable of all collections that match the filters
collections = api.registries().collections(filter=collection_filters)

Find all artifact versions that contains the string model and has either the tag image-classification or an latest alias:

# Find all artifact versions that contains the string `model` and 
# has either the tag `image-classification` or an `latest` alias
registry_filters = {
    "name": {"$regex": "model"}
}

# Use logical $or operator to filter artifact versions
version_filters = {
    "$or": [
        {"tag": "image-classification"},
        {"alias": "production"}
    ]
}

# Returns an iterable of all artifact versions that match the filters
artifacts = api.registries(filter=registry_filters).collections().versions(filter=version_filters)

See the MongoDB documentation for more information on logical query operators.

Each item in the artifacts iterable in the previous code snippet is an instance of the Artifact class. This means that you can access each artifact’s attributes, such as name, collection, aliases, tags, created_at, and more:

for art in artifacts:
    print(f"artifact name: {art.name}")
    print(f"collection artifact belongs to: { art.collection.name}")
    print(f"artifact aliases: {art.aliases}")
    print(f"tags attached to artifact: {art.tags}")
    print(f"artifact created at: {art.created_at}\n")

For a complete list of an artifact object’s attributes, see the Artifacts Class in the API Reference docs.

Filter all artifact versions, independent of registry or collection, created between 2024-01-08 and 2025-03-04 at 13:10 UTC:

# Find all artifact versions created between 2024-01-08 and 2025-03-04 at 13:10 UTC. 

artifact_filters = {
    "alias": "latest",
    "created_at" : {"$gte": "2024-01-08", "$lte": "2025-03-04 13:10:00"},
}

# Returns an iterable of all artifact versions that match the filters
artifacts = api.registries().collections().versions(filter=artifact_filters)

Specify the date and time in the format YYYY-MM-DD HH:MM:SS. You can omit the hours, minutes, and seconds if you want to filter by date only.

See the MongoDB documentation for more information on query comparisons.

3.8 - Organize versions with tags

Create and add tags to organize your collections or artifact versions within your registry. Add, modify, view, or remove tags to a collection or artifact version with the W&B App UI or the W&B Python SDK.

Add a tag to a collection

Use the W&B App UI or Python SDK to add a tag to a collection:

• W&B App
• Python SDK

Use the W&B App UI to add a tag to a collection:

1. Navigate to the W&B Registry at https://wandb.ai/registry
2. Click on a registry card
3. Click View details next to the name of a collection
4. Within the collection card, click on the plus icon (+) next to the Tags field and type in the name of the tag
5. Press Enter on your keyboard

import wandb

COLLECTION_TYPE = "<collection_type>"
ORG_NAME = "<org_name>"
REGISTRY_NAME = "<registry_name>"
COLLECTION_NAME = "<collection_name>"

full_name = f"{ORG_NAME}/wandb-registry-{REGISTRY_NAME}/{COLLECTION_NAME}"

collection = wandb.Api().artifact_collection(
  type_name = COLLECTION_TYPE, 
  name = full_name
  )

collection.tags = ["your-tag"]
collection.save()

Update tags that belong to a collection

Update a tag programmatically by reassigning or by mutating the tags attribute. W&B recommends, and it is good Python practice, that you reassign the tags attribute instead of in-place mutation.

For example, the proceeding code snippet shows common ways to update a list with reassignment. For brevity, we continue the code example from the Add a tag to a collection section:

collection.tags = [*collection.tags, "new-tag", "other-tag"]
collection.tags = collection.tags + ["new-tag", "other-tag"]

collection.tags = set(collection.tags) - set(tags_to_delete)
collection.tags = []  # deletes all tags

The following code snippet shows how you can use in-place mutation to update tags that belong to an artifact version:

collection.tags += ["new-tag", "other-tag"]
collection.tags.append("new-tag")

collection.tags.extend(["new-tag", "other-tag"])
collection.tags[:] = ["new-tag", "other-tag"]
collection.tags.remove("existing-tag")
collection.tags.pop()
collection.tags.clear()

View tags that belong to a collection

Use the W&B App UI to view tags added to a collection:

1. Navigate to the W&B Registry at https://wandb.ai/registry
2. Click on a registry card
3. Click View details next to the name of a collection

If a collection has one or more tags, you can view those tags within the collection card next to the Tags field.

Tags added to a collection also appear next to the name of that collection.

For example, in the proceeding image, a tag called “tag1” was added to the “zoo-dataset-tensors” collection.

Remove a tag from a collection

Use the W&B App UI to remove a tag from a collection:

1. Navigate to the W&B Registry at https://wandb.ai/registry
2. Click on a registry card
3. Click View details next to the name of a collection
4. Within the collection card, hover your mouse over the name of the tag you want to remove
5. Click on the cancel button (X icon)

Add a tag to an artifact version

Add a tag to an artifact version linked to a collection with the W&B App UI or with the Python SDK.

• W&B App
• Python SDK

1. Navigate to the W&B Registry at https://wandb.ai/registry
2. Click on a registry card
3. Click View details next to the name of the collection you want to add a tag to
4. Scroll down to Versions
5. Click View next to an artifact version
6. Within the Version tab, click on the plus icon (+) next to the Tags field and type in the name of the tag
7. Press Enter on your keyboard

Fetch the artifact version you want to add or update a tag to. Once you have the artifact version, you can access the artifact object’s tag attribute to add or modify tags to that artifact. Pass in one or more tags as list to the artifacts tag attribute.

Like other artifacts, you can fetch an artifact from W&B without creating a run or you can create a run and fetch the artifact within that run. In either case, ensure to call the artifact object’s save method to update the artifact on the W&B servers.

Copy and paste an appropriate code cells below to add or modify an artifact version’s tag. Replace the values in <> with your own.

The proceeding code snippet shows how to fetch an artifact and add a tag without creating a new run:

import wandb

ARTIFACT_TYPE = "<TYPE>"
ORG_NAME = "<org_name>"
REGISTRY_NAME = "<registry_name>"
COLLECTION_NAME = "<collection_name>"
VERSION = "<artifact_version>"

artifact_name = f"{ORG_NAME}/wandb-registry-{REGISTRY_NAME}/{COLLECTION_NAME}:v{VERSION}"

artifact = wandb.Api().artifact(name = artifact_name, type = ARTIFACT_TYPE)
artifact.tags = ["tag2"] # Provide one or more tags in a list
artifact.save()

The proceeding code snippet shows how to fetch an artifact and add a tag by creating a new run:

import wandb

ORG_NAME = "<org_name>"
REGISTRY_NAME = "<registry_name>"
COLLECTION_NAME = "<collection_name>"
VERSION = "<artifact_version>"

run = wandb.init(entity = "<entity>", project="<project>")

artifact_name = f"{ORG_NAME}/wandb-registry-{REGISTRY_NAME}/{COLLECTION_NAME}:v{VERSION}"

artifact = run.use_artifact(artifact_or_name = artifact_name)
artifact.tags = ["tag2"] # Provide one or more tags in a list
artifact.save()

Update tags that belong to an artifact version

Update a tag programmatically by reassigning or by mutating the tags attribute. W&B recommends, and it is good Python practice, that you reassign the tags attribute instead of in-place mutation.

For example, the proceeding code snippet shows common ways to update a list with reassignment. For brevity, we continue the code example from the Add a tag to an artifact version section:

artifact.tags = [*artifact.tags, "new-tag", "other-tag"]
artifact.tags = artifact.tags + ["new-tag", "other-tag"]

artifact.tags = set(artifact.tags) - set(tags_to_delete)
artifact.tags = []  # deletes all tags

The following code snippet shows how you can use in-place mutation to update tags that belong to an artifact version:

artifact.tags += ["new-tag", "other-tag"]
artifact.tags.append("new-tag")

artifact.tags.extend(["new-tag", "other-tag"])
artifact.tags[:] = ["new-tag", "other-tag"]
artifact.tags.remove("existing-tag")
artifact.tags.pop()
artifact.tags.clear()

View tags that belong to an artifact version

View tags that belong to an artifact version that is linked to a registry with the W&B App UI or with the Python SDK.

• W&B App
• Python SDK

1. Navigate to the W&B Registry at https://wandb.ai/registry
2. Click on a registry card
3. Click View details next to the name of the collection you want to add a tag to
4. Scroll down to Versions section

If an artifact version has one or more tags, you can view those tags within the Tags column.

Fetch the artifact version to view its tags. Once you have the artifact version, you can view tags that belong to that artifact by viewing the artifact object’s tag attribute.

Like other artifacts, you can fetch an artifact from W&B without creating a run or you can create a run and fetch the artifact within that run.

Copy and paste an appropriate code cells below to add or modify an artifact version’s tag. Replace the values in <> with your own.

The proceeding code snippet shows how to fetch and view an artifact version’s tags without creating a new run:

import wandb

ARTIFACT_TYPE = "<TYPE>"
ORG_NAME = "<org_name>"
REGISTRY_NAME = "<registry_name>"
COLLECTION_NAME = "<collection_name>"
VERSION = "<artifact_version>"

artifact_name = f"{ORG_NAME}/wandb-registry-{REGISTRY_NAME}/{COLLECTION_NAME}:v{VERSION}"

artifact = wandb.Api().artifact(name = artifact_name, type = artifact_type)
print(artifact.tags)

The proceeding code snippet shows how to fetch and view artifact version’s tags by creating a new run:

import wandb

ORG_NAME = "<org_name>"
REGISTRY_NAME = "<registry_name>"
COLLECTION_NAME = "<collection_name>"
VERSION = "<artifact_version>"

run = wandb.init(entity = "<entity>", project="<project>")

artifact_name = f"{ORG_NAME}/wandb-registry-{REGISTRY_NAME}/{COLLECTION_NAME}:v{VERSION}"

artifact = run.use_artifact(artifact_or_name = artifact_name)
print(artifact.tags)

Remove a tag from an artifact version

1. Navigate to the W&B Registry at https://wandb.ai/registry
2. Click on a registry card
3. Click View details next to the name of the collection you want to add a tag to
4. Scroll down to Versions
5. Click View next to an artifact version
6. Within the Version tab, hover your mouse over the name of the tag
7. Click on the cancel button (X icon)

Search existing tags

Use the W&B App UI to search existing tags in collections and artifact versions:

1. Navigate to the W&B Registry at https://wandb.ai/registry
2. Click on a registry card
3. Within the search bar, type in the name of a tag

Find artifact versions with a specific tag

Use the W&B Python SDK to find artifact versions that have a set of tags:

import wandb

api = wandb.Api()
tagged_artifact_versions = api.artifacts(
    type_name = "<artifact_type>",
    name = "<artifact_name>",
    tags = ["<tag_1>", "<tag_2>"]
)

for artifact_version in tagged_artifact_versions:
    print(artifact_version.tags)

3.9 - Annotate collections

Add human-friendly text to your collections to help users understand the purpose of the collection and the artifacts it contains.

Depending on the collection, you might want to include information about the training data, model architecture, task, license, references, and deployment. The proceeding lists some topics worth documenting in a collection:

W&B recommends including at minimum these details:

• Summary: The purpose of the collection. The machine learning framework used for the machine learning experiment.
• License: The legal terms and permissions associated with the use of the machine learning model. It helps model users understand the legal framework under which they can utilize the model. Common licenses include Apache 2.0, MIT, and GPL.
• References: Citations or references to relevant research papers, datasets, or external resources.

If your collection contains training data, consider including these additional details:

• Training data: Describe the training data used
• Processing: Processing done on the training data set.
• Data storage: Where is that data stored and how to access it.

If your collection contains a machine learning model, consider including these additional details:

• Architecture: Information about the model architecture, layers, and any specific design choices.
• Task: The specific type of task or problem that the machine that the collection model is designed to perform. It’s a categorization of the model’s intended capability.
• Deserialize the model: Provide information on how someone on your team can load the model into memory.
• Task: The specific type of task or problem that the machine learning model is designed to perform. It’s a categorization of the model’s intended capability.
• Deployment: Details on how and where the model is deployed and guidance on how the model is integrated into other enterprise systems, such as a workflow orchestration platforms.

Add a description to a collection

Interactively or programmatically add a description to a collection with the W&B Registry UI or Python SDK.

• W&B Registry UI
• Python SDK

wandb-registry-{REGISTRY_NAME}/{COLLECTION_NAME}

import wandb

api = wandb.Api()

collection = api.artifact_collection(
  type_name = "<collection_type>", 
  name = "<collection_name>"
  )


collection.description = "This is a description."
collection.save()  

For example, the proceeding image shows a collection that documents a model’s architecture, intended use, performance information and more.

3.10 - Create and view lineage maps

Within a collection in the W&B Registry, you can view a history of the artifacts that an ML experiment uses. This history is called a lineage graph.

Lineage graphs can show the specific run that logs an artifact. In addition, lineage graphs can also show which run used an artifact as an input. In other words, lineage graphs can show the input and output of a run.

For example, the proceeding image shows artifacts created and used throughout an ML experiment:

From left to right, the image shows:

1. Multiple runs log the split_zoo_dataset:v4 artifact.
2. The “rural-feather-20” run uses the split_zoo_dataset:v4 artifact for training.
3. The output of the “rural-feather-20” run is a model artifact called zoo-ylbchv20:v0.
4. A run called “northern-lake-21” uses the model artifact zoo-ylbchv20:v0 to evaluate the model.

Track the input of a run

Mark an artifact as an input or dependency of a run with the wandb.init.use_artifact API.

The proceeding code snippet shows how to use the use_artifact. Replace values enclosed in angle brackets (< >) with your values:

import wandb

# Initialize a run
run = wandb.init(project="<project>", entity="<entity>")

# Get artifact, mark it as a dependency
artifact = run.use_artifact(artifact_or_name="<name>", aliases="<alias>")

Track the output of a run

Use (wandb.init.log_artifact) to declare an artifact as an output of a run.

The proceeding code snippet shows how to use the wandb.init.log_artifact API. Ensure to replace values enclosed in angle brackets (< >) with your values:

import wandb

# Initialize a run
run = wandb.init(entity  "<entity>", project = "<project>",)
artifact = wandb.Artifact(name = "<artifact_name>", type = "<artifact_type>")
artifact.add_file(local_path = "<local_filepath>", name="<optional-name>")

# Log the artifact as an output of the run
run.log_artifact(artifact_or_path = artifact)

For more information on about creating artifacts, see Create an artifact.

View lineage graphs in a collection

View the lineage of an artifact linked to a collection in the W&B Registry.

1. Navigate to the W&B Registry.
2. Select the collection that contains the artifact.
3. From the dropdown, click the artifact version you want to view its lineage graph.
4. Select the “Lineage” tab.

Once you are in an artifact’s lineage graph page, you can view additional information about any node in that lineage graph.

Select a run node to view that run’s details, such as the run’s ID, the run’s name, the run’s state, and more. As an example, the proceeding image shows information about the rural-feather-20 run:

Select an artifact node to view that artifact’s details, such as its full name, type, creation time, and associated aliases.

3.11 - Migrate from legacy Model Registry

W&B will transition assets from the legacy W&B Model Registry to the new W&B Registry. This migration will be fully managed and triggered by W&B, requiring no intervention from users. The process is designed to be as seamless as possible, with minimal disruption to existing workflows.

The transition will take place once the new W&B Registry includes all the functionalities currently available in the Model Registry. W&B will attempt to preserve current workflows, codebases, and references.

This guide is a living document and will be updated regularly as more information becomes available. For any questions or support, contact support@wandb.com.

How W&B Registry differs from the legacy Model Registry

W&B Registry introduces a range of new features and enhancements designed to provide a more robust and flexible environment for managing models, datasets, and other artifacts.

To view the legacy Model Registry, navigate to the Model Registry in the W&B App. A banner appears at the top of the page that enables you to use the legacy Model Registry App UI.

Organizational visibility

Artifacts linked to the legacy Model Registry have team level visibility. This means that only members of your team can view your artifacts in the legacy W&B Model Registry. W&B Registry has organization level visibility. This means that members across an organization, with correct permissions, can view artifacts linked to a registry.

Restrict visibility to a registry

Restrict who can view and access a custom registry. You can restrict visibility to a registry when you create a custom registry or after you create a custom registry. In a Restricted registry, only selected members can access the content, maintaining privacy and control. For more information about registry visibility, see Registry visibility types.

Create custom registries

Unlike the legacy Model Registry, W&B Registry is not limited to models or dataset registries. You can create custom registries tailored to specific workflows or project needs, capable of holding any arbitrary object type. This flexibility allows teams to organize and manage artifacts according to their unique requirements. For more information on how to create a custom registry, see Create a custom registry.

Custom access control

Each registry supports detailed access control, where members can be assigned specific roles such as Admin, Member, or Viewer. Admins can manage registry settings, including adding or removing members, setting roles, and configuring visibility. This ensures that teams have the necessary control over who can view, manage, and interact with the artifacts in their registries.

Terminology update

Registered models are now referred to as collections.

Summary of changes

Preparing for the migration

W&B will migrate registered models (now called collections) and associated artifact versions from the legacy Model Registry to the W&B Registry. This process will be conducted automatically, with no action required from users.

Team visibility to organization visibility

After the migration, your model registry will have organization level visibility. You can restrict who has access to a registry by assigning roles. This helps ensure that only specific members have access to specific registries.

The migration will preserve existing permission boundaries of your current team-level registered models (soon to be called collections) in the legacy W&B Model Registry. Permissions currently defined in the legacy Model Registry will be preserved in the new Registry. This means that collections currently restricted to specific team members will remain protected during and after the migration.

Artifact path continuity

No action is currently required.

During the migration

W&B will initiate the migration process. The migration will occur during a time window that minimizes disruption to W&B services. The legacy Model Registry will transition to a read-only state once the migration begins and will remain accessible for reference.

After the migration

Post-migration, collections, artifact versions, and associated attributes will be fully accessible within the new W&B Registry. The focus is on ensuring that current workflows remain intact, with ongoing support available to help navigate any changes.

Using the new registry

Users are encouraged to explore the new features and capabilities available in the W&B Registry. The Registry will not only support the functionalities currently relied upon but also introduces enhancements such as custom registries, improved visibility, and flexible access controls.

Support is available if you are interested in trying the W&B Registry early, or for new users that prefer to start with Registry and not the legacy W&B Model Registry. Contact support@wandb.com or your Sales MLE to enable this functionality. Note that any early migration will be into a BETA version. The BETA version of W&B Registry might not have all the functionality or features of the legacy Model Registry.

For more details and to learn about the full range of features in the W&B Registry, visit the W&B Registry Guide.

FAQs

Why is W&B migrating assets from Model Registry to W&B Registry?

W&B is evolving its platform to offer more advanced features and capabilities with the new Registry. This migration is a step towards providing a more integrated and powerful toolset for managing models, datasets, and other artifacts.

What needs to be done before the migration?

No action is required from users before the migration. W&B will handle the transition, ensuring that workflows and references are preserved.

Will access to model artifacts be lost?

No, access to model artifacts will be retained after the migration. The legacy Model Registry will remain in a read-only state, and all relevant data will be migrated to the new Registry.

Will metadata related to artifacts be preserved?

Yes, important metadata related to artifact creation, lineage, and other attributes will be preserved during the migration. Users will continue to have access to all relevant metadata after the migration, ensuring that the integrity and traceability of their artifacts remain intact.

Who do I contact if I need help?

Support is available for any questions or concerns. Reach out to support@wandb.com for assistance.

3.12 - Model registry

The W&B Model Registry houses a team’s trained models where ML Practitioners can publish candidates for production to be consumed by downstream teams and stakeholders. It is used to house staged/candidate models and manage workflows associated with staging.

With W&B Model Registry, you can:

• Bookmark your best model versions for each machine learning task.
• Automate downstream processes and model CI/CD.
• Move model versions through its ML lifecycle; from staging to production.
• Track a model’s lineage and audit the history of changes to production models.

How it works

Track and manage your staged models with a few simple steps.

1. Log a model version: In your training script, add a few lines of code to save the model files as an artifact to W&B.
2. Compare performance: Check live charts to compare the metrics and sample predictions from model training and validation. Identify which model version performed the best.
3. Link to registry: Bookmark the best model version by linking it to a registered model, either programmatically in Python or interactively in the W&B UI.

The following code snippet demonstrates how to log and link a model to the Model Registry:

import wandb
import random

# Start a new W&B run
run = wandb.init(project="models_quickstart")

# Simulate logging model metrics
run.log({"acc": random.random()})

# Create a simulated model file
with open("my_model.h5", "w") as f:
    f.write("Model: " + str(random.random()))

# Log and link the model to the Model Registry
run.link_model(path="./my_model.h5", registered_model_name="MNIST")

run.finish()

4. Connect model transitions to CI/CD workflows: transition candidate models through workflow stages and automate downstream actions with webhooks.

How to get started

Depending on your use case, explore the following resources to get started with W&B Models:

• Check out the two-part video series:
  1. Logging and registering models
  2. Consuming models and automating downstream processes in the Model Registry.
• Read the models walkthrough for a step-by-step outline of the W&B Python SDK commands you could use to create, track, and use a dataset artifact.
• Learn about:
  • Protected models and access control.
  • How to connect Registry to CI/CD processes.
  • Set up Slack notifications when a new model version is linked to a registered model.
• Review this report on how the Model Registry fits into your ML workflow and the benefits of using one for model management.
• Take the W&B Enterprise Model Management course and learn how to:
  • Use the W&B Model Registry to manage and version your models, track lineage, and promote models through different lifecycle stages
  • Automate your model management workflows using webhooks.
  • See how the Model Registry integrates with external ML systems and tools in your model development lifecycle for model evaluation, monitoring, and deployment.

3.12.1 - Tutorial: Use W&B for model management

The following walkthrough shows you how to log a model to W&B. By the end of the walkthrough you will:

• Create and train a model with the MNIST dataset and the Keras framework.
• Log the model that you trained to a W&B project
• Mark the dataset used as a dependency to the model you created
• Link the model to the W&B Registry.
• Evaluate the performance of the model you link to the registry
• Mark a model version ready for production.

Setting up

Before you get started, import the Python dependencies required for this walkthrough:

import wandb
import numpy as np
from tensorflow import keras
from tensorflow.keras import layers
from wandb.integration.keras import WandbMetricsLogger
from sklearn.model_selection import train_test_split

Provide your W&B entity to the entity variable:

entity = "<entity>"

Create a dataset artifact

First, create a dataset. The proceeding code snippet creates a function that downloads the MNIST dataset:

def generate_raw_data(train_size=6000):
    eval_size = int(train_size / 6)
    (x_train, y_train), (x_eval, y_eval) = keras.datasets.mnist.load_data()

    x_train = x_train.astype("float32") / 255
    x_eval = x_eval.astype("float32") / 255
    x_train = np.expand_dims(x_train, -1)
    x_eval = np.expand_dims(x_eval, -1)

    print("Generated {} rows of training data.".format(train_size))
    print("Generated {} rows of eval data.".format(eval_size))

    return (x_train[:train_size], y_train[:train_size]), (
        x_eval[:eval_size],
        y_eval[:eval_size],
    )

# Create dataset
(x_train, y_train), (x_eval, y_eval) = generate_raw_data()

Next, upload the dataset to W&B. To do this, create an artifact object and add the dataset to that artifact.

project = "model-registry-dev"

model_use_case_id = "mnist"
job_type = "build_dataset"

# Initialize a W&B run
run = wandb.init(entity=entity, project=project, job_type=job_type)

# Create W&B Table for training data
train_table = wandb.Table(data=[], columns=[])
train_table.add_column("x_train", x_train)
train_table.add_column("y_train", y_train)
train_table.add_computed_columns(lambda ndx, row: {"img": wandb.Image(row["x_train"])})

# Create W&B Table for eval data
eval_table = wandb.Table(data=[], columns=[])
eval_table.add_column("x_eval", x_eval)
eval_table.add_column("y_eval", y_eval)
eval_table.add_computed_columns(lambda ndx, row: {"img": wandb.Image(row["x_eval"])})

# Create an artifact object
artifact_name = "{}_dataset".format(model_use_case_id)
artifact = wandb.Artifact(name=artifact_name, type="dataset")

# Add wandb.WBValue obj to the artifact.
artifact.add(train_table, "train_table")
artifact.add(eval_table, "eval_table")

# Persist any changes made to the artifact.
artifact.save()

# Tell W&B this run is finished.
run.finish()

Train a model

Train a model with the artifact dataset you created in the previous step.

Declare dataset artifact as an input to the run

Declare the dataset artifact you created in a previous step as the input to the W&B run. This is particularly useful in the context of logging models because declaring an artifact as an input to a run lets you track the dataset (and the version of the dataset) used to train a specific model. W&B uses the information collected to create a lineage map.

Use the use_artifact API to both declare the dataset artifact as the input of the run and to retrieve the artifact itself.

job_type = "train_model"
config = {
    "optimizer": "adam",
    "batch_size": 128,
    "epochs": 5,
    "validation_split": 0.1,
}

# Initialize a W&B run
run = wandb.init(project=project, job_type=job_type, config=config)

# Retrieve the dataset artifact
version = "latest"
name = "{}:{}".format("{}_dataset".format(model_use_case_id), version)
artifact = run.use_artifact(artifact_or_name=name)

# Get specific content from the dataframe
train_table = artifact.get("train_table")
x_train = train_table.get_column("x_train", convert_to="numpy")
y_train = train_table.get_column("y_train", convert_to="numpy")

For more information about tracking the inputs and output of a model, see Create model lineage map.

Define and train model

For this walkthrough, define a 2D Convolutional Neural Network (CNN) with Keras to classify images from the MNIST dataset.

# Store values from our config dictionary into variables for easy accessing
num_classes = 10
input_shape = (28, 28, 1)
loss = "categorical_crossentropy"
optimizer = run.config["optimizer"]
metrics = ["accuracy"]
batch_size = run.config["batch_size"]
epochs = run.config["epochs"]
validation_split = run.config["validation_split"]

# Create model architecture
model = keras.Sequential(
    [
        layers.Input(shape=input_shape),
        layers.Conv2D(32, kernel_size=(3, 3), activation="relu"),
        layers.MaxPooling2D(pool_size=(2, 2)),
        layers.Conv2D(64, kernel_size=(3, 3), activation="relu"),
        layers.MaxPooling2D(pool_size=(2, 2)),
        layers.Flatten(),
        layers.Dropout(0.5),
        layers.Dense(num_classes, activation="softmax"),
    ]
)
model.compile(loss=loss, optimizer=optimizer, metrics=metrics)

# Generate labels for training data
y_train = keras.utils.to_categorical(y_train, num_classes)

# Create training and test set
x_t, x_v, y_t, y_v = train_test_split(x_train, y_train, test_size=0.33)

# Train the model
model.fit(
    x=x_t,
    y=y_t,
    batch_size=batch_size,
    epochs=epochs,
    validation_data=(x_v, y_v),
    callbacks=[WandbCallback(log_weights=True, log_evaluation=True)],
)

# Save model locally
path = "model.h5"
model.save(path)

Log and link a model to the Model Registry

Use the link_model API to log model one ore more files to a W&B run and link it to the W&B Model Registry.

path = "./model.h5"
registered_model_name = "MNIST-dev"

run.link_model(path=path, registered_model_name=registered_model_name)
run.finish()

W&B creates a registered model for you if the name you specify for registered-model-name does not already exist.

See link_model in the API Reference guide for more information on optional parameters.

Evaluate the performance of a model

It is common practice to evaluate the performance of a one or more models.

First, get the evaluation dataset artifact stored in W&B in a previous step.

job_type = "evaluate_model"

# Initialize a run
run = wandb.init(project=project, entity=entity, job_type=job_type)

model_use_case_id = "mnist"
version = "latest"

# Get dataset artifact, mark it as a dependency
artifact = run.use_artifact(
    "{}:{}".format("{}_dataset".format(model_use_case_id), version)
)

# Get desired dataframe
eval_table = artifact.get("eval_table")
x_eval = eval_table.get_column("x_eval", convert_to="numpy")
y_eval = eval_table.get_column("y_eval", convert_to="numpy")

Download the model version from W&B that you want to evaluate. Use the use_model API to access and download your model.

alias = "latest"  # alias
name = "mnist_model"  # name of the model artifact

# Access and download model. Returns path to downloaded artifact
downloaded_model_path = run.use_model(name=f"{name}:{alias}")

Load the Keras model and compute the loss:

model = keras.models.load_model(downloaded_model_path)

y_eval = keras.utils.to_categorical(y_eval, 10)
(loss, _) = model.evaluate(x_eval, y_eval)
score = (loss, _)

Finally, log the loss metric to the W&B run:

# # Log metrics, images, tables, or any data useful for evaluation.
run.log(data={"loss": (loss, _)})

Promote a model version

Mark a model version ready for the next stage of your machine learning workflow with a model alias. Each registered model can have one or more model aliases. A model alias can only belong to a single model version at a time.

For example, suppose that after evaluating a model’s performance, you are confident that the model is ready for production. To promote that model version, add the production alias to that specific model version.

You can add an alias to a model version interactively with the W&B App UI or programmatically with the Python SDK. The following steps show how to add an alias with the W&B Model Registry App:

1. Navigate to the Model Registry App at https://wandb.ai/registry/model.
2. Click View details next to the name of your registered model.
3. Within the Versions section, click the View button next to the name of the model version you want to promote.
4. Next to the Aliases field, click the plus icon (+).
5. Type in production into the field that appears.
6. Press Enter on your keyboard.

3.12.2 - Model Registry Terms and Concepts

The following terms describe key components of the W&B Model Registry: model version, model artifact, and registered model.

Model version

A model version represents a single model checkpoint. Model versions are a snapshot at a point in time of a model and its files within an experiment.

A model version is an immutable directory of data and metadata that describes a trained model. W&B suggests that you add files to your model version that let you store (and restore) your model architecture and learned parameters at a later date.

A model version belongs to one, and only one, model artifact. A model version can belong to zero or more, registered models. Model versions are stored in a model artifact in the order they are logged to the model artifact. W&B automatically creates a new model version if it detects that a model you log (to the same model artifact) has different contents than a previous model version.

Store files within model versions that are produced from the serialization process provided by your modeling library (for example, PyTorch and Keras).

Model alias

Model aliases are mutable strings that allow you to uniquely identify or reference a model version in your registered model with a semantically related identifier. You can only assign an alias to one version of a registered model. This is because an alias should refer to a unique version when used programmatically. It also allows aliases to be used to capture a model’s state (champion, candidate, production).

It is common practice to use aliases such as "best", "latest", "production", or "staging" to mark model versions with special purposes.

For example, suppose you create a model and assign it a "best" alias. You can refer to that specific model with run.use_model

import wandb
run = wandb.init()
name = f"{entity/project/model_artifact_name}:{alias}"
run.use_model(name=name)

Model tags

Model tags are keywords or labels that belong to one or more registered models.

Use model tags to organize registered models into categories and to search over those categories in the Model Registry’s search bar. Model tags appear at the top of the Registered Model Card. You might choose to use them to group your registered models by ML task, owning team, or priority. The same model tag can be added to multiple registered models to allow for grouping.

Model artifact

A model artifact is a collection of logged model versions. Model versions are stored in a model artifact in the order they are logged to the model artifact.

A model artifact can contain one or more model versions. A model artifact can be empty if no model versions are logged to it.

For example, suppose you create a model artifact. During model training, you periodically save your model during checkpoints. Each checkpoint corresponds to its own model version. All of the model versions created during your model training and checkpoint saving are stored in the same model artifact you created at the beginning of your training script.

The proceeding image shows a model artifact that contains three model versions: v0, v1, and v2.

View an example model artifact here.

Registered model

A registered model is a collection of pointers (links) to model versions. You can think of a registered model as a folder of “bookmarks” of candidate models for the same ML task. Each “bookmark” of a registered model is a pointer to a model version that belongs to a model artifact. You can use model tags to group your registered models.

Registered models often represent candidate models for a single modeling use case or task. For example, you might create registered model for different image classification task based on the model you use: ImageClassifier-ResNet50, ImageClassifier-VGG16, DogBreedClassifier-MobileNetV2 and so on. Model versions are assigned version numbers in the order in which they were linked to the registered model.

View an example Registered Model here.

3.12.3 - Track a model

Track a model, the model’s dependencies, and other information relevant to that model with the W&B Python SDK.

Under the hood, W&B creates a lineage of model artifact that you can view with the W&B App UI or programmatically with the W&B Python SDK. See the Create model lineage map for more information.

How to log a model

Use the run.log_model API to log a model. Provide the path where your model files are saved to the path parameter. The path can be a local file, directory, or reference URI to an external bucket such as s3://bucket/path.

Optionally provide a name for the model artifact for the name parameter. If name is not specified, W&B uses the basename of the input path prepended with the run ID.

Copy and paste the proceeding code snippet. Ensure to replace values enclosed in <> with your own.

import wandb

# Initialize a W&B run
run = wandb.init(project="<project>", entity="<entity>")

# Log the model
run.log_model(path="<path-to-model>", name="<name>")

import os
import wandb
from tensorflow import keras
from tensorflow.keras import layers

config = {"optimizer": "adam", "loss": "categorical_crossentropy"}

# Initialize a W&B run
run = wandb.init(entity="charlie", project="mnist-project", config=config)

# Training algorithm
loss = run.config["loss"]
optimizer = run.config["optimizer"]
metrics = ["accuracy"]
num_classes = 10
input_shape = (28, 28, 1)

model = keras.Sequential(
    [
        layers.Input(shape=input_shape),
        layers.Conv2D(32, kernel_size=(3, 3), activation="relu"),
        layers.MaxPooling2D(pool_size=(2, 2)),
        layers.Conv2D(64, kernel_size=(3, 3), activation="relu"),
        layers.MaxPooling2D(pool_size=(2, 2)),
        layers.Flatten(),
        layers.Dropout(0.5),
        layers.Dense(num_classes, activation="softmax"),
    ]
)

model.compile(loss=loss, optimizer=optimizer, metrics=metrics)

# Save model
model_filename = "model.h5"
local_filepath = "./"
full_path = os.path.join(local_filepath, model_filename)
model.save(filepath=full_path)

# Log the model
run.log_model(path=full_path, name="MNIST")

# Explicitly tell W&B to end the run.
run.finish()

3.12.4 - Create a registered model

Create a registered model to hold all the candidate models for your modeling tasks. You can create a registered model interactively within the Model Registry or programmatically with the Python SDK.

Programmatically create registered a model

Programmatically register a model with the W&B Python SDK. W&B automatically creates a registered model for you if the registered model doesn’t exist.

Ensure to replace other the values enclosed in <> with your own:

import wandb

run = wandb.init(entity="<entity>", project="<project>")
run.link_model(path="<path-to-model>", registered_model_name="<registered-model-name>")
run.finish()

The name you provide for registered_model_name is the name that appears in the Model Registry App.

Interactively create a registered model

Interactively create a registered model within the Model Registry App.

1. Navigate to the Model Registry App at https://wandb.ai/registry/model.
2. Click the New registered model button located in the top right of the Model Registry page.
3. From the panel that appears, select the entity you want the registered model to belong to from the Owning Entity dropdown.
4. Provide a name for your model in the Name field.
5. From the Type dropdown, select the type of artifacts to link to the registered model.
6. (Optional) Add a description about your model in the Description field.
7. (Optional) Within the Tags field, add one or more tags.
8. Click Register model.

3.12.5 - Link a model version

Link a model version to a registered model with the W&B App or programmatically with the Python SDK.

Programmatically link a model

Use the link_model method to programmatically log model files to a W&B run and link it to the W&B Model Registry.

Ensure to replace other the values enclosed in <> with your own:

import wandb

run = wandb.init(entity="<entity>", project="<project>")
run.link_model(path="<path-to-model>", registered_model_name="<registered-model-name>")
run.finish()

W&B creates a registered model for you if the name you specify for the registered-model-name parameter does not already exist.

For example, suppose you have an existing registered model named “Fine-Tuned-Review-Autocompletion”(registered-model-name="Fine-Tuned-Review-Autocompletion") in your Model Registry. And suppose that a few model versions are linked to it: v0, v1, v2. If you programmatically link a new model and use the same registered model name (registered-model-name="Fine-Tuned-Review-Autocompletion"), W&B links this model to the existing registered model and assigns it a model version v3. If no registered model with this name exists, a new one registered model is created and it will have a model version v0.

See an example “Fine-Tuned-Review-Autocompletion” registered model here.

Interactively link a model

Interactively link a model with the Model Registry or with the Artifact browser.

• Model Registry
• Artifact browser

1. Navigate to the Model Registry App at https://wandb.ai/registry/model.
2. Hover your mouse next to the name of the registered model you want to link a new model to.
3. Select the meatball menu icon (three horizontal dots) next to View details.
4. From the dropdown, select Link new version.
5. From the Project dropdown, select the name of the project that contains your model.
6. From the Model Artifact dropdown, select the name of the model artifact.
7. From the Version dropdown, select the model version you want to link to the registered model.

1. Navigate to your project’s artifact browser on the W&B App at: https://wandb.ai/<entity>/<project>/artifacts
2. Select the Artifacts icon on the left sidebar.
3. Click on the model version you want to link to your registry.
4. Within the Version overview section, click the Link to registry button.
5. From the modal that appears on the right of the screen, select a registered model from the Select a register model menu dropdown.
6. Click Next step.
7. (Optional) Select an alias from the Aliases dropdown.
8. Click Link to registry.

View the source of linked models

There are two ways to view the source of linked models: The artifact browser within the project that the model is logged to and the W&B Model Registry.

A pointer connects a specific model version in the model registry to the source model artifact (located within the project the model is logged to). The source model artifact also has a pointer to the model registry.

• Model Registry
• Artifact browser

1. Navigate to your model registry at https://wandb.ai/registry/model.
2. Select View details next the name of your registered model.
3. Within the Versions section, select View next to the model version you want to investigate.
4. Click on the Version tab within the right panel.
5. Within the Version overview section there is a row that contains a Source Version field. The Source Version field shows both the name of the model and the model’s version.

For example, the following image shows a v0 model version called mnist_model (see Source version field mnist_model:v0), linked to a registered model called MNIST-dev.

1. Navigate to your project’s artifact browser on the W&B App at: https://wandb.ai/<entity>/<project>/artifacts
2. Select the Artifacts icon on the left sidebar.
3. Expand the model dropdown menu from the Artifacts panel.
4. Select the name and version of the model linked to the model registry.
5. Click on the Version tab within the right panel.
6. Within the Version overview section there is a row that contains a Linked To field. The Linked To field shows both the name of the registered model and the version it possesses(registered-model-name:version).

For example, in the following image, there is a registered model called MNIST-dev (see the Linked To field). A model version called mnist_model with a version v0(mnist_model:v0) points to the MNIST-dev registered model.

3.12.6 - Organize models

Use model tags to organize registered models into categories and to search over those categories.

1. Navigate to the W&B Model Registry app at https://wandb.ai/registry/model.

2. Select View details next to the name of the registered model you want to add a model tag to.

3. Scroll to the Model card section.

4. Click the plus button (+) next to the Tags field.

5. Type in the name for your tag or search for a pre-existing model tag. For example. the following image shows multiple model tags added to a registered model called FineTuned-Review-Autocompletion:

   

3.12.7 - Create model lineage map

This page describes creating lineage graphs in the legacy W&B Model Registry. To learn about lineage graphs in W&B Registry, refer to Create and view lineage maps.

A useful feature of logging model artifacts to W&B are lineage graphs. Lineage graphs show artifacts logged by a run as well as artifacts used by specific run.

This means that, when you log a model artifact, you at a minimum have access to view the W&B run that used or produced the model artifact. If you track a dependency, you also see the inputs used by the model artifact.

For example, the proceeding image shows artifacts created and used throughout an ML experiment:

From left to right, the image shows:

1. The jumping-monkey-1 W&B run created the mnist_dataset:v0 dataset artifact.
2. The vague-morning-5 W&B run trained a model using the mnist_dataset:v0 dataset artifact. The output of this W&B run was a model artifact called mnist_model:v0.
3. A run called serene-haze-6 used the model artifact (mnist_model:v0) to evaluate the model.

Track an artifact dependency

Declare an dataset artifact as an input to a W&B run with the use_artifact API to track a dependency.

The proceeding code snippet shows how to use the use_artifact API:

# Initialize a run
run = wandb.init(project=project, entity=entity)

# Get artifact, mark it as a dependency
artifact = run.use_artifact(artifact_or_name="name", aliases="<alias>")

Once you have retrieved your artifact, you can use that artifact to (for example), evaluate the performance of a model.

job_type = "train_model"

config = {
    "optimizer": "adam",
    "batch_size": 128,
    "epochs": 5,
    "validation_split": 0.1,
}

run = wandb.init(project=project, job_type=job_type, config=config)

version = "latest"
name = "{}:{}".format("{}_dataset".format(model_use_case_id), version)

artifact = run.use_artifact(name)

train_table = artifact.get("train_table")
x_train = train_table.get_column("x_train", convert_to="numpy")
y_train = train_table.get_column("y_train", convert_to="numpy")

# Store values from our config dictionary into variables for easy accessing
num_classes = 10
input_shape = (28, 28, 1)
loss = "categorical_crossentropy"
optimizer = run.config["optimizer"]
metrics = ["accuracy"]
batch_size = run.config["batch_size"]
epochs = run.config["epochs"]
validation_split = run.config["validation_split"]

# Create model architecture
model = keras.Sequential(
    [
        layers.Input(shape=input_shape),
        layers.Conv2D(32, kernel_size=(3, 3), activation="relu"),
        layers.MaxPooling2D(pool_size=(2, 2)),
        layers.Conv2D(64, kernel_size=(3, 3), activation="relu"),
        layers.MaxPooling2D(pool_size=(2, 2)),
        layers.Flatten(),
        layers.Dropout(0.5),
        layers.Dense(num_classes, activation="softmax"),
    ]
)
model.compile(loss=loss, optimizer=optimizer, metrics=metrics)

# Generate labels for training data
y_train = keras.utils.to_categorical(y_train, num_classes)

# Create training and test set
x_t, x_v, y_t, y_v = train_test_split(x_train, y_train, test_size=0.33)

# Train the model
model.fit(
    x=x_t,
    y=y_t,
    batch_size=batch_size,
    epochs=epochs,
    validation_data=(x_v, y_v),
    callbacks=[WandbCallback(log_weights=True, log_evaluation=True)],
)

# Save model locally
path = "model.h5"
model.save(path)

path = "./model.h5"
registered_model_name = "MNIST-dev"
name = "mnist_model"

run.link_model(path=path, registered_model_name=registered_model_name, name=name)
run.finish()

3.12.8 - Document machine learning model

Add a description to the model card of your registered model to document aspects of your machine learning model. Some topics worth documenting include:

• Summary: A summary of what the model is. The purpose of the model. The machine learning framework the model uses, and so forth.
• Training data: Describe the training data used, processing done on the training data set, where is that data stored and so forth.
• Architecture: Information about the model architecture, layers, and any specific design choices.
• Deserialize the model: Provide information on how someone on your team can load the model into memory.
• Task: The specific type of task or problem that the machine learning model is designed to perform. It’s a categorization of the model’s intended capability.
• License: The legal terms and permissions associated with the use of the machine learning model. It helps model users understand the legal framework under which they can utilize the model.
• References: Citations or references to relevant research papers, datasets, or external resources.
• Deployment: Details on how and where the model is deployed and guidance on how the model is integrated into other enterprise systems, such as a workflow orchestration platforms.

Add a description to the model card

1. Navigate to the W&B Model Registry app at https://wandb.ai/registry/model.
2. Select View details next to the name of the registered model you want to create a model card for.
3. Go to the Model card section.
4. Within the Description field, provide information about your machine learning model. Format text within a model card with Markdown markup language.

For example, the following images shows the model card of a Credit-card Default Prediction registered model.

3.12.9 - Download a model version

Use the W&B Python SDK to download a model artifact that you linked to the Model Registry.

Replace values within <> with your own:

import wandb

# Initialize a run
run = wandb.init(project="<project>", entity="<entity>")

# Access and download model. Returns path to downloaded artifact
downloaded_model_path = run.use_model(name="<your-model-name>")

Reference a model version with one of following formats listed:

• latest - Use latest alias to specify the model version that is most recently linked.
• v# - Use v0, v1, v2, and so on to fetch a specific version in the Registered Model
• alias - Specify the custom alias that you and your team assigned to your model version