Skip to main content

Manage artifact data retention

Try in Colab

Schedule when artifacts are deleted from W&B with W&B Artifact time-to-live (TTL) policy. When you delete an artifact, W&B marks that artifact as a soft-delete. In other words, the artifact is marked for deletion but files are not immediately deleted from storage. For more information on how W&B deletes artifacts, see the Delete artifacts page.

Check out this video tutorial to learn how to manage data retention with Artifacts TTL in the W&B App.

note

W&B deactivates the option to set a TTL policy for model artifacts linked to the Model Registry. This is to help ensure that linked models do not accidentally expire if used in production workflows.

info
  • Only team admins can view a team's settings and access team level TTL settings such as (1) permitting who can set or edit a TTL policy or (2) setting a team default TTL.
  • If you do not see the option to set or edit a TTL policy in an artifact's details in the W&B App UI or if setting a TTL programmatically does not successfully change an artifact's TTL property, your team admin has not given you permissions to do so.

Auto-generated Artifacts​

Only user-generated artifacts can use TTL policies. Artifacts auto-generated by W&B cannot have TTL policies set for them.

The following Artifact types indicate an auto-generated Artifact:

  • run_table
  • code
  • job
  • Any Artifact type starting with: wandb-*

You can check an Artifact's type on the W&B platform or programmatically:

import wandb
run = wandb.init(project="<my-project-name>")
artifact = run.use_artifact(artifact_or_name="<my-artifact-name>")
print(artifact.type)

Replace the values enclosed with <> with your own.

Define who can edit and set TTL policies​

Define who can set and edit TTL policies within a team. You can either grant TTL permissions only to team admins, or you can grant both team admins and team members TTL permissions.

info

Only team admins can define who can set or edit a TTL policy.

  1. Navigate to your team’s profile page.
  2. Select the Settings tab.
  3. Navigate to the Artifacts time-to-live (TTL) section.
  4. From the TTL permissions dropdown, select who can set and edit TTL policies.
  5. Click on Review and save settings.
  6. Confirm the changes and select Save settings.

Create a TTL policy​

Set a TTL policy for an artifact either when you create the artifact or retroactively after the artifact is created.

For all the code snippets below, replace the content wrapped in <> with your information to use the code snippet.

Set a TTL policy when you create an artifact​

Use the W&B Python SDK to define a TTL policy when you create an artifact. TTL policies are typically defined in days.

tip

Defining a TTL policy when you create an artifact is similar to how you normally create an artifact. With the exception that you pass in a time delta to the artifact's ttl attribute.

The steps are as follows:

  1. Create an artifact.
  2. Add content to the artifact such as files, a directory, or a reference.
  3. Define a TTL time limit with the datetime.timedelta data type that is part of Python's standard library.
  4. Log the artifact.

The following code snippet demonstrates how to create an artifact and set a TTL policy.

import wandb
from datetime import timedelta

run = wandb.init(project="<my-project-name>", entity="<my-entity>")
artifact = wandb.Artifact(name="<artifact-name>", type="<type>")
artifact.add_file("<my_file>")

artifact.ttl = timedelta(days=30)  # Set TTL policy
run.log_artifact(artifact)

The preceding code snippet sets the TTL policy for the artifact to 30 days. In other words, W&B deletes the artifact after 30 days.

Set or edit a TTL policy after you create an artifact​

Use the W&B App UI or the W&B Python SDK to define a TTL policy for an artifact that already exists.

note

When you modify an artifact's TTL, the time the artifact takes to expire is still calculated using the artifact's createdAt timestamp.

  1. Fetch your artifact.
  2. Pass in a time delta to the artifact's ttl attribute.
  3. Update the artifact with the save method.

The following code snippet shows how to set a TTL policy for an artifact:

import wandb
from datetime import timedelta

artifact = run.use_artifact("<my-entity/my-project/my-artifact:alias>")
artifact.ttl = timedelta(days=365 * 2)  # Delete in two years
artifact.save()

The preceding code example sets the TTL policy to two years.

Set default TTL policies for a team​

info

Only team admins can set a default TTL policy for a team.

Set a default TTL policy for your team. Default TTL policies apply to all existing and future artifacts based on their respective creation dates. Artifacts with existing version-level TTL policies are not affected by the team's default TTL.

  1. Navigate to your team’s profile page.
  2. Select the Settings tab.
  3. Navigate to the Artifacts time-to-live (TTL) section.
  4. Click on the Set team's default TTL policy.
  5. Within the Duration field, set the TTL policy in units of days.
  6. Click on Review and save settings. 7/ Confirm the changes and then select Save settings.

Deactivate a TTL policy​

Use the W&B Python SDK or W&B App UI to deactivate a TTL policy for a specific artifact version.

  1. Fetch your artifact.
  2. Set the artifact's ttl attribute to None.
  3. Update the artifact with the save method.

The following code snippet shows how to turn off a TTL policy for an artifact:

artifact = run.use_artifact("<my-entity/my-project/my-artifact:alias>")
artifact.ttl = None
artifact.save()

View TTL policies​

View TTL policies for artifacts with the Python SDK or with the W&B App UI.

Use a print statement to view an artifact's TTL policy. The following example shows how to retrieve an artifact and view its TTL policy:

artifact = run.use_artifact("<my-entity/my-project/my-artifact:alias>")
print(artifact.ttl)
On this page
Was this page helpful?👍👎