BYOB (Secure storage connector)
Bring your own bucket (BYOB) allows you to store artifacts and other sensitive data for Dedicated Cloud deployments or Self-Managed instances. For Dedicated Cloud deployments, data that you store in your bucket is not stored in the W&B managed infrastructure.
- Communication between W&B and your buckets occurs using pre-signed URLs.
- W&B uses a garbage collection process to delete W&B Artifacts. For more information, see Deleting Artifacts.
Configuration options
There are two types of levels you can configure your storage bucket: at the Instance level or at a Team level.
- Instance level: Any user that has relevant permissions within your organization can access files stored in your instance level storage bucket.
- Team level: Members of a W&B Team can access files stored in the bucket configured at the Team level. Team level storage buckets allow greater data access control and data isolation for teams with highly sensitive data or strict compliance requirements.
You can configure your bucket at both the instance level and separately for one or more teams within your organization.
For example, suppose you have a team called Kappa in your organization. Your organization (and Team Kappa) use the Instance level storage bucket by default. Next, you create a team called Omega. When you create Team Omega, you configure a Team level storage bucket for that team. Files generated by Team Omega are not accessible by Team Kappa. However, files created by Team Kappa are accessible by Team Omega. If you want to isolate data for Team Kappa, you must configure a Team level storage bucket for them as well.
Team level storage bucket provides the same benefits for Self-Managed instances, especially when different business units and departments share an instance to efficiently utilize the infrastructure and administrative resources. This also applies to firms that have separate project teams managing AI workflows for separate customer engagements.
Availability matrix
The following table shows the availability of BYOB across different W&B Server deployment types. An X means the feature is available on the specific deployment type.
| W&B Server deployment type | Instance level | Team level | Additional information |
|---|---|---|---|
| Dedicated Cloud | X | X | Both the instance and team level BYOB are available for Amazon Web Services and Google Cloud Platform. Only instance level BYOB is available for Azure cloud. |
| SaaS Cloud | X | The team level BYOB is available only for Amazon Web Services and Google Cloud Platform. W&B fully manages the default and only storage bucket for Azure cloud. | |
| Self-managed | X | X | Instance level BYOB is the default since the deployment is fully managed by you. It is also possible to use a S3-compatible secure storage solution like MinIO. |
For Dedicated Cloud and Self-managed instances on Azure, it is possible to use non-Azure storage buckets at team level using the environment variable called SUPPORTED_FILE_STORES. Reach out to W&B Support at support@wandb.com for more information.
Configure your storage object
Based on your use case, configure a storage bucket at the Team level or at the Instance level.
Only system administrators have the permissions to configure an storage object.
- Team level
- Instance level
Configure a cloud storage bucket at the Team level when you create a W&B Team:
- Provide a name for your team in the Team Name field.
- Choose the Company or Organization you want this team to belong to from the Company/Organization dropdown.
- Select External Storage for the Choose storage type option.
- Choose either New bucket from the dropdown or select an existing bucket.tip
Multiple W&B Teams can use the same cloud storage bucket. To enable this, select an existing cloud storage bucket from the dropdown.
- From the Cloud provider dropdown, select your cloud provider.
- Provide the name of your storage object for the Name field.
- (Optional if you use AWS) Provide the ARN of your encryption key for the KMS key ARN field.
- Select the Create Team button.
An error or warning appears at the bottom of the page if there are issues accessing the bucket or the bucket has invalid settings.
Reach out to W&B Support at support@wandb.com to configure Instance level BYOB for your Dedicated Cloud or Self-managed instance.