Access BYOB using pre-signed URLs

W&B uses pre-signed URLs to simplify access to blob storage from your AI workloads or user browsers. For basic information on pre-signed URLs, refer to the cloud provider’s documentation:

• Pre-signed URLs for AWS S3, which also applies to S3-compatible storage like CoreWeave AI Object Storage.
• Signed URLs for Google Cloud Storage
• Shared Access Signature for Azure Blob Storage

How it works:

1. When needed, AI workloads or user browser clients within your network request pre-signed URLs from W&B.
2. W&B responds to the request by accessing the blob storage to generate the pre-signed URL with the required permissions.
3. W&B returns the pre-signed URL to the client.
4. The client uses the pre-signed URL to read or write to the blob storage.

A pre-signed URL expires after:

• Reading: 1 hour
• Writing: 24 hours, to allow more time to upload large objects in chunks.

Team-level access control

Each pre-signed URL is restricted to specific buckets based on team level access control in the W&B platform. If a user is part of a team which is mapped to a storage bucket using secure storage connector, and if that user is part of only that team, then the pre-signed URLs generated for their requests would not have permissions to access storage buckets mapped to other teams.

Network restriction

W&B recommends using IAM policies to restrict the networks that can use pre-signed URLs to access external storage using pre-signed URLs. This helps to ensure that your W&B specific buckets are accessed only from networks where your AI workloads are running, or from gateway IP addresses that map to your user machines.

• For CoreWeave AI Object Storage, refer to Bucket policy reference in the CoreWeave documentation.
• For AWS S3 or S3-compatible storage like MiniIO hosted on your premises, refer to the S3 userguide, the MinIO documentation, or the documentation for your S3-compatible storage provider.

Audit logs

W&B recommends using W&B audit logs together with blob storage specific audit logs. For blob storage audit logs, refer to the documentation for each cloud provider:

• CoreWeave audit logs
• AWS S3 access logs
• Google Cloud Storage audit logs
• Monitor Azure blob storage.

Admin and security teams can use audit logs to keep track of which user is doing what in the W&B product and take necessary action if they determine that some operations need to be limited for certain users.

Determine the user that requested a pre-signed URL

When W&B returns a pre-signed URL, a query parameter in the URL contains the requester’s username: