This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Monitoring and usage

1: Track user activity with audit logs
2: Use Prometheus monitoring
3: Configure Slack alerts
4: View organization dashboard

1 - Track user activity with audit logs

Use W&B audit logs to track user activity within your organization and to conform to your enterprise governance requirements. Audit logs are available in JSON format. How to access audit logs depends on your W&B platform deployment type:

W&B Platform Deployment type	Audit logs access mechanism
Self-managed	Synced to instance-level bucket every 10 minutes. Also available using the API.
Dedicated Cloud with secure storage connector (BYOB)	Synced to instance-level bucket (BYOB) every 10 minutes. Also available using the API.
Dedicated Cloud with W&B managed storage (without BYOB)	Only available using the API.

Audit logs are not available for SaaS Cloud.

Once you’ve access to your audit logs, analyze those using your preferred tools, such as Pandas, Amazon Redshift, Google BigQuery, Microsoft Fabric, and more. You may need to transform the JSON-formatted audit logs into a format relevant to the tool before analysis. Information on how to transform your audit logs for specific tools is outside the scope of W&B documentation.

Audit Log Retention: If a compliance, security or risk team in your organization requires audit logs to be retained for a specific period of time, W&B recommends to periodically transfer the logs from your instance-level bucket to a long-term retention storage. If you’re instead using the API to access the audit logs, you can implement a simple script that runs periodically (like daily or every few days) to fetch any logs that may have been generated since the time of the last script run, and store those in a short-term storage for analysis or directly transfer to a long-term retention storage.

HIPAA compliance requires that you retain audit logs for a minimum of 6 years. For HIPAA-compliant Dedicated Cloud instances with BYOB, you must configure guardrails for your managed storage including any long-term retention storage, to ensure that no internal or external user can delete audit logs before the end of the mandatory retention period.

Audit log schema

The following table lists all the different keys that might be present in your audit logs. Each log contains only the assets relevant to the corresponding action, and others are omitted from the log.

Key	Definition
timestamp	Time stamp in RFC3339 format. For example: `2023-01-23T12:34:56Z`, represents `12:34:56 UTC` time on Jan 23, 2023.
action	What action did the user take.
actor_user_id	If present, ID of the logged-in user who performed the action.
response_code	Http response code for the action.
artifact_asset	If present, action was taken on this artifact id
artifact_sequence_asset	If present, action was taken on this artifact sequence id
entity_asset	If present, action was taken on this entity or team id.
project_asset	If present, action was taken on this project id.
report_asset	If present, action was taken on this report id.
user_asset	If present, action was taken on this user asset.
cli_version	If the action is taken via python SDK, this will contain the version
actor_ip	IP address of the logged-in user.
actor_email	if present, action was taken on this actor email.
artifact_digest	if present, action was taken on this artifact digest.
artifact_qualified_name	if present, action was taken on this artifact.
entity_name	if present, action was taken on this entity or team name.
project_name	if present, action was taken on this project name.
report_name	if present, action was taken on this report name.
user_email	if present, action was taken on this user email.

Personally identifiable information (PII), such as email ids and the names of projects, teams, and reports, is available only using the API endpoint option, and can be turned off as described below.

Fetch audit logs using API

An instance admin can fetch the audit logs for your W&B instance using the following API:

Construct the full API endpoint using a combination of the base endpoint <wandb-platform-url>/admin/audit_logs and the following URL parameters:
- numDays: logs will be fetched starting from today - numdays to most recent; defaults to 0, which returns logs only for today.
- anonymize: if set to true, remove any PII; defaults to false
Execute HTTP GET request on the constructed full API endpoint, either by directly running it within a modern browser, or by using a tool like Postman, HTTPie, cURL command or more.

If your W&B instance URL is https://mycompany.wandb.io and you would like to get audit logs without PII for user activity within the last week, you must use the API endpoint https://mycompany.wandb.io?numDays=7&anonymize=true.

Only W&B instance admins can fetch audit logs using the API. If you are not an instance admin or not logged into your organization, you get a HTTP 403 Forbidden error.

The API response contains new-line separated JSON objects. Objects will include the fields described in the schema. It’s the same format which is used when syncing audit log files to an instance-level bucket (wherever applicable as mentioned earlier). In those cases, the audit logs are located at the /wandb-audit-logs directory in your bucket.

Actions

The following table describes possible actions that can be recorded by W&B:

Action	Definition
artifact:create	Artifact is created.
artifact:delete	Artifact is deleted.
artifact:read	Artifact is read.
project:delete	Project is deleted.
project:read	Project is read.
report:read	Report is read.
run:delete	Run is deleted.
run:delete_many	Runs are deleted in batch.
run:update_many	Runs are updated in batch.
run:stop	Run is stopped.
run:undelete_many	Runs are brought back from trash in batch.
run:update	Run is updated.
sweep:create_agent	Sweep agent is created.
team:invite_user	User is invited to team.
team:create_service_account	Service account is created for the team.
team:create	Team is created.
team:uninvite	User or service account is uninvited from team.
team:delete	Team is deleted.
user:create	User is created.
user:delete_api_key	API key for the user is deleted.
user:deactivate	User is deactivated.
user:create_api_key	API key for the user is created.
user:permanently_delete	User is permanently deleted.
user:reactivate	User is reactivated.
user:update	User is updated.
user:read	User profile is read.
user:login	User logs in.
user:initiate_login	User initiates log in.
user:logout	User logs out.

2 - Use Prometheus monitoring

Use Prometheus with W&B Server. Prometheus installs are exposed as a kubernetes ClusterIP service.

Prometheus monitoring is only available with Self-managed instances.

Follow the procedure below to access your Prometheus metrics endpoint (/metrics):

Connect to the cluster with Kubernetes CLI toolkit, kubectl. See kubernetes’ Accessing Clusters documentation for more information.
Find the internal address of the cluster with:
```
kubectl describe svc prometheus
```
Start a shell session inside your container running in your Kubernetes cluster with kubectl exec. Hit the endpoint at <internal address>/metrics.

Copy the command below and execute it in your terminal and replace <internal address> with your internal address:
```
kubectl exec <internal address>/metrics
```

A test pod starts, which you can exec into just to access anything in the network:

kubectl run -it testpod --image=alpine bin/ash --restart=Never --rm

From there you can choose to keep access internal to the network or expose it yourself with a kubernetes nodeport service.

3 - Configure Slack alerts

Integrate W&B Server with Slack.

Create the Slack application

Follow the procedure below to create a Slack application.

Visit https://api.slack.com/apps and select Create an App.
Provide a name for your app in the App Name field.
Select a Slack workspace where you want to develop your app in. Ensure that the Slack workspace you use is the same workspace you intend to use for alerts.

Configure the Slack application

On the left sidebar, select OAth & Permissions.
Within the Scopes section, provide the bot with the incoming_webhook scope. Scopes give your app permission to perform actions in your development workspace.

For more information about OAuth scopes for Bots, see the Understanding OAuth scopes for Bots tutorial in the Slack API documentation.
Configure the Redirect URL to point to your W&B installation. Use the same URL that your host URL is set to in your local system settings. You can specify multiple URLs if you have different DNS mappings to your instance.
Select Save URLs.
You can optionally specify an IP range under Restrict API Token Usage, allow-list the IP or IP range of your W&B instances. Limiting the allowed IP address helps further secure your Slack application.

Register your Slack application with W&B

Navigate to the System Settings or System Console page of your W&B instance, depending on your deployment
Depending on the System page you are on follow one of the below options:
- If you are in the System Console: go to Settings then to Notifications
- If you are in the System Settings: toggle the Enable a custom Slack application to dispatch alerts to enable a custom Slack application
Supply your Slack client ID and Slack secret then click Save. Navigate to Basic Information in Settings to find your application’s client ID and secret.
Verify that everything is working by setting up a Slack integration in the W&B app.

4 - View organization dashboard

Organization dashboard is only available with Dedicated Cloud and Self-managed instances.

View organization usage of W&B

Use the organization dashboard to get a holistic view of users that belong to your organization, how users of your organization use W&B, along with properties such as:

Name: The name of the user and their W&B username.
Last active: The time the user last used W&B. This includes any activity that requires authentication, including viewing pages in the product, logging runs or taking any other action, or logging in.
Role: The role of the user.
Email: The email of the user.
Team: The names of teams the user belongs to.

View the status of a user

The Last Active column shows if a user is pending an invitation or an active user. A user is one of three states:

Invite pending: Admin has sent invite but user has not accepted invitation.
Active: User has accepted the invite and created an account.
Deactivated: Admin has revoked access of the user.

View how your organization uses W&B in CSV format.

Select the three dots next to the Add user button.
From the dropdown, select Export as CSV.

This exports a CSV file that lists all users of an organization along with details about the user, such as their user name, time stamp of when they were last active, roles, email, and more.

View user activity

Use the Last Active column to get an Activity summary of an individual user.

Hover your mouse over the Last Active entry for a user.
A tooltip appears and provides a summary of information about the user’s activity.

A user is active if they:

log in to W&B.
view any page in the W&B App.
log runs.
use the SDK to track an experiment.
interact with the W&B Server in any way.

View active users over time

Use the Users active over time plot in the Organization dashboard to get an aggregate overview of how many users are active over time (right most plot in image below).

You can use the dropdown menu to filter results based on days, months, or all time.