Skip to main content

Log Media & Objects

We support images, video, audio, and more. Log rich media to explore your results and visually compare your runs, models, and datasets. Read on for examples and how-to guides.

info

Looking for reference docs for our media types? You want this page.

info

You can see working code to log all of these media objects in this Colab Notebook, check out what the results look like on wandb.ai here, and follow along with a video tutorial, linked above.

Images​

Log images to track inputs, outputs, filter weights, activations, and more!

Inputs and outputs of an autoencoder network performing in-painting.

Images can be logged directly from NumPy arrays, as PIL images, or from the filesystem.

info

It's recommended to log fewer than 50 images per step to prevent logging from becoming a bottleneck during training and image loading from becoming a bottleneck when viewing results.

Provide arrays directly when constructing images manually, e.g. using make_grid from torchvision.

Arrays are converted to png using Pillow.

images = wandb.Image(image_array, caption="Top: Output, Bottom: Input")
          
wandb.log({"examples": images}

We assume the image is gray scale if the last dimension is 1, RGB if it's 3, and RGBA if it's 4. If the array contains floats, we convert them to integers between 0 and 255. If you want to normalize your images differently, you can specify the mode manually or just supply a PIL.Image, as described in the "Logging PIL Images" tab of this panel.

Image Overlays​

Log semantic segmentation masks and interact with them (altering opacity, viewing changes over time, and more) via the W&B UI.

Interactive mask viewing in the W&B UI.

To log an overlay, you'll need to provide a dictionary with the following keys and values to the masks keyword argument of wandb.Image:

  • one of two keys representing the image mask:
    • "mask_data": a 2D NumPy array containing an integer class label for each pixel
    • "path": (string) a path to a saved image mask file
  • "class_labels": (optional) a dictionary mapping the integer class labels in the image mask to their readable class names

To log multiple masks, log a mask dictionary with multiple keys, as in the code snippet below.

See a live example →

Sample code →

mask_data = np.array([[1, 2, 2, ... , 2, 2, 1], ...])

class_labels = {
  1: "tree",
  2: "car",
  3: "road"
}

mask_img = wandb.Image(image, masks={
  "predictions": {
    "mask_data": mask_data,
    "class_labels": class_labels
  },
  "ground_truth": {
    ...
  },
  ...
})

Image Overlays in Tables​

Interactive Segmentation Masks in Tables

To log Segmentation Masks in tables, you will need to provide a wandb.Image object for each row in the table.

An example is provided in the Code snippet below:

table = wandb.Table(columns=['ID', 'Image'])

for id, img, label in zip(ids, images, labels):
    mask_img = wandb.Image(img, masks = {
        "prediction" : {
            "mask_data" : label,
            "class_labels" : class_labels
        },
        ...
    })
    
    table.add_data(id, img)

wandb.log({"Table" : table})

Histograms​

If a sequence of numbers (e.g. list, array, tensor) is provided as the first argument, we will construct the histogram automatically by calling `np.histogram`. Note that all arrays/tensors are flattened. You can use the optional `num_bins` keyword argument to override the default of `64` bins. The maximum number of bins supported is `512`.

In the UI, histograms are plotted with the training step on the x-axis, the metric value on the y-axis, and the count represented by color, to ease comparison of histograms logged throughout training. See the "Histograms in Summary" tab of this panel for details on logging one-off histograms.

wandb.log({"gradients": wandb.Histogram(grads)})

Gradients for the discriminator in a GAN.

If histograms are in your summary they will appear on the Overview tab of the Run Page. If they are in your history, we plot a heatmap of bins over time on the Charts tab.

3D Visualizations​

Log files in the formats 'obj', 'gltf', 'glb', 'babylon', 'stl', 'pts.json', and we will render them in the UI when your run finishes.

wandb.log({"generated_samples":
           [wandb.Object3D(open("sample.obj")),
            wandb.Object3D(open("sample.gltf")),
            wandb.Object3D(open("sample.glb"))]})

Ground truth and prediction of a headphones point cloud

See a live example →

Other Media​

Weights & Biases also supports logging of a variety of other media types.

wandb.log(
  {"whale songs": wandb.Audio(np_array, caption="OooOoo", sample_rate=32)})

The maximum number of audio clips that can be logged per step is 100.

Frequently Asked Questions​

How can I compare images or media across epochs or steps?​

Each time you log images from a step, we save them to show in the UI. Expand the image panel, and use the step slider to look at images from different steps. This makes it easy to compare how a model's output changes during training.

What if I want to integrate W&B into my project, but I don't want to upload any images or media?​

W&B can be used even for projects that only log scalars — you specify any files or data you'd like to upload explicitly. Here's a quick example in PyTorch that does not log images.

How do I log a PNG?​

wandb.Image converts numpy arrays or instances of PILImage to PNGs by default.

wandb.log({"example": wandb.Image(...)})
# Or multiple images
wandb.log({"example": [wandb.Image(...) for img in images]})

How do I log a video?​

Videos are logged using the wandb.Video data type:

wandb.log({"example": wandb.Video("myvideo.mp4")})

Now you can view videos in the media browser. Go to your project workspace, run workspace, or report and click "Add visualization" to add a rich media panel.

How do I navigate and zoom in point clouds?​

You can hold control and use the mouse to move around inside the space.

How do I log a 2D view of a molecule?​

You can log a 2D view of a molecule using the wandb.Image data type and rdkit:

molecule = rdkit.Chem.MolFromSmiles("CC(=O)O")
rdkit.Chem.AllChem.Compute2DCoords(molecule)
rdkit.Chem.AllChem.GenerateDepictionMatching2DStructure(molecule, molecule)
pil_image = rdkit.Chem.Draw.MolToImage(molecule, size=(300, 300))

wandb.log({"acetic_acid": wandb.Image(pil_image)})
Was this page helpful?👍👎