Data Types

• 1: Audio
• 2: box3d()
• 3: Histogram
• 4: Html
• 5: Image
• 6: Molecule
• 7: Object3D
• 8: Plotly
• 9: Table
• 10: Video

import wandb
import matplotlib.pyplot as plt

# Generate an image for demonstration purposes
path_to_img = "/path/to/cafe.png"
im = plt.imread(path_to_img)

# Initialize a new run
with wandb.init(project="awesome-project") as run:

    # Log the image
    run.log({"img": [wandb.Image(im, caption="Cafe")]})

import wandb

# Initialize a new run
with wandb.init(project="visualize-predictions", name="tables") as run:

    # Create tabular data, using a list of lists
    data = [["Cat", "1", "1"],["Dog", "0", "-1"]]
    run.log({"Table 1": wandb.Table(data=data, columns=["Text", "Predicted Label", "True Label"])})

    # Create tabular data, using `wandb.Table.add_data()` method
    table = wandb.Table(columns=["Text", "Predicted Label", "True Label"])
    table.add_data("Cat", "1", "1")
    table.add_data("Dog", "0", "-1")
    run.log({"Table 2": table})

1 - Audio

class Audio

W&B class for audio clips.

method Audio.__init__

__init__(
    data_or_path: Union[str, pathlib.Path, list, ForwardRef('np.ndarray')],
    sample_rate: Optional[int] = None,
    caption: Optional[str] = None
)

Accept a path to an audio file or a numpy array of audio data.

Args:

• data_or_path: A path to an audio file or a NumPy array of audio data.
• sample_rate: Sample rate, required when passing in raw NumPy array of audio data.
• caption: Caption to display with audio.

classmethod Audio.durations

durations(audio_list)

Calculate the duration of the audio files.

classmethod Audio.sample_rates

sample_rates(audio_list)

Get sample rates of the audio files.

2 - box3d()

function box3d

box3d(
    center: 'npt.ArrayLike',
    size: 'npt.ArrayLike',
    orientation: 'npt.ArrayLike',
    color: 'RGBColor',
    label: 'Optional[str]' = None,
    score: 'Optional[numeric]' = None
) → Box3D

A 3D bounding box. The box is specified by its center, size and orientation.

Args:

• center: The center point of the box as a length-3 ndarray.
• size: The box’s X, Y and Z dimensions as a length-3 ndarray.
• orientation: The rotation transforming global XYZ coordinates into the box’s local XYZ coordinates, given as a length-4 ndarray [r, x, y, z] corresponding to the non-zero quaternion r + xi + yj + zk.
• color: The box’s color as an (r, g, b) tuple with 0 <= r,g,b <= 1.
• label: An optional label for the box.
• score: An optional score for the box. Typically used to indicate the confidence of a detection.

Returns: A Box3D object.

Example: The following example creates a point cloud with 60 boxes rotating around the X, Y and Z axes.

import wandb

import math
import numpy as np
from scipy.spatial.transform import Rotation


with wandb.init() as run:
    run.log(
         {
             "points": wandb.Object3D.from_point_cloud(
                 points=np.random.uniform(-5, 5, size=(100, 3)),
                 boxes=[
                     wandb.box3d(
                         center=(0.3 * t - 3, 0, 0),
                         size=(0.1, 0.1, 0.1),
                         orientation=Rotation.from_euler(
                             "xyz", [t * math.pi / 10, 0, 0]
                         ).as_quat(),
                         color=(0.5 + t / 40, 0.5, 0.5),
                         label=f"box {t}",
                         score=0.9,
                     )
                     for t in range(20)
                 ]
                 + [
                     wandb.box3d(
                         center=(0, 0.3 * t - 3, 0.3),
                         size=(0.1, 0.1, 0.1),
                         orientation=Rotation.from_euler(
                             "xyz", [0, t * math.pi / 10, 0]
                         ).as_quat(),
                         color=(0.5, 0.5 + t / 40, 0.5),
                         label=f"box {t}",
                         score=0.9,
                     )
                     for t in range(20)
                 ]
                 + [
                     wandb.box3d(
                         center=(0.3, 0.3, 0.3 * t - 3),
                         size=(0.1, 0.1, 0.1),
                         orientation=Rotation.from_euler(
                             "xyz", [0, 0, t * math.pi / 10]
                         ).as_quat(),
                         color=(0.5, 0.5, 0.5 + t / 40),
                         label=f"box {t}",
                         score=0.9,
                     )
                     for t in range(20)
                 ],
             ),
         }
    )

3 - Histogram

class Histogram

W&B class for histograms.

This object works just like numpy’s histogram function https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram.html

Attributes:

• bins ([float]): Edges of bins
• histogram ([int]): Number of elements falling in each bin.

method Histogram.__init__

__init__(
    sequence: Optional[Sequence] = None,
    np_histogram: Optional[ForwardRef('NumpyHistogram')] = None,
    num_bins: int = 64
) → None

Initialize a Histogram object.

Args: sequence: Input data for histogram. np_histogram: Alternative input of a precomputed histogram. num_bins: Number of bins for the histogram. The default number of bins is 64. The maximum number of bins is 512.

Examples: Generate histogram from a sequence.

import wandb

wandb.Histogram([1, 2, 3])

Efficiently initialize from np.histogram.

import numpy as np
import wandb

hist = np.histogram(data)
wandb.Histogram(np_histogram=hist)

4 - Html

class Html

W&B class for logging HTML content to W&B.

method Html.__init__

__init__(
    data: Union[str, pathlib.Path, ForwardRef('TextIO')],
    inject: bool = True,
    data_is_not_path: bool = False
) → None

Creates a W&B HTML object.

Args: data: A string that is a path to a file with the extension “.html”, or a string or IO object containing literal HTML.

• inject: Add a stylesheet to the HTML object. If set to False the HTML will pass through unchanged.
• data_is_not_path: If set to False, the data will be treated as a path to a file.

Examples: It can be initialized by providing a path to a file:

with wandb.init() as run:
    run.log({"html": wandb.Html("./index.html")})

Alternatively, it can be initialized by providing literal HTML, in either a string or IO object:

with wandb.init() as run:
    run.log({"html": wandb.Html("<h1>Hello, world!</h1>")})

5 - Image

class Image

A class for logging images to W&B.

method Image.__init__

__init__(
    data_or_path: 'ImageDataOrPathType',
    mode: Optional[str] = None,
    caption: Optional[str] = None,
    grouping: Optional[int] = None,
    classes: Optional[ForwardRef('Classes'), Sequence[dict]] = None,
    boxes: Optional[Dict[str, ForwardRef('BoundingBoxes2D')], Dict[str, dict]] = None,
    masks: Optional[Dict[str, ForwardRef('ImageMask')], Dict[str, dict]] = None,
    file_type: Optional[str] = None,
    normalize: bool = True
) → None

Initialize a wandb.Image object.

Args:

• data_or_path: Accepts NumPy array/pytorch tensor of image data, a PIL image object, or a path to an image file. If a NumPy array or pytorch tensor is provided, the image data will be saved to the given file type. If the values are not in the range [0, 255] or all values are in the range [0, 1], the image pixel values will be normalized to the range [0, 255] unless normalize is set to False.
  • pytorch tensor should be in the format (channel, height, width)
  • NumPy array should be in the format (height, width, channel)
• mode: The PIL mode for an image. Most common are “L”, “RGB”,
• "RGBA". Full explanation at https: //pillow.readthedocs.io/en/stable/handbook/concepts.html#modes
• caption: Label for display of image.
• grouping: The grouping number for the image.
• classes: A list of class information for the image, used for labeling bounding boxes, and image masks.
• boxes: A dictionary containing bounding box information for the image.
• see: https://docs.wandb.ai/ref/python/data-types/boundingboxes2d/
• masks: A dictionary containing mask information for the image.
• see: https://docs.wandb.ai/ref/python/data-types/imagemask/
• file_type: The file type to save the image as. This parameter has no effect if data_or_path is a path to an image file.
• normalize: If True, normalize the image pixel values to fall within the range of [0, 255]. Normalize is only applied if data_or_path is a numpy array or pytorch tensor.

Examples: Create a wandb.Image from a numpy array

import numpy as np
import wandb

with wandb.init() as run:
    examples = []
    for i in range(3):
         pixels = np.random.randint(low=0, high=256, size=(100, 100, 3))
         image = wandb.Image(pixels, caption=f"random field {i}")
         examples.append(image)
    run.log({"examples": examples})

Create a wandb.Image from a PILImage

import numpy as np
from PIL import Image as PILImage
import wandb

with wandb.init() as run:
    examples = []
    for i in range(3):
         pixels = np.random.randint(
             low=0, high=256, size=(100, 100, 3), dtype=np.uint8
         )
         pil_image = PILImage.fromarray(pixels, mode="RGB")
         image = wandb.Image(pil_image, caption=f"random field {i}")
         examples.append(image)
    run.log({"examples": examples})

Log .jpg rather than .png (default)

import numpy as np
import wandb

with wandb.init() as run:
    examples = []
    for i in range(3):
         pixels = np.random.randint(low=0, high=256, size=(100, 100, 3))
         image = wandb.Image(
             pixels, caption=f"random field {i}", file_type="jpg"
         )
         examples.append(image)
    run.log({"examples": examples})

property Image.image

6 - Molecule

class Molecule

W&B class for 3D Molecular data.

method Molecule.__init__

__init__(
    data_or_path: Union[str, pathlib.Path, ForwardRef('TextIO')],
    caption: Optional[str] = None,
    **kwargs: str
) → None

Initialize a Molecule object.

Args:

• data_or_path: Molecule can be initialized from a file name or an io object.
• caption: Caption associated with the molecule for display.

7 - Object3D

class Object3D

W&B class for 3D point clouds.

method Object3D.__init__

__init__(
    data_or_path: Union[ForwardRef('np.ndarray'), str, pathlib.Path, ForwardRef('TextIO'), dict],
    caption: Optional[str] = None,
    **kwargs: Optional[str, ForwardRef('FileFormat3D')]
) → None

Creates a W&B Object3D object.

Args:

• data_or_path: Object3D can be initialized from a file or a numpy array.
• caption: Caption associated with the object for display.

Examples: The shape of the numpy array must be one of either

[[x y z],       ...] nx3
[[x y z c],     ...] nx4 where c is a category with supported range [1, 14]
[[x y z r g b], ...] nx6 where is rgb is color

8 - Plotly

class Plotly

W&B class for Plotly plots.

method Plotly.__init__

__init__(
    val: Union[ForwardRef('plotly.Figure'), ForwardRef('matplotlib.artist.Artist')]
)

Initialize a Plotly object.

Args:

• val: Matplotlib or Plotly figure.

9 - Table

class Table

The Table class used to display and analyze tabular data.

Unlike traditional spreadsheets, Tables support numerous types of data: scalar values, strings, numpy arrays, and most subclasses of wandb.data_types.Media. This means you can embed Images, Video, Audio, and other sorts of rich, annotated media directly in Tables, alongside other traditional scalar values.

This class is the primary class used to generate W&B Tables https://docs.wandb.ai/guides/models/tables/.

method Table.__init__

__init__(
    columns=None,
    data=None,
    rows=None,
    dataframe=None,
    dtype=None,
    optional=True,
    allow_mixed_types=False,
    log_mode: Optional[Literal['IMMUTABLE', 'MUTABLE', 'INCREMENTAL']] = 'IMMUTABLE'
)

Initializes a Table object.

The rows is available for legacy reasons and should not be used. The Table class uses data to mimic the Pandas API.

Args:

• columns: (List[str]) Names of the columns in the table. Defaults to [“Input”, “Output”, “Expected”].
• data: (List[List[any]]) 2D row-oriented array of values.
• dataframe: (pandas.DataFrame) DataFrame object used to create the table. When set, data and columns arguments are ignored.
• rows: (List[List[any]]) 2D row-oriented array of values.
• optional: (Union[bool,List[bool]]) Determines if None values are allowed. Default to True - If a singular bool value, then the optionality is enforced for all columns specified at construction time - If a list of bool values, then the optionality is applied to each column - should be the same length as columns applies to all columns. A list of bool values applies to each respective column.
• allow_mixed_types: (bool) Determines if columns are allowed to have mixed types (disables type validation). Defaults to False
• log_mode: Optional[str] Controls how the Table is logged when mutations occur. Options: - “IMMUTABLE” (default): Table can only be logged once; subsequent logging attempts after the table has been mutated will be no-ops. - “MUTABLE”: Table can be re-logged after mutations, creating a new artifact version each time it’s logged. - “INCREMENTAL”: Table data is logged incrementally, with each log creating a new artifact entry containing the new data since the last log.

method Table.add_column

add_column(name, data, optional=False)

Adds a column of data to the table.

Args:

• name: (str) - the unique name of the column
• data: (list | np.array) - a column of homogeneous data
• optional: (bool) - if null-like values are permitted

method Table.add_computed_columns

add_computed_columns(fn)

Adds one or more computed columns based on existing data.

Args:

• fn: A function which accepts one or two parameters, ndx (int) and row (dict), which is expected to return a dict representing new columns for that row, keyed by the new column names.
  • ndx is an integer representing the index of the row. Only included if include_ndx is set to True.
  • row is a dictionary keyed by existing columns

method Table.add_data

add_data(*data)

Adds a new row of data to the table.

The maximum amount ofrows in a table is determined by wandb.Table.MAX_ARTIFACT_ROWS.

The length of the data should match the length of the table column.

method Table.add_row

add_row(*row)

Deprecated. Use Table.add_data method instead.

method Table.cast

cast(col_name, dtype, optional=False)

Casts a column to a specific data type.

This can be one of the normal python classes, an internal W&B type, or an example object, like an instance of wandb.Image or wandb.Classes.

Args:

• col_name (str): The name of the column to cast.
• dtype (class, wandb.wandb_sdk.interface._dtypes.Type, any): The target dtype.
• optional (bool): If the column should allow Nones.

method Table.get_column

get_column(name, convert_to=None)

Retrieves a column from the table and optionally converts it to a NumPy object.

Args:

• name: (str) - the name of the column
• convert_to: (str, optional) - “numpy”: will convert the underlying data to numpy object

method Table.get_dataframe

get_dataframe()

Returns a pandas.DataFrame of the table.

method Table.get_index

get_index()

Returns an array of row indexes for use in other tables to create links.

10 - Video

class Video

A class for logging videos to W&B.

method Video.__init__

__init__(
    data_or_path: Union[str, pathlib.Path, ForwardRef('np.ndarray'), ForwardRef('TextIO'), ForwardRef('BytesIO')],
    caption: Optional[str] = None,
    fps: Optional[int] = None,
    format: Optional[Literal['gif', 'mp4', 'webm', 'ogg']] = None
)

Initialize a W&B Video object.

Args:

• data_or_path: Video can be initialized with a path to a file or an io object. Video can be initialized with a numpy tensor. The numpy tensor must be either 4 dimensional or 5 dimensional. The dimensions should be (number of frames, channel, height, width) or (batch, number of frames, channel, height, width) The format parameter must be specified with the format argument when initializing with a numpy array or io object.
• caption: Caption associated with the video for display.
• fps: The frame rate to use when encoding raw video frames. Default value is 4. This parameter has no effect when data_or_path is a string, or bytes.
• format: Format of video, necessary if initializing with a numpy array or io object. This parameter will be used to determine the format to use when encoding the video data. Accepted values are “gif”, “mp4”, “webm”, or “ogg”. If no value is provided, the default format will be “gif”.

Examples: Log a numpy array as a video

import numpy as np
import wandb

with wandb.init() as run:
    # axes are (number of frames, channel, height, width)
    frames = np.random.randint(
         low=0, high=256, size=(10, 3, 100, 100), dtype=np.uint8
    )
    run.log({"video": wandb.Video(frames, format="mp4", fps=4)})