This is the multi-page printable view of this section.
Click here to print.
Return to the regular view of this page.
Data Types
This module defines data types for logging rich, interactive visualizations to W&B.
Data types include common media types, like images, audio, and videos,
flexible containers for information, like tables and HTML, and more.
For more on logging media, see our guide
For more on logging structured data for interactive dataset and model analysis,
see our guide to W&B Tables.
All of these special data types are subclasses of WBValue. All the data types
serialize to JSON, since that is what wandb uses to save the objects locally
and upload them to the W&B server.
Classes
class Audio: Wandb class for audio clips.
class BoundingBoxes2D: Format images with 2D bounding box overlays for logging to W&B.
class Graph: Wandb class for graphs.
class Histogram: wandb class for histograms.
class Html: Wandb class for arbitrary html.
class Image: Format images for logging to W&B.
class ImageMask: Format image masks or overlays for logging to W&B.
class Molecule: Wandb class for 3D Molecular data.
class Object3D: Wandb class for 3D point clouds.
class Plotly: Wandb class for plotly plots.
class Table: The Table class used to display and analyze tabular data.
class Video: Format a video for logging to W&B.
class WBTraceTree: Media object for trace tree data.
1 - Audio
Wandb class for audio clips.
Audio(
data_or_path, sample_rate=None, caption=None
)
| Args |
|
data_or_path |
(string or numpy array) A path to an audio file or a numpy array of audio data. |
sample_rate |
(int) Sample rate, required when passing in raw numpy array of audio data. |
caption |
(string) Caption to display with audio. |
Methods
durations
View source
@classmethod
durations(
audio_list
)
resolve_ref
View source
sample_rates
View source
@classmethod
sample_rates(
audio_list
)
2 - BoundingBoxes2D
Format images with 2D bounding box overlays for logging to W&B.
BoundingBoxes2D(
val: dict,
key: str
) -> None
| Args |
|
val |
(dictionary) A dictionary of the following form: box_data: (list of dictionaries) One dictionary for each bounding box, containing: position: (dictionary) the position and size of the bounding box, in one of two formats Note that boxes need not all use the same format. {“minX”, “minY”, “maxX”, “maxY”}: (dictionary) A set of coordinates defining the upper and lower bounds of the box (the bottom left and top right corners) {“middle”, “width”, “height”}: (dictionary) A set of coordinates defining the center and dimensions of the box, with “middle” as a list [x, y] for the center point and “width” and “height” as numbers domain: (string) One of two options for the bounding box coordinate domain null: By default, or if no argument is passed, the coordinate domain is assumed to be relative to the original image, expressing this box as a fraction or percentage of the original image. This means all coordinates and dimensions passed into the “position” argument are floating point numbers between 0 and 1. “pixel”: (string literal) The coordinate domain is set to the pixel space. This means all coordinates and dimensions passed into “position” are integers within the bounds of the image dimensions. class_id: (integer) The class label id for this box scores: (dictionary of string to number, optional) A mapping of named fields to numerical values (float or int), can be used for filtering boxes in the UI based on a range of values for the corresponding field box_caption: (string, optional) A string to be displayed as the label text above this box in the UI, often composed of the class label, class name, and/or scores class_labels: (dictionary, optional) A map of integer class labels to their readable class names |
key |
(string) The readable name or id for this set of bounding boxes (e.g. predictions, ground_truth) |
Examples:
Log bounding boxes for a single image
import numpy as np
import wandb
wandb.init()
image = np.random.randint(low=0, high=256, size=(200, 300, 3))
class_labels = {0: "person", 1: "car", 2: "road", 3: "building"}
img = wandb.Image(
image,
boxes={
"predictions": {
"box_data": [
{
# one box expressed in the default relative/fractional domain
"position": {
"minX": 0.1,
"maxX": 0.2,
"minY": 0.3,
"maxY": 0.4,
},
"class_id": 1,
"box_caption": class_labels[1],
"scores": {"acc": 0.2, "loss": 1.2},
},
{
# another box expressed in the pixel domain
"position": {"middle": [150, 20], "width": 68, "height": 112},
"domain": "pixel",
"class_id": 3,
"box_caption": "a building",
"scores": {"acc": 0.5, "loss": 0.7},
},
# Log as many boxes an as needed
],
"class_labels": class_labels,
}
},
)
wandb.log({"driving_scene": img})
Log a bounding box overlay to a Table
import numpy as np
import wandb
wandb.init()
image = np.random.randint(low=0, high=256, size=(200, 300, 3))
class_labels = {0: "person", 1: "car", 2: "road", 3: "building"}
class_set = wandb.Classes(
[
{"name": "person", "id": 0},
{"name": "car", "id": 1},
{"name": "road", "id": 2},
{"name": "building", "id": 3},
]
)
img = wandb.Image(
image,
boxes={
"predictions": {
"box_data": [
{
# one box expressed in the default relative/fractional domain
"position": {
"minX": 0.1,
"maxX": 0.2,
"minY": 0.3,
"maxY": 0.4,
},
"class_id": 1,
"box_caption": class_labels[1],
"scores": {"acc": 0.2, "loss": 1.2},
},
{
# another box expressed in the pixel domain
"position": {"middle": [150, 20], "width": 68, "height": 112},
"domain": "pixel",
"class_id": 3,
"box_caption": "a building",
"scores": {"acc": 0.5, "loss": 0.7},
},
# Log as many boxes an as needed
],
"class_labels": class_labels,
}
},
classes=class_set,
)
table = wandb.Table(columns=["image"])
table.add_data(img)
wandb.log({"driving_scene": table})
Methods
type_name
View source
@classmethod
type_name() -> str
validate
View source
validate(
val: dict
) -> bool
3 - Graph
Wandb class for graphs.
This class is typically used for saving and displaying neural net models. It
represents the graph as an array of nodes and edges. The nodes can have
labels that can be visualized by wandb.
Examples:
Import a keras model:
Graph.from_keras(keras_model)
Methods
add_edge
View source
add_edge(
from_node, to_node
)
add_node
View source
add_node(
node=None, **node_kwargs
)
from_keras
View source
@classmethod
from_keras(
model
)
pprint
View source
__getitem__
View source
4 - Histogram
wandb class for histograms.
Histogram(
sequence: Optional[Sequence] = None,
np_histogram: Optional['NumpyHistogram'] = None,
num_bins: int = 64
) -> None
This object works just like numpy’s histogram function
https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram.html
Examples:
Generate histogram from a sequence
wandb.Histogram([1, 2, 3])
Efficiently initialize from np.histogram.
hist = np.histogram(data)
wandb.Histogram(np_histogram=hist)
| Args |
|
sequence |
(array_like) input data for histogram |
np_histogram |
(numpy histogram) alternative input of a precomputed histogram |
num_bins |
(int) Number of bins for the histogram. The default number of bins is 64. The maximum number of bins is 512 |
| Attributes |
|
bins |
([float]) edges of bins |
histogram |
([int]) number of elements falling in each bin |
| Class Variables |
|
MAX_LENGTH |
512 |
5 - Html
Wandb class for arbitrary html.
Html(
data: Union[str, 'TextIO'],
inject: bool = (True)
) -> None
| Args |
|
data |
(string or io object) HTML to display in wandb |
inject |
(boolean) Add a stylesheet to the HTML object. If set to False the HTML will pass through unchanged. |
Methods
inject_head
View source
6 - Image
Format images for logging to W&B.
Image(
data_or_path: "ImageDataOrPathType",
mode: Optional[str] = None,
caption: Optional[str] = None,
grouping: Optional[int] = None,
classes: Optional[Union['Classes', Sequence[dict]]] = None,
boxes: Optional[Union[Dict[str, 'BoundingBoxes2D'], Dict[str, dict]]] = None,
masks: Optional[Union[Dict[str, 'ImageMask'], Dict[str, dict]]] = None,
file_type: Optional[str] = None
) -> None
| Args |
|
data_or_path |
(numpy array, string, io) Accepts numpy array of image data, or a PIL image. The class attempts to infer the data format and converts it. |
mode |
(string) The PIL mode for an image. Most common are “L”, “RGB”, “RGBA”. Full explanation at https://pillow.readthedocs.io/en/stable/handbook/concepts.html#modes |
caption |
(string) Label for display of image. |
Note : When logging a torch.Tensor as a wandb.Image, images are normalized. If you do not want to normalize your images, please convert your tensors to a PIL Image.
Examples:
Create a wandb.Image from a numpy array
import numpy as np
import wandb
with wandb.init() as run:
examples = []
for i in range(3):
pixels = np.random.randint(low=0, high=256, size=(100, 100, 3))
image = wandb.Image(pixels, caption=f"random field {i}")
examples.append(image)
run.log({"examples": examples})
Create a wandb.Image from a PILImage
import numpy as np
from PIL import Image as PILImage
import wandb
with wandb.init() as run:
examples = []
for i in range(3):
pixels = np.random.randint(
low=0, high=256, size=(100, 100, 3), dtype=np.uint8
)
pil_image = PILImage.fromarray(pixels, mode="RGB")
image = wandb.Image(pil_image, caption=f"random field {i}")
examples.append(image)
run.log({"examples": examples})
log .jpg rather than .png (default)
import numpy as np
import wandb
with wandb.init() as run:
examples = []
for i in range(3):
pixels = np.random.randint(low=0, high=256, size=(100, 100, 3))
image = wandb.Image(pixels, caption=f"random field {i}", file_type="jpg")
examples.append(image)
run.log({"examples": examples})
Methods
all_boxes
View source
@classmethod
all_boxes(
images: Sequence['Image'],
run: "LocalRun",
run_key: str,
step: Union[int, str]
) -> Union[List[Optional[dict]], bool]
all_captions
View source
@classmethod
all_captions(
images: Sequence['Media']
) -> Union[bool, Sequence[Optional[str]]]
all_masks
View source
@classmethod
all_masks(
images: Sequence['Image'],
run: "LocalRun",
run_key: str,
step: Union[int, str]
) -> Union[List[Optional[dict]], bool]
guess_mode
View source
guess_mode(
data: "np.ndarray"
) -> str
Guess what type of image the np.array is representing.
to_uint8
View source
@classmethod
to_uint8(
data: "np.ndarray"
) -> "np.ndarray"
Convert image data to uint8.
Convert floating point image on the range [0,1] and integer images on the range
[0,255] to uint8, clipping if necessary.
| Class Variables |
|
MAX_DIMENSION |
65500 |
MAX_ITEMS |
108 |
7 - ImageMask
Format image masks or overlays for logging to W&B.
ImageMask(
val: dict,
key: str
) -> None
| Args |
|
val |
(dictionary) One of these two keys to represent the image: mask_data : (2D numpy array) The mask containing an integer class label for each pixel in the image path : (string) The path to a saved image file of the mask class_labels : (dictionary of integers to strings, optional) A mapping of the integer class labels in the mask to readable class names. These will default to class_0, class_1, class_2, etc. |
key |
(string) The readable name or id for this mask type (e.g. predictions, ground_truth) |
Examples:
Logging a single masked image
import numpy as np
import wandb
wandb.init()
image = np.random.randint(low=0, high=256, size=(100, 100, 3), dtype=np.uint8)
predicted_mask = np.empty((100, 100), dtype=np.uint8)
ground_truth_mask = np.empty((100, 100), dtype=np.uint8)
predicted_mask[:50, :50] = 0
predicted_mask[50:, :50] = 1
predicted_mask[:50, 50:] = 2
predicted_mask[50:, 50:] = 3
ground_truth_mask[:25, :25] = 0
ground_truth_mask[25:, :25] = 1
ground_truth_mask[:25, 25:] = 2
ground_truth_mask[25:, 25:] = 3
class_labels = {0: "person", 1: "tree", 2: "car", 3: "road"}
masked_image = wandb.Image(
image,
masks={
"predictions": {"mask_data": predicted_mask, "class_labels": class_labels},
"ground_truth": {
"mask_data": ground_truth_mask,
"class_labels": class_labels,
},
},
)
wandb.log({"img_with_masks": masked_image})
Log a masked image inside a Table
import numpy as np
import wandb
wandb.init()
image = np.random.randint(low=0, high=256, size=(100, 100, 3), dtype=np.uint8)
predicted_mask = np.empty((100, 100), dtype=np.uint8)
ground_truth_mask = np.empty((100, 100), dtype=np.uint8)
predicted_mask[:50, :50] = 0
predicted_mask[50:, :50] = 1
predicted_mask[:50, 50:] = 2
predicted_mask[50:, 50:] = 3
ground_truth_mask[:25, :25] = 0
ground_truth_mask[25:, :25] = 1
ground_truth_mask[:25, 25:] = 2
ground_truth_mask[25:, 25:] = 3
class_labels = {0: "person", 1: "tree", 2: "car", 3: "road"}
class_set = wandb.Classes(
[
{"name": "person", "id": 0},
{"name": "tree", "id": 1},
{"name": "car", "id": 2},
{"name": "road", "id": 3},
]
)
masked_image = wandb.Image(
image,
masks={
"predictions": {"mask_data": predicted_mask, "class_labels": class_labels},
"ground_truth": {
"mask_data": ground_truth_mask,
"class_labels": class_labels,
},
},
classes=class_set,
)
table = wandb.Table(columns=["image"])
table.add_data(masked_image)
wandb.log({"random_field": table})
Methods
type_name
View source
@classmethod
type_name() -> str
validate
View source
validate(
val: dict
) -> bool
8 - Molecule
Wandb class for 3D Molecular data.
Molecule(
data_or_path: Union[str, 'TextIO'],
caption: Optional[str] = None,
**kwargs
) -> None
| Args |
|
data_or_path |
(string, io) Molecule can be initialized from a file name or an io object. |
caption |
(string) Caption associated with the molecule for display. |
Methods
from_rdkit
View source
@classmethod
from_rdkit(
data_or_path: "RDKitDataType",
caption: Optional[str] = None,
convert_to_3d_and_optimize: bool = (True),
mmff_optimize_molecule_max_iterations: int = 200
) -> "Molecule"
Convert RDKit-supported file/object types to wandb.Molecule.
| Args |
|
data_or_path |
(string, rdkit.Chem.rdchem.Mol) Molecule can be initialized from a file name or an rdkit.Chem.rdchem.Mol object. |
caption |
(string) Caption associated with the molecule for display. |
convert_to_3d_and_optimize |
(bool) Convert to rdkit.Chem.rdchem.Mol with 3D coordinates. This is an expensive operation that may take a long time for complicated molecules. |
mmff_optimize_molecule_max_iterations |
(int) Number of iterations to use in rdkit.Chem.AllChem.MMFFOptimizeMolecule |
from_smiles
View source
@classmethod
from_smiles(
data: str,
caption: Optional[str] = None,
sanitize: bool = (True),
convert_to_3d_and_optimize: bool = (True),
mmff_optimize_molecule_max_iterations: int = 200
) -> "Molecule"
Convert SMILES string to wandb.Molecule.
| Args |
|
data |
(string) SMILES string. |
caption |
(string) Caption associated with the molecule for display |
sanitize |
(bool) Check if the molecule is chemically reasonable by the RDKit’s definition. |
convert_to_3d_and_optimize |
(bool) Convert to rdkit.Chem.rdchem.Mol with 3D coordinates. This is an expensive operation that may take a long time for complicated molecules. |
mmff_optimize_molecule_max_iterations |
(int) Number of iterations to use in rdkit.Chem.AllChem.MMFFOptimizeMolecule |
| Class Variables |
|
SUPPORTED_RDKIT_TYPES |
|
SUPPORTED_TYPES |
|
9 - Object3D
Wandb class for 3D point clouds.
Object3D(
data_or_path: Union['np.ndarray', str, 'TextIO', dict],
**kwargs
) -> None
| Args |
|
data_or_path |
(numpy array, string, io) Object3D can be initialized from a file or a numpy array. You can pass a path to a file or an io object and a file_type which must be one of SUPPORTED_TYPES |
The shape of the numpy array must be one of either:
[[x y z], ...] nx3
[[x y z c], ...] nx4 where c is a category with supported range [1, 14]
[[x y z r g b], ...] nx6 where is rgb is color
Methods
from_file
View source
@classmethod
from_file(
data_or_path: Union['TextIO', str],
file_type: Optional['FileFormat3D'] = None
) -> "Object3D"
Initializes Object3D from a file or stream.
| Args |
|
data_or_path (Union[“TextIO”, str]): A path to a file or a TextIO stream. file_type (str): Specifies the data format passed to data_or_path. Required when data_or_path is a TextIO stream. This parameter is ignored if a file path is provided. The type is taken from the file extension. |
|
from_numpy
View source
@classmethod
from_numpy(
data: "np.ndarray"
) -> "Object3D"
Initializes Object3D from a numpy array.
| Args |
|
| data (numpy array): Each entry in the array will represent one point in the point cloud. |
|
The shape of the numpy array must be one of either:
[[x y z], ...] # nx3.
[[x y z c], ...] # nx4 where c is a category with supported range [1, 14].
[[x y z r g b], ...] # nx6 where is rgb is color.
from_point_cloud
View source
@classmethod
from_point_cloud(
points: Sequence['Point'],
boxes: Sequence['Box3D'],
vectors: Optional[Sequence['Vector3D']] = None,
point_cloud_type: "PointCloudType" = "lidar/beta"
) -> "Object3D"
Initializes Object3D from a python object.
| Args |
|
| points (Sequence[“Point”]): The points in the point cloud. boxes (Sequence[“Box3D”]): 3D bounding boxes for labeling the point cloud. Boxes are displayed in point cloud visualizations. vectors (Optional[Sequence[“Vector3D”]]): Each vector is displayed in the point cloud visualization. Can be used to indicate directionality of bounding boxes. Defaults to None. point_cloud_type (“lidar/beta”): At this time, only the “lidar/beta” type is supported. Defaults to “lidar/beta”. |
|
| Class Variables |
|
SUPPORTED_POINT_CLOUD_TYPES |
|
SUPPORTED_TYPES |
|
10 - Plotly
Wandb class for plotly plots.
Plotly(
val: Union['plotly.Figure', 'matplotlib.artist.Artist']
)
| Args |
|
val |
matplotlib or plotly figure |
Methods
View source
@classmethod
make_plot_media(
val: Union['plotly.Figure', 'matplotlib.artist.Artist']
) -> Union[Image, 'Plotly']
11 - Table
The Table class used to display and analyze tabular data.
Table(
columns=None, data=None, rows=None, dataframe=None, dtype=None, optional=(True),
allow_mixed_types=(False)
)
Unlike traditional spreadsheets, Tables support numerous types of data:
scalar values, strings, numpy arrays, and most subclasses of wandb.data_types.Media.
This means you can embed Images, Video, Audio, and other sorts of rich, annotated media
directly in Tables, alongside other traditional scalar values.
This class is the primary class used to generate the Table Visualizer
in the UI: https://docs.wandb.ai/guides/data-vis/tables.
| Args |
|
columns |
(List[str]) Names of the columns in the table. Defaults to [“Input”, “Output”, “Expected”]. |
data |
(List[List[any]]) 2D row-oriented array of values. |
dataframe |
(pandas.DataFrame) DataFrame object used to create the table. When set, data and columns arguments are ignored. |
optional |
(Union[bool,List[bool]]) Determines if None values are allowed. Default to True - If a singular bool value, then the optionality is enforced for all columns specified at construction time - If a list of bool values, then the optionality is applied to each column - should be the same length as columns applies to all columns. A list of bool values applies to each respective column. |
allow_mixed_types |
(bool) Determines if columns are allowed to have mixed types (disables type validation). Defaults to False |
Methods
add_column
View source
add_column(
name, data, optional=(False)
)
Adds a column of data to the table.
| Args |
|
name |
(str) - the unique name of the column |
data |
(list |
optional |
(bool) - if null-like values are permitted |
add_computed_columns
View source
add_computed_columns(
fn
)
Adds one or more computed columns based on existing data.
| Args |
|
fn |
A function which accepts one or two parameters, ndx (int) and row (dict), which is expected to return a dict representing new columns for that row, keyed by the new column names. ndx is an integer representing the index of the row. Only included if include_ndx is set to True. row is a dictionary keyed by existing columns |
add_data
View source
Adds a new row of data to the table. The maximum amount of rows in a table is determined by wandb.Table.MAX_ARTIFACT_ROWS.
The length of the data should match the length of the table column.
add_row
View source
Deprecated; use add_data instead.
cast
View source
cast(
col_name, dtype, optional=(False)
)
Casts a column to a specific data type.
This can be one of the normal python classes, an internal W&B type, or an
example object, like an instance of wandb.Image or wandb.Classes.
| Args |
|
col_name |
(str) - The name of the column to cast. |
dtype |
(class, wandb.wandb_sdk.interface._dtypes.Type, any) - The target dtype. |
optional |
(bool) - If the column should allow Nones. |
get_column
View source
get_column(
name, convert_to=None
)
Retrieves a column from the table and optionally converts it to a NumPy object.
| Args |
|
name |
(str) - the name of the column |
convert_to |
(str, optional) - “numpy”: will convert the underlying data to numpy object |
get_dataframe
View source
Returns a pandas.DataFrame of the table.
get_index
View source
Returns an array of row indexes for use in other tables to create links.
index_ref
View source
Gets a reference of the index of a row in the table.
iterrows
View source
Returns the table data by row, showing the index of the row and the relevant data.
index : int
The index of the row. Using this value in other W&B tables
will automatically build a relationship between the tables
row : List[any]
The data of the row.
set_fk
View source
set_fk(
col_name, table, table_col
)
set_pk
View source
| Class Variables |
|
MAX_ARTIFACT_ROWS |
200000 |
MAX_ROWS |
10000 |
12 - Video
Format a video for logging to W&B.
Video(
data_or_path: Union['np.ndarray', str, 'TextIO', 'BytesIO'],
caption: Optional[str] = None,
fps: Optional[int] = None,
format: Optional[str] = None
)
| Args |
|
data_or_path |
(numpy array, string, io) Video can be initialized with a path to a file or an io object. The format must be “gif”, “mp4”, “webm” or “ogg”. The format must be specified with the format argument. Video can be initialized with a numpy tensor. The numpy tensor must be either 4 dimensional or 5 dimensional. Channels should be (time, channel, height, width) or (batch, time, channel, height width) |
caption |
(string) caption associated with the video for display |
fps |
(int) The frame rate to use when encoding raw video frames. Default value is 4. This parameter has no effect when data_or_path is a string, or bytes. |
format |
(string) format of video, necessary if initializing with path or io object. |
Examples:
Log a numpy array as a video
import numpy as np
import wandb
wandb.init()
# axes are (time, channel, height, width)
frames = np.random.randint(low=0, high=256, size=(10, 3, 100, 100), dtype=np.uint8)
wandb.log({"video": wandb.Video(frames, fps=4)})
Methods
encode
View source
encode(
fps: int = 4
) -> None
13 - WBTraceTree
Media object for trace tree data.
WBTraceTree(
root_span: Span,
model_dict: typing.Optional[dict] = None
)
| Args |
|
root_span (Span): The root span of the trace tree. model_dict (dict, optional): A dictionary containing the model dump. NOTE: model_dict is a completely-user-defined dict. The UI will render a JSON viewer for this dict, giving special treatment to dictionaries with a _kind key. This is because model vendors have such different serialization formats that we need to be flexible here. |
|