Log Data & Media
Call wandb.log(dict) to log a dictionary of metrics or custom objects to a step. Each time you log, we increment the step by default, letting you view metrics over time.
wandb.log({"loss": 0.314, "epoch": 5,"inputs": wandb.Image(inputs),"logits": wandb.Histogram(ouputs),"captions": wandb.HTML(captions)})
Compare the best accuracy: To compare the best value of a metric across runs, set the summary value for that metric. By default, summary is set to the last value you logged for each key. This is useful in the table in the UI, where you can sort and filter runs based on their summary metrics — so you could compare runs in a table or bar chart based on their best accuracy, instead of final accuracy. For example, you could set summary like so:
wandb.run.summary["accuracy"] = best_accuracyMultiple metrics on one chart: Log multiple metrics in the same call to wandb.log(), like this:
wandb.log({'acc': 0.9, 'loss': 0.1})and they will both be available to plot against in the UICustom x-axis: Add a custom x-axis to the same log call to visualize your metrics against a different axis in the W&B dashboard. For example:
wandb.log({'acc': 0.9, 'epoch': 3, 'batch': 117})
View the reference docs, generated from the wandb Python library.
We support images, video, audio, custom graphs, and more. Log rich media to explore your results and visually compare your runs, models, and datasets.
You can see working code to log all of these media objects in this Colab Notebook, check out what the results look like on wandb.ai here, and follow along with a video tutorial, linked below.
Looking for reference docs for our media types? You want this page.
Log images to track inputs, outputs, filter weights, activations, and more!
Images can be logged directly from numpy arrays, as PIL images, or from the filesystem.
It's recommended to log fewer than 50 images per step to prevent logging from becoming a bottleneck during training and image loading from becoming a bottleneck when viewing results.
Provide arrays directly when constructing images manually, e.g. using make_grid from torchvision.
Arrays are converted to png using Pillow.
images = wandb.Image(image_array, caption="Top: Output, Bottom: Input")wandb.log({"examples": images}
We assume the image is gray scale if the last dimension is 1, RGB if it's 3, and RGBA if it's 4. If the array contains floats, we convert them to integers between 0 and 255. If you want to normalize your images differently, you can specify the mode manually or just supply a PIL.Image, as described in the "Logging PIL Images" tab of this panel.
For full control over the conversion of arrays to images, construct the PIL.Image yourself and provide it directly.
images = [PIL.Image.fromarray(image) for image in image_array]wandb.log({"examples": [wandb.Image(image) for image in images]}
For even more control, create images however you like, save them to disk, and provide a path.
im = PIL.fromarray(...)rgb_im = im.convert('RGB')rgb_im.save('myimage.jpg')wandb.log({"example": wandb.Image("myimage.jpg")})
Log semantic segmentation masks and interact with them (altering opacity, viewing changes over time, and more) via the W&B UI.
To log an overlay, you'll need to provide a dictionary with the following keys and values to the masks keyword argument of wandb.Image:
"mask_data": a 2D numpy array containing an integer class label for each pixel"class_labels": a dictionary mapping the numbers frommask_datato readable labels
To log multiple masks, log a mask dictionary with multiple keys, as in the code snippet below.
mask_data = np.array([[1, 2, 2, ... , 2, 2, 1], ...])class_labels = {1: "tree",2: "car",3: "road"}mask_img = wandb.Image(image, masks={"predictions": {"mask_data": mask_data,"class_labels": class_labels},"ground_truth": {...},...})
Log bounding boxes with images, and use filters and toggles to dynamically visualize different sets of boxes in the UI.
To log a bounding box, you'll need to provide a dictionary with the following keys and values to the boxes keyword argument of wandb.Image:
box_data: a list of dictionaries, one for each box. The box dictionary format is described below.position: a dictionary representing the position and size of the box in one of two formats, as described below. Boxes need not all use the same format.Option 1:
{"minX", "maxX", "minY", "maxY"}. Provide a set of coordinates defining the upper and lower bounds of each box dimension.Option 2:
{"middle", "width", "height"}. Provide a set of coordinates specifying themiddlecoordinates as[x,y], andwidthandheightas scalars.
class_id: an integer representing the class identity of the box. Seeclass_labelskey below.scores: a dictionary of string labels and numeric values for scores. Can be used for filtering boxes in the UI.
class_labels: (optional) A dictionary mappingclass_ids to strings. By default we will generate class labelsclass_0,class_1, etc.
Check out this example:
class_id_to_label = {1: "car",2: "road",3: "building",....}img = wandb.Image(image, boxes={"predictions": {"box_data": [{"position": {"minX": 0.1,"maxX": 0.2,"minY": 0.3,"maxY": 0.4,},"class_id" : 2,"box_caption": "minMax(pixel)","scores" : {"acc": 0.1,"loss": 1.2},},# Log as many boxes as needed...],"class_labels": class_id_to_label},"ground_truth": {# Log each group of boxes with a unique key name...}})wandb.log({"driving_scene": img})
wandb.log({"whale songs": wandb.Audio(np_array, caption="OooOoo", sample_rate=32)})
The maximum number of audio clips that can be logged per step is 100.
wandb.log({"video": wandb.Video(numpy_array_or_path_to_video, fps=4, format="gif")})
If a numpy array is supplied we assume the dimensions are, in order: time, channels, width, height. By default we create a 4 fps gif image (ffmpeg and the moviepy python library are required when passing numpy objects). Supported formats are "gif", "mp4", "webm", and "ogg". If you pass a string to wandb.Video we assert the file exists and is a supported format before uploading to wandb. Passing a BytesIO object will create a tempfile with the specified format as the extension.
On the W&B Run and Project Pages, you will see your videos in the Media section.
Use wandb.Table to log text in tables to show up in the UI. By default, the column headers are ["Input", "Output", "Expected"]. To ensure optimal UI performance, the default maximum number of rows is set to 10,000. However, users can explicitly override the maximum with wandb.Table.MAX_ROWS = {DESIRED_MAX}.
# Method 1data = [["I love my phone", "1", "1"],["My phone sucks", "0", "-1"]]wandb.log({"examples": wandb.Table(data=data, columns=["Text", "Predicted Label", "True Label"])})# Method 2table = wandb.Table(columns=["Text", "Predicted Label", "True Label"])table.add_data("I love my phone", "1", "1")table.add_data("My phone sucks", "0", "-1")wandb.log({"examples": table})
You can also pass a pandas DataFrame object.
table = wandb.Table(dataframe=my_dataframe)
wandb.log({"custom_file": wandb.Html(open("some.html"))})wandb.log({"custom_string": wandb.Html('<a href="https://mysite">Link</a>')})
Custom html can be logged at any key, and this exposes an HTML panel on the run page. By default we inject default styles, you can disable default styles by passing inject=False.
wandb.log({"custom_file": wandb.Html(open("some.html"), inject=False)})
wandb.log({"protein": wandb.Molecule(open("6lu7.pdb"))}
Log molecular data in any of 10 file types:
'pdb', 'pqr', 'mmcif', 'mcif', 'cif', 'sdf', 'sd', 'gro', 'mol2', 'mmtf'
When your run finishes, you'll be able to interact with 3D visualizations of your molecules in the UI.
If a sequence of numbers (e.g. list, array, tensor) is provided as the first argument, we will construct the histogram automatically by calling np.histogram. Note that all arrays/tensors are flattened. You can use the optional num_bins keyword argument to override the default of 64 bins. The maximum number of bins supported is 512.
In the UI, histograms are plotted with the training step on the x-axis, the metric value on the y-axis, and the count represented by color, to ease comparison of histograms logged throughout training. See the "Histograms in Summary" tab of this panel for details on logging one-off histograms.
wandb.log({"gradients": wandb.Histogram(grads)})
If you want more control, call np.histogram and pass the returned tuple to the np_histogram keyword argument.
np_hist_grads = np.histogram(grads, density=True, range=(0., 1.))wandb.log({"gradients": wandb.Histogram(np_hist_grads)})
wandb.run.summary.update( # if only in summary, only visible on overview tab{"final_logits": wandb.Histogram(logits)})
If histograms are in your summary they will appear on the Overview tab of the Run Page. If they are in your history, we plot a heatmap of bins over time on the Charts tab.
These presets have built-in wandb.plot methods that make it fast to log charts directly from your script and see the exact visualizations you're looking for in the UI.
wandb.plot.line()
Log a custom line plot—a list of connected and ordered points (x,y) on arbitrary axes x and y.
data = [[x, y] for (x, y) in zip(x_values, y_values)]table = wandb.Table(data=data, columns = ["x", "y"])wandb.log({"my_custom_plot_id" : wandb.plot.line(table, "x", "y",title="Custom Y vs X Line Plot")})
You can use this to log curves on any two dimensions. Note that if you're plotting two lists of values against each other, the number of values in the lists must match exactly (i.e. each point must have an x and a y).
wandb.plot.scatter()
Log a custom scatter plot—a list of points (x, y) on a pair of arbitrary axes x and y.
data = [[x, y] for (x, y) in zip(class_x_scores, class_y_scores)]table = wandb.Table(data=data, columns = ["class_x", "class_y"])wandb.log({"my_custom_id" : wandb.plot.scatter(table,"class_x", "class_y")})
You can use this to log scatter points on any two dimensions. Note that if you're plotting two lists of values against each other, the number of values in the lists must match exactly (i.e. each point must have an x and a y).
wandb.plot.bar()
Log a custom bar chart—a list of labeled values as bars—natively in a few lines:
data = [[label, val] for (label, val) in zip(labels, values)]table = wandb.Table(data=data, columns = ["label", "value"])wandb.log({"my_bar_chart_id" : wandb.plot.bar(table, "label","value", title="Custom Bar Chart")
You can use this to log arbitrary bar charts. Note that the number of labels and values in the lists must match exactly (i.e. each data point must have both).
wandb.plot.histogram()
Log a custom histogram—sort a list of values into bins by count/frequency of occurrence—natively in a few lines. Let's say I have a list of prediction confidence scores (scores) and want to visualize their distribution:
data = [[s] for s in scores]table = wandb.Table(data=data, columns=["scores"])wandb.log({'my_histogram': wandb.plot.histogram(table, "scores",title="Histogram")})
You can use this to log arbitrary histograms. Note that data is a list of lists, intended to support a 2D array of rows and columns.
wandb.plot.pr_curve()
Log a Precision-Recall curve in one line:
wandb.log({"pr" : wandb.plot.pr_curve(ground_truth, predictions,labels=None, classes_to_plot=None)})
You can log this whenever your code has access to:
a model's predicted scores (
predictions) on a set of examplesthe corresponding ground truth labels (
ground_truth) for those examples(optionally) a list of the labels/class names (
labels=["cat", "dog", "bird"...]if label index 0 means cat, 1 = dog, 2 = bird, etc.)(optionally) a subset (still in list format) of the labels to visualize in the plot
wandb.plot.roc_curve()
Log an ROC curve in one line:
wandb.log({"roc" : wandb.plot.roc_curve(ground_truth,predictions, labels=None, classes_to_plot=None)})
You can log this whenever your code has access to:
a model's predicted scores (
predictions) on a set of examplesthe corresponding ground truth labels (
ground_truth) for those examples(optionally) a list of the labels/ class names (
labels=["cat", "dog", "bird"...]if label index 0 means cat, 1 = dog, 2 = bird, etc.)(optionally) a subset (still in list format) of these labels to visualize on the plot
wandb.plot.confusion_matrix()
Log a multi-class confusion matrix in one line:
wandb.log({"conf_mat" : wandb.plot.confusion_matrix(probs=None,y_true=ground_truth,preds=predictions,class_names=class_names})
You can log this wherever your code has access to:
a model's predicted labels on a set of examples (
preds) or the normalized probability scores (probs). The probabilities must have the shape (number of examples, number of classes). You can supply either probabilities or predictions but not both.the corresponding ground truth labels for those examples (
y_true)a full list of the labels/class names as strings (
class_names, e.g.class_names=["cat", "dog", "bird"]if index 0 is cat, 1=dog, 2=bird, etc)
wandb.plot.line_series()
Plot multiple lines, or multiple different lists of x-y coordinate pairs, on one shared set of x-y axes:
wandb.log({"my_custom_id" : wandb.plot.line_series(xs=[0, 1, 2, 3, 4],ys=[[10, 20, 30, 40, 50], [0.5, 11, 72, 3, 41]],keys=["metric Y", "metric Z"],title="Two Random Metrics",xname="x units")})
Note that the number of x and y points must match exactly. You can supply one list of x values to match multiple lists of y values, or a separate list of x values for each list of y values.
Tweak a built-in Custom Chart preset, or create a new preset, then save the chart. Use the chart ID to log data to that custom preset directly from your script.
# Create a table with the columns to plottable = wandb.Table(data=data, columns=["step", "height"])# Map from the table's columns to the chart's fieldsfields = {"x": "step","value": "height"}# Use the table to populate the new custom chart preset# To use your own saved chart preset, change the vega_spec_namemy_custom_chart = wandb.plot_table(vega_spec_name="carey/new_chart",data_table=table,fields=fields,)
import matplotlib.pyplot as pltplt.plot([1, 2, 3, 4])plt.ylabel("some interesting numbers")wandb.log({"chart": plt})
You can pass a matplotlib pyplot or figure object to wandb.log(). By default we'll convert the plot into a Plotly plot. If you want to explicitly log the plot as an image, you can pass the plot into wandb.Image. We also accept directly logging Plotly charts.
If you’re getting an error “You attempted to log an empty plot” then you can store the figure separately from the plot with x = plt.figure() and then log x in your call to wandb.log.
Log files in the formats obj, gltf, or glb, and we will render them in the UI when your run finishes.
wandb.log({"generated_samples":[wandb.Object3D(open("sample.obj")),wandb.Object3D(open("sample.gltf")),wandb.Object3D(open("sample.glb"))]})
Log 3D point clouds and Lidar scenes with bounding boxes. Pass in a numpy array containing coordinates and colors for the points to render. In the UI, we truncate to 300,000 points.
point_cloud = np.array([[0, 0, 0, COLOR...], ...])wandb.log({"point_cloud": wandb.Object3D(point_cloud)})
Three different shapes of numpy arrays are supported for flexible color schemes.
[[x, y, z], ...]nx3[[x, y, z, c], ...]nx4| c is a categoryin the range[1, 14](Useful for segmentation)[[x, y, z, r, g, b], ...]nx6 | r,g,bare values in the range[0,255]for red, green, and blue color channels.
Here's an example of logging code below:
pointsis a numpy array with the same format as the simple point cloud renderer shown above.boxesis a numpy array of python dictionaries with three attributes:corners- a list of eight cornerslabel- a string representing the label to be rendered on the box (Optional)color- rgb values representing the color of the box
typeis a string representing the scene type to render. Currently the only supported value islidar/beta
# Log points and boxes in W&Bwandb.log({"point_scene": wandb.Object3D({"type": "lidar/beta","points": np.array([[0.4, 1, 1.3],[1, 1, 1],[1.2, 1, 1.2]]),"boxes": np.array([{"corners": [[0,0,0],[0,1,0],[0,0,1],[1,0,0],[1,1,0],[0,1,1],[1,0,1],[1,1,1]],"label": "Box","color": [123,321,111],},{"corners": [[0,0,0],[0,2,0],[0,0,2],[2,0,0],[2,2,0],[0,2,2],[2,0,2],[2,2,2]],"label": "Box-2","color": [111,321,0],}]),"vectors": [{"start": [0,0,0], "end": [0.1,0.2,0.5]}]})})
If you want to plot your metrics against different x-axes, you can log the step as a metric, like wandb.log({'loss': 0.1, 'epoch': 1, 'batch': 3}). In the UI you can switch between x-axes in the chart settings.
If you want to log to a single history step from lots of different places in your code you can pass a step index to wandb.log() as follows:
wandb.log({'loss': 0.2}, step=step)
As long as you keep passing the same value for step, W&B will collect the keys and values from each call in one unified dictionary. As soon you call wandb.log() with a different value for step than the previous one, W&B will write all the collected keys and values to the history, and start collection over again. Note that this means you should only use this with consecutive values for step: 0, 1, 2, .... This feature doesn't let you write to absolutely any history step that you'd like, only the "current" one and the "next" one.
You can also set commit=False in wandb.log to accumulate metrics, just be sure to eventually call wandb.log with commit=True (the default) to persist the metrics.
wandb.log({'loss': 0.2}, commit=False)# Somewhere else when I'm ready to report this step:wandb.log({'accuracy': 0.8})
The summary statistics are used to track single metrics per model. If a summary metric is modified, only the updated state is saved. We automatically set summary to the last history row added unless you modify it manually. If you change a summary metric, we only persist the last value it was set to.
wandb.init(config=args)best_accuracy = 0for epoch in range(1, args.epochs + 1):test_loss, test_accuracy = test()if (test_accuracy > best_accuracy):wandb.run.summary["best_accuracy"] = test_accuracybest_accuracy = test_accuracy
You may want to store evaluation metrics in a runs summary after training has completed. Summary can handle numpy arrays, PyTorch tensors or TensorFlow tensors. When a value is one of these types we persist the entire tensor in a binary file and store high level metrics in the summary object such as min, mean, variance, 95th percentile, etc.
api = wandb.Api()run = api.run("username/project/run_id")run.summary["tensor"] = np.random.random(1000)run.summary.update()
The history object is used to track metrics logged by wandb.log. You can access the history object via run.history().
api = wandb.Api()run = api.run("username/project/run_id")print(run.history())
Each time you log images from a step, we save them to show in the UI. Pin the image panel, and use the step slider to look at images from different steps. This makes it easy to compare how a model's output changes over training.
If you'd like to log certain metrics in every batch and standardize plots, you can log x axis values that you want to plot with your metrics. Then in the custom plots, click edit and select the custom x-axis.
wandb.log({'batch': batch_idx, 'loss': 0.3})wandb.log({'epoch': epoch, 'val_acc': 0.94})
wandb.Image converts numpy arrays or instances of PILImage to PNGs by default.
wandb.log({"example": wandb.Image(...)})# Or multiple imageswandb.log({"example": [wandb.Image(...) for img in images]})
wandb.log({"example": wandb.Video("myvideo.mp4")})
Now you can view videos in the media browser. Go to your project workspace, run workspace, or report and click "Add visualization" to add a rich media panel.
By default, we increment the global step every time you call wandb.log. If you'd like, you can log your own monotonically increasing step and then select it as a custom x-axis on your graphs.
For example, if you have training and validation steps you'd like to align, pass us your own step counter: wandb.log({"acc": 1, "global_step": 1}). Then in the graphs choose "global_step" as the x-axis.
wandb.log({"acc": 1, "batch": 10}, step=epoch) would enable you to choose "batch" as an x axis in addition to the default step axis
You can hold control and use the mouse to move around inside the space
If you're seeing "No visualization data logged yet" that means that we haven't gotten the first wandb.log call from your script yet. This could be because your run takes a long time to finish a step. If you're logging at the end of each epoch, you could log a few times per epoch to see data stream in more quickly.
If you're logging different types under the same key, we have to split them out in the database. This means you'll see multiple entries of the same metric name in a dropdown in the UI. The types we group by are number, string, bool, other (mostly arrays), and any wandb type (Histogram, Image, etc). Please send only one type to each key to avoid this behavior.
Sampling
The more points you send us, the longer it will take to load your graphs in the UI. If you have more than 1000 points on a line, we sample down to 1000 points on the backend before we send your browser the data. This sampling is nondeterministic, so if you refresh the page you'll see a different set of sampled points.
If you'd like all the original data, you can use our data API to pull down unsampled data.
Guidelines
We recommend that you try to log less than 10,000 points per metric. If you log more than 1 million points in a line, it will take us while to load the page. For more on strategies for reducing logging footprint without sacrificing accuracy, check out this Colab. If you have more than 500 columns of config and summary metrics, we'll only show 500 in the table.
We store metrics in a case-insensitive fashion, so make sure you don't have two metrics with the same name like "My-Metric" and "my-metric".
"I want to integrate W&B in my project, but I don't want to upload any images"
Our integration doesn't automatically upload images— you specify any files you'd like to upload explicitly. Here's a quick example in PyTorch that explicitly logs images: http://bit.ly/pytorch-mnist-colab
wandb.log({"Examples": example_images,"Test Accuracy": 100. * correct / len(test_loader.dataset),"Test Loss": test_loss})
