Custom Charts Walkthrough
Tutorial of using the custom charts feature in the Weights & Biases UI
To go beyond the built-in charts in Weights & Biases, use the new Custom Charts feature to control the details of exactly what data you're loading in to a panel and how you visualize that data.
Overview
- 1.Log data to W&B
- 2.Create a query
- 3.Customize the chart
First, log data in your script. Use wandb.config for single points set at the beginning of training, like hyperparameters. Use wandb.log() for multiple points over time, and log custom 2D arrays with wandb.Table(). We recommend logging up to 10,000 data points per logged key.
# Logging a custom table of data
my_custom_data = [[x1, y1, z1], [x2, y2, z2]]
wandb.log({"custom_data_table": wandb.Table(data=my_custom_data,
columns = ["x", "y", "z"])})
Try a quick example notebook to log the data tables, and in the next step we'll set up custom charts. See what the resulting charts look like in the live report.
Once you've logged data to visualize, go to your project page and click the
+ button to add a new panel, then select Custom Chart. You can follow along in this workspace.A new, blank custom chart ready to be configured
- 1.Click
summaryand selecthistoryTableto set up a new query pulling data from the run history. - 2.Type in the key where you logged the wandb.Table(). In the code snippet above, it was
my_custom_table. In the example notebook, the keys arepr_curveandroc_curve.
Now that the query is loading in these columns, they're available as options to select in the Vega fields dropdown menus:
Pulling in columns from the query results to set Vega fields
- x-axis: runSets_historyTable_r (recall)
- y-axis: runSets_historyTable_p (precision)
- color: runSets_historyTable_c (class label)
Now that looks pretty good, but I'd like to switch from a scatter plot to a line plot. Click Edit to change the Vega spec for this built in chart. Follow along in this workspace.
I updated the Vega spec to customize the visualization:
- add titles for the plot, legend, x-axis, and y-axis (set “title” for each field)
- change the value of “mark” from “point” to “line”
- remove the unused “size” field
To save this as a preset that you can use elsewhere in this project, click Save as at the top of the page. Here's what the result looks like, along with an ROC curve:
Histograms can visualize numerical distributions to help us understand larger datasets. Composite histograms show multiple distributions across the same bins, letting us compare two or more metrics across different models or across different classes within our model. For a semantic segmentation model detecting objects in driving scenes, we might compare the effectiveness of optimizing for accuracy versus intersection over union (IOU), or we might want to know how well different models detect cars (large, common regions in the data) versus traffic signs (much smaller, less common regions). In the demo Colab, you can compare the confidence scores for two of the ten classes of living things.
To create your own version of the custom composite histogram panel:
- 1.Create a new Custom Chart panel in your Workspace or Report (by adding a “Custom Chart” visualization). Hit the “Edit” button in the top right to modify the Vega spec starting from any built-in panel type.
- 2.Replace that built-in Vega spec with my MVP code for a composite histogram in Vega. You can modify the main title, axis titles, input domain, and any other details directly in this Vega spec using Vega syntax (you could change the colors or even add a third histogram :)
- 3.Modify the query in the right hand side to load the correct data from your wandb logs. Add the field “summaryTable” and set the corresponding “tableKey” to “class_scores” to fetch the wandb.Table logged by your run. This will let you populate the two histogram bin sets (“red_bins” and “blue_bins”) via the dropdown menus with the columns of the wandb.Table logged as “class_scores”. For my example, I chose the “animal” class prediction scores for the red bins and “plant” for the blue bins.
- 4.You can keep making changes to the Vega spec and query until you’re happy with the plot you see in the preview rendering. Once you’re done, click “Save as” in the top and give your custom plot a name so you can reuse it. Then click “Apply from panel library” to finish your plot.
Here’s what my results look like from a very brief experiment: training on only 1000 examples for one epoch yields a model that’s very confident that most images are not plants and very uncertain about which images might be animals.
Last modified 9mo ago