Custom Charts Walkthrough
Tutorial of using the custom charts feature in the Weights & Biases UI
To go beyond the built-in charts in Weights & Biases, use the new Custom Charts feature to control the details of exactly what data you're loading in to a panel and how you visualize that data.
- 1.Log data to W&B
- 2.Create a query
- 3.Customize the chart
# Logging a custom table of data
my_custom_data = [[x1, y1, z1], [x2, y2, z2]]
columns = ["x", "y", "z"])})
A new, blank custom chart ready to be configured
historyTableto set up a new query pulling data from the run history.
Now that the query is loading in these columns, they're available as options to select in the Vega fields dropdown menus:
Pulling in columns from the query results to set Vega fields
- x-axis: runSets_historyTable_r (recall)
- y-axis: runSets_historyTable_p (precision)
- color: runSets_historyTable_c (class label)
I updated the Vega spec to customize the visualization:
- add titles for the plot, legend, x-axis, and y-axis (set “title” for each field)
- change the value of “mark” from “point” to “line”
- remove the unused “size” field
To save this as a preset that you can use elsewhere in this project, click Save as at the top of the page. Here's what the result looks like, along with an ROC curve:
Histograms can visualize numerical distributions to help us understand larger datasets. Composite histograms show multiple distributions across the same bins, letting us compare two or more metrics across different models or across different classes within our model. For a semantic segmentation model detecting objects in driving scenes, we might compare the effectiveness of optimizing for accuracy versus intersection over union (IOU), or we might want to know how well different models detect cars (large, common regions in the data) versus traffic signs (much smaller, less common regions). In the demo Colab, you can compare the confidence scores for two of the ten classes of living things.
To create your own version of the custom composite histogram panel:
- 1.Create a new Custom Chart panel in your Workspace or Report (by adding a “Custom Chart” visualization). Hit the “Edit” button in the top right to modify the Vega spec starting from any built-in panel type.
- 3.Modify the query in the right hand side to load the correct data from your wandb logs. Add the field “summaryTable” and set the corresponding “tableKey” to “class_scores” to fetch the wandb.Table logged by your run. This will let you populate the two histogram bin sets (“red_bins” and “blue_bins”) via the dropdown menus with the columns of the wandb.Table logged as “class_scores”. For my example, I chose the “animal” class prediction scores for the red bins and “plant” for the blue bins.
- 4.You can keep making changes to the Vega spec and query until you’re happy with the plot you see in the preview rendering. Once you’re done, click “Save as” in the top and give your custom plot a name so you can reuse it. Then click “Apply from panel library” to finish your plot.
Here’s what my results look like from a very brief experiment: training on only 1000 examples for one epoch yields a model that’s very confident that most images are not plants and very uncertain about which images might be animals.