This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Reports

Project management and collaboration tools for machine learning projects

1: Create a report
2: Edit a report
3: Collaborate on reports
4: Clone and export reports
5: Embed a report
6: Compare runs across projects
7: Example reports

Use W&B Reports to:

Organize Runs.
Embed and automate visualizations.
Describe your findings.
Share updates with collaborators, either as a LaTeX zip file a PDF.

The following image shows a section of a report created from metrics that were logged to W&B over the course of training.

View the report where the above image was taken from here.

How it works

Create a collaborative report with a few clicks.

Navigate to your W&B project workspace in the W&B App.
Click the Create report button in the upper right corner of your workspace.

A modal titled Create Report will appear. Select the charts and panels you want to add to your report. (You can add or remove charts and panels later).
Click Create report.
Edit the report to your desired state.
Click Publish to project.
Click the Share button to share your report with collaborators.

See the Create a report page for more information on how to create reports interactively an programmatically with the W&B Python SDK.

How to get started

Depending on your use case, explore the following resources to get started with W&B Reports:

Check out our video demonstration to get an overview of W&B Reports.
Explore the Reports gallery for examples of live reports.
Try the Programmatic Workspaces tutorial to learn how to create and customize your workspace.
Read curated Reports in W&B Fully Connected.

1 - Create a report

Create a W&B Report with the App UI or programmatically with the Weights & Biases SDK.

Create a report interactively with the W&B App UI or programmatically with the W&B Python SDK.

See this Google Colab for an example.

Navigate to your project workspace in the W&B App.
Click Create report in the upper right corner of your workspace.
A modal will appear. Select the charts you would like to start with. You can add or delete charts later from the report interface.
Select the Filter run sets option to prevent new runs from being added to your report. You can toggle this option on or off. Once you click Create report, a draft report will be available in the report tab to continue working on.

Navigate to your project workspace in the W&B App.
Select to the Reports tab (clipboard image) in your project.
Select the Create Report button on the report page.

Create a report programmatically with the wandb library.

Install W&B SDK and Workspaces API:
```
pip install wandb wandb-workspaces
```

Next, import workspaces

import wandb
import wandb_workspaces.reports.v2 as wr

Create a report with wandb_workspaces.reports.v2.Report. Create a report instance with the Report Class Public API (wandb.apis.reports). Specify a name for the project.
```
report = wr.Report(project="report_standard")
```
Save the report. Reports are not uploaded to the W&B server until you call the .save() method:
```
report.save()
```

For information on how to edit a report interactively with the App UI or programmatically, see Edit a report.

2 - Edit a report

Edit a report interactively with the App UI or programmatically with the W&B SDK.

Reports consist of blocks. Blocks make up the body of a report. Within these blocks you can add text, images, embedded visualizations, plots from experiments and run, and panels grids.

Panel grids are a specific type of block that hold panels and run sets. Run sets are a collection of runs logged to a project in W&B. Panels are visualizations of run set data.

Check out the Programmatic workspaces tutorial for a step by step example on how create and customize a saved workspace view.

Ensure that you have wandb-workspaces installed in addition to the W&B Python SDK if you want to programmatically edit a report:

pip install wandb wandb-workspaces

Add plots

Each panel grid has a set of run sets and a set of panels. The run sets at the bottom of the section control what data shows up on the panels in the grid. Create a new panel grid if you want to add charts that pull data from a different set of runs.

Enter a forward slash (/) in the report to display a dropdown menu. Select Add panel to add a panel. You can add any panel that is supported by W&B, including a line plot, scatter plot or parallel coordinates chart.

Add plots to a report programmatically with the SDK. Pass a list of one or more plot or chart objects to the panels parameter in the PanelGrid Public API Class. Create a plot or chart object with its associated Python Class.

The proceeding examples demonstrates how to create a line plot and scatter plot.

import wandb
import wandb_workspaces.reports.v2 as wr

report = wr.Report(
    project="report-editing",
    title="An amazing title",
    description="A descriptive description.",
)

blocks = [
    wr.PanelGrid(
        panels=[
            wr.LinePlot(x="time", y="velocity"),
            wr.ScatterPlot(x="time", y="acceleration"),
        ]
    )
]

report.blocks = blocks
report.save()

For more information about available plots and charts you can add to a report programmatically, see wr.panels.

Add run sets

Add run sets from projects interactively with the App UI or the W&B SDK.

Enter a forward slash (/) in the report to display a dropdown menu. From the dropdown, choose Panel Grid. This will automatically import the run set from the project the report was created from.

Add run sets from projects with the wr.Runset() and wr.PanelGrid Classes. The proceeding procedure describes how to add a runset:

Create a wr.Runset() object instance. Provide the name of the project that contains the runsets for the project parameter and the entity that owns the project for the entity parameter.
Create a wr.PanelGrid() object instance. Pass a list of one or more runset objects to the runsets parameter.
Store one or more wr.PanelGrid() object instances in a list.
Update the report instance blocks attribute with the list of panel grid instances.

import wandb
import wandb_workspaces.reports.v2 as wr

report = wr.Report(
    project="report-editing",
    title="An amazing title",
    description="A descriptive description.",
)

panel_grids = wr.PanelGrid(
    runsets=[wr.RunSet(project="<project-name>", entity="<entity-name>")]
)

report.blocks = [panel_grids]
report.save()

You can optionally add runsets and panels with one call to the SDK:

import wandb

report = wr.Report(
    project="report-editing",
    title="An amazing title",
    description="A descriptive description.",
)

panel_grids = wr.PanelGrid(
    panels=[
        wr.LinePlot(
            title="line title",
            x="x",
            y=["y"],
            range_x=[0, 100],
            range_y=[0, 100],
            log_x=True,
            log_y=True,
            title_x="x axis title",
            title_y="y axis title",
            ignore_outliers=True,
            groupby="hyperparam1",
            groupby_aggfunc="mean",
            groupby_rangefunc="minmax",
            smoothing_factor=0.5,
            smoothing_type="gaussian",
            smoothing_show_original=True,
            max_runs_to_show=10,
            plot_type="stacked-area",
            font_size="large",
            legend_position="west",
        ),
        wr.ScatterPlot(
            title="scatter title",
            x="y",
            y="y",
            # z='x',
            range_x=[0, 0.0005],
            range_y=[0, 0.0005],
            # range_z=[0,1],
            log_x=False,
            log_y=False,
            # log_z=True,
            running_ymin=True,
            running_ymean=True,
            running_ymax=True,
            font_size="small",
            regression=True,
        ),
    ],
    runsets=[wr.RunSet(project="<project-name>", entity="<entity-name>")],
)


report.blocks = [panel_grids]
report.save()

Add code blocks

Add code blocks to your report interactively with the App UI or with the W&B SDK.

Enter a forward slash (/) in the report to display a dropdown menu. From the dropdown choose Code.

Select the name of the programming language on the right hand of the code block. This will expand a dropdown. From the dropdown, select your programming language syntax. You can choose from Javascript, Python, CSS, JSON, HTML, Markdown, and YAML.

Use the wr.CodeBlock Class to create a code block programmatically. Provide the name of the language and the code you want to display for the language and code parameters, respectively.

For example the proceeding example demonstrates a list in YAML file:

import wandb
import wandb_workspaces.reports.v2 as wr

report = wr.Report(project="report-editing")

report.blocks = [
    wr.CodeBlock(
        code=["this:", "- is", "- a", "cool:", "- yaml", "- file"], language="yaml"
    )
]

report.save()

This will render a code block similar to:

this:
- is
- a
cool:
- yaml
- file

The proceeding example demonstrates a Python code block:

report = wr.Report(project="report-editing")


report.blocks = [wr.CodeBlock(code=["Hello, World!"], language="python")]

report.save()

This will render a code block similar to:

Hello, World!

Add markdown

Add markdown to your report interactively with the App UI or with the W&B SDK.

Enter a forward slash (/) in the report to display a dropdown menu. From the dropdown choose Markdown.

Use the wandb.apis.reports.MarkdownBlock Class to create a markdown block programmatically. Pass a string to the text parameter:

import wandb
import wandb_workspaces.reports.v2 as wr

report = wr.Report(project="report-editing")

report.blocks = [
    wr.MarkdownBlock(text="Markdown cell with *italics* and **bold** and $e=mc^2$")
]

This will render a markdown block similar to:

Add HTML elements

Add HTML elements to your report interactively with the App UI or with the W&B SDK.

Enter a forward slash (/) in the report to display a dropdown menu. From the dropdown select a type of text block. For example, to create an H2 heading block, select the Heading 2 option.

Pass a list of one or more HTML elements to wandb.apis.reports.blocks attribute. The proceeding example demonstrates how to create an H1, H2, and an unordered list:

import wandb
import wandb_workspaces.reports.v2 as wr

report = wr.Report(project="report-editing")

report.blocks = [
    wr.H1(text="How Programmatic Reports work"),
    wr.H2(text="Heading 2"),
    wr.UnorderedList(items=["Bullet 1", "Bullet 2"]),
]

report.save()

This will render a HTML elements to the following:

Embed rich media links

Embed rich media within the report with the App UI or with the W&B SDK.

Copy and past URLs into reports to embed rich media within the report. The following animations demonstrate how to copy and paste URLs from Twitter, YouTube, and SoundCloud.

Twitter

Copy and paste a Tweet link URL into a report to view the Tweet within the report.

Youtube

Copy and paste a YouTube video URL link to embed a video in the report.

SoundCloud

Copy and paste a SoundCloud link to embed an audio file into a report.

Pass a list of one or more embedded media objects to the wandb.apis.reports.blocks attribute. The proceeding example demonstrates how to embed video and Twitter media into a report:

import wandb
import wandb_workspaces.reports.v2 as wr

report = wr.Report(project="report-editing")

report.blocks = [
    wr.Video(url="https://www.youtube.com/embed/6riDJMI-Y8U"),
    wr.Twitter(
        embed_html='<blockquote class="twitter-tweet"><p lang="en" dir="ltr">The voice of an angel, truly. <a href="https://twitter.com/hashtag/MassEffect?src=hash&amp;ref_src=twsrc%5Etfw">#MassEffect</a> <a href="https://t.co/nMev97Uw7F">pic.twitter.com/nMev97Uw7F</a></p>&mdash; Mass Effect (@masseffect) <a href="https://twitter.com/masseffect/status/1428748886655569924?ref_src=twsrc%5Etfw">August 20, 2021</a></blockquote>\n'
    ),
]
report.save()

Duplicate and delete panel grids

If you have a layout that you would like to reuse, you can select a panel grid and copy-paste it to duplicate it in the same report or even paste it into a different report.

Highlight a whole panel grid section by selecting the drag handle in the upper right corner. Click and drag to highlight and select a region in a report such as panel grids, text, and headings.

Select a panel grid and press delete on your keyboard to delete a panel grid.

Collapse headers to organize Reports

Collapse headers in a Report to hide content within a text block. When the report is loaded, only headers that are expanded will show content. Collapsing headers in reports can help organize your content and prevent excessive data loading. The proceeding gif demonstrates the process.

3 - Collaborate on reports

Collaborate and share W&B Reports with peers, co-workers, and your team.

Once you have saved a report, you can select the Share button to collaborate. A draft copy of the report is created when you select the Edit button. Draft reports auto-save. Select Save to report to publish your changes to the shared report.

A warning notification will appear if an edit conflict occurs. This can occur if you and another collaborator edit the same report at the same time. The warning notification will guide you to resolve potential edit conflicts.

Report sharing modal for a report in a 'Public' project

Comment on reports

Click the comment button on a panel in a report to add a comment directly to that panel.

4 - Clone and export reports

Export a W&B Report as a PDF or LaTeX.

Export reports

Export a report as a PDF or LaTeX. Within your report, select the kebab icon to expand the dropdown menu. Choose Download and select either PDF or LaTeX output format.

Cloning reports

Within your report, select the kebab icon to expand the dropdown menu. Choose the Clone this report button. Pick a destination for your cloned report in the modal. Choose Clone report.

Clone a report to reuse a project’s template and format. Cloned projects are visible to your team if you clone a project within the team’s account. Projects cloned within an individual’s account are only visible to that user.

Load a Report from a URL to use it as a template.

report = wr.Report(
    project=PROJECT, title="Quickstart Report", description="That was easy!"
)  # Create
report.save()  # Save
new_report = wr.Report.from_url(report.url)  # Load

Edit the content within new_report.blocks.

pg = wr.PanelGrid(
    runsets=[
        wr.Runset(ENTITY, PROJECT, "First Run Set"),
        wr.Runset(ENTITY, PROJECT, "Elephants Only!", query="elephant"),
    ],
    panels=[
        wr.LinePlot(x="Step", y=["val_acc"], smoothing_factor=0.8),
        wr.BarPlot(metrics=["acc"]),
        wr.MediaBrowser(media_keys="img", num_columns=1),
        wr.RunComparer(diff_only="split", layout={"w": 24, "h": 9}),
    ],
)
new_report.blocks = (
    report.blocks[:1] + [wr.H1("Panel Grid Example"), pg] + report.blocks[1:]
)
new_report.save()

5 - Embed a report

Embed W&B reports directly into Notion or with an HTML IFrame element.

HTML iframe element

Select the Share button on the upper right hand corner within a report. A modal window will appear. Within the modal window, select Copy embed code. The copied code will render within an Inline Frame (IFrame) HTML element. Paste the copied code into an iframe HTML element of your choice.

Only public reports are viewable when embedded.

Confluence

The proceeding animation demonstrates how to insert the direct link to the report within an IFrame cell in Confluence.

Notion

The proceeding animation demonstrates how to insert a report into a Notion document using an Embed block in Notion and the report’s embedded code.

Gradio

You can use the gr.HTML element to embed W&B Reports within Gradio Apps and use them within Hugging Face Spaces.

import gradio as gr


def wandb_report(url):
    iframe = f'<iframe src={url} style="border:none;height:1024px;width:100%">'
    return gr.HTML(iframe)


with gr.Blocks() as demo:
    report = wandb_report(
        "https://wandb.ai/_scott/pytorch-sweeps-demo/reports/loss-22-10-07-16-00-17---VmlldzoyNzU2NzAx"
    )
demo.launch()

6 - Compare runs across projects

Compare runs from two different projects with cross-project reports.

Compare runs from two different projects with cross-project reports. Use the project selector in the run set table to pick a project.

The visualizations in the section pull columns from the first active runset. Make sure that the first run set checked in the section has that column available if you do not see the metric you are looking for in the line plot.

This feature supports history data on time series lines, but we don’t support pulling different summary metrics from different projects. In other words, you can not create a scatter plot from columns that are only logged in another project.

If you need to compare runs from two projects and the columns are not working, add a tag to the runs in one project and then move those runs to the other project. You can still filter only the runs from each project, but the report includes all the columns for both sets of runs.

View-only report links

Share a view-only link to a report that is in a private project or team project.

View-only report links add a secret access token to the URL, so anyone who opens the link can view the page. Anyone can use the magic link to view the report without logging in first. For customers on W&B Local private cloud installations, these links remain behind your firewall, so only members of your team with access to your private instance and access to the view-only link can view the report.

In view-only mode, someone who is not logged in can see the charts and mouse over to see tooltips of values, zoom in and out on charts, and scroll through columns in the table. When in view mode, they cannot create new charts or new table queries to explore the data. View-only visitors to the report link won’t be able to click a run to get to the run page. Also, the view-only visitors would not be able to see the share modal but instead would see a tooltip on hover which says: Sharing not available for view only access.

The magic links are only available for “Private” and “Team” projects. For “Public” (anyone can view) or “Open” (anyone can view and contribute runs) projects, the links can’t turn on/off because this project is public implying that it is already available to anyone with the link.

Send a graph to a report

Send a graph from your workspace to a report to keep track of your progress. Click the dropdown menu on the chart or panel you’d like to copy to a report and click Add to report to select the destination report.

7 - Example reports

Reports gallery

Notes: Add a visualization with a quick summary

Capture an important observation, an idea for future work, or a milestone reached in the development of a project. All experiment runs in your report will link to their parameters, metrics, logs, and code, so you can save the full context of your work.

Jot down some text and pull in relevant charts to illustrate your insight.

See the What To Do When Inception-ResNet-V2 Is Too Slow W&B Report for an example of how you can share comparisons of training time.

Save the best examples from a complex code base for easy reference and future interaction. See the LIDAR point clouds W&B Report for an example of how to visualize LIDAR point clouds from the Lyft dataset and annotate with 3D bounding boxes.

Explain how to get started with a project, share what you’ve observed so far, and synthesize the latest findings. Your colleagues can make suggestions or discuss details using comments on any panel or at the end of the report.

Include dynamic settings so that your colleagues can explore for themselves, get additional insights, and better plan their next steps. In this example, three types of experiments can be visualized independently, compared, or averaged.

See the SafeLife benchmark experiments W&B Report for an example of how to share first runs and observations of a benchmark.

Use sliders and configurable media panels to showcase a model’s results or training progress. View the Cute Animals and Post-Modern Style Transfer: StarGAN v2 for Multi-Domain Image Synthesis report for an example W&B Report with sliders.

Work log: Track what you’ve tried and plan next steps

Write down your thoughts on experiments, your findings, and any gotchas and next steps as you work through a project, keeping everything organized in one place. This lets you “document” all the important pieces beyond your scripts. See the Who Is Them? Text Disambiguation With Transformers W&B Report for an example of how you can report your findings.

Tell the story of a project, which you and others can reference later to understand how and why a model was developed. See The View from the Driver’s Seat W&B Report for how you can report your findings.

See the Learning Dexterity End-to-End Using W&B Reports for an example of how W&B Reports were used to explore how the OpenAI Robotics team used W&B Reports to run massive machine learning projects.