W&B Inference

Access open-source foundation models through W&B Weave and an OpenAI-compatible API

W&B Inference gives you access to leading open-source foundation models through W&B Weave and an OpenAI-compatible API. You can:

  • Build AI applications and agents without signing up for a hosting provider or self-hosting a model
  • Try supported models in the W&B Weave Playground

With Weave, you can trace, evaluate, monitor, and improve your W&B Inference-powered applications.

Quickstart

Here’s a simple example using Python:

import openai

client = openai.OpenAI(
    # The custom base URL points to W&B Inference
    base_url='https://api.inference.wandb.ai/v1',
    
    # Get your API key from https://wandb.ai/authorize
    api_key="<your-api-key>"
)

response = client.chat.completions.create(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me a joke."}
    ],
)

print(response.choices[0].message.content)

Next steps

  1. Review the available models and usage information and limits
  2. Set up your account using the prerequisites
  3. Use the service through the API or UI
  4. Try the usage examples

Usage details


Prerequisites

Set up your environment to use W&B Inference

Available Models

Browse the foundation models available through W&B Inference

Usage Information and Limits

Understand pricing, usage limits, and account restrictions for W&B Inference

API Reference

Complete API reference for W&B Inference service

Usage Examples

Learn how to use W&B Inference with practical code examples

UI Guide

Access W&B Inference models through the web interface

Support

Find answers to common W&B Inference questions