Prerequisites
Set up your environment to use W&B Inference
2 minute read
W&B Inference gives you access to leading open-source foundation models through W&B Weave and an OpenAI-compatible API. You can:
With Weave, you can trace, evaluate, monitor, and improve your W&B Inference-powered applications.
Here’s a simple example using Python:
import openai
client = openai.OpenAI(
# The custom base URL points to W&B Inference
base_url='https://api.inference.wandb.ai/v1',
# Get your API key from https://wandb.ai/authorize
api_key="<your-api-key>"
)
response = client.chat.completions.create(
model="meta-llama/Llama-3.1-8B-Instruct",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell me a joke."}
],
)
print(response.choices[0].message.content)
W&B Inference credits come with Free, Pro, and Academic plans for a limited time. Availability may vary for Enterprise accounts. When credits run out:
To learn more, visit the pricing page or see model-specific costs.
Set up your environment to use W&B Inference
Browse the foundation models available through W&B Inference
Understand pricing, usage limits, and account restrictions for W&B Inference
Complete API reference for W&B Inference service
Learn how to use W&B Inference with practical code examples
Access W&B Inference models through the web interface
Find answers to common W&B Inference questions
Was this page helpful?
Glad to hear it! If you have more to say, please let us know.
Sorry to hear that. Please tell us how we can improve.