Documentation
Search…
OpenAI API
Fine-tune OpenAI GPT-3 on your own data and track results with W&B
Beta Integration: This is a new feature, and we're actively working on making this better. Please reach out if you have any feedback — [email protected]
OpenAI’s API gives practitioners access to GPT-3, an incredibly powerful natural language model that can be applied to virtually any task that involves understanding or generating natural language.
If you use OpenAI's API to fine-tune GPT-3, you can now use the W&B integration to track experiments, models, and datasets in your central dashboard.
All it takes is one line:
openai wandb sync✨ Check out interactive examples
✨
🎉 Sync your fine-tunes with one line!
🎉
Make sure you are using latest version of openai and wandb.
1
$ pip install --upgrade openai wandb
Copied!
Then sync your results from the command line or from your script.
Command Line
Python
1
$ # one line command
2
$ openai wandb sync
3
4
$ # passing optional parameters
5
$ openai wandb sync --help
Copied!
1
from openai.wandb_logger import WandbLogger
2
3
# one line command
4
WandbLogger.sync()
5
6
# passing optional parameters
7
WandbLogger.sync(
8
id=None,
9
n_fine_tunes=None,
10
project="GPT-3",
11
entity=None,
12
force=False,
13
**kwargs_wandb_init
14
)
Copied!
We scan for new completed fine-tunes and automatically add them to your dashboard.
In addition your training and validation files are logged and versioned, as well as details of your fine-tune results. This let you interactively explore your training and validation data.
⚙ Optional arguments
⚙
Argument
Description
-i ID, --id ID
The id of the fine-tune (optional)
-n N, --n_fine_tunes N
Number of most recent fine-tunes to log when an id is not provided. By default, every fine-tune is synced.
--project PROJECT
Name of the project where you're sending runs. By default, it is "GPT-3".
--entity ENTITY
Username or team name where you're sending runs. By default, your default entity is used, which is usually your username.
--force
Forces logging and overwrite existing wandb run of the same fine-tune.
**kwargs_wandb_init
🔍 Inspect sample predictions
Create a new run:
1
run = wandb.init(project="GPT-3", job_type="eval")
Copied!
Retrieve a model id for inference.
You can use automatically logged artifacts to retrieve your latest model:
1
artifact_job = run.use_artifact("ENTITY/PROJECT/fine_tune_details:latest")
2
fine_tuned_model = artifact_job.metadata["fine_tuned_model"]
Copied!
You can also retrieve your validation file:
1
artifact_valid = run.use_artifact("ENTITY/PROJECT/FILENAME:latest")
2
valid_file = artifact_valid.get_path("FILENAME").download()
Copied!
Perform some inferences using OpenAI API:
1
# perfom inference and record results
2
my_prompts = ["PROMPT_1", "PROMPT_2"]
3
results = []
4
for prompt in my_prompts:
5
res = openai.Completion.create(model=fine_tuned_model,
6
prompt=prompt,
7
...)
8
results.append(res["choices"][0]["text"])
Copied!
Log your results with a Table:
1
table = wandb.Table(columns=['prompt', 'completion'],
2
data=list(zip(my_prompts, results)))
Copied!
❓Frequently Asked Questions
❓
How do I share runs with my team?
Sync all your runs to your team account with:
1
$ openai wandb sync --entity MY_TEAM_ACCOUNT
Copied!
How can I organize my runs?
Your runs are automatically organized and can be filtered/sorted based on any configuration parameter such as job type, base model, learning rate, training filename and any other hyper-parameter.
In addition, you can rename your runs, add notes or create tags to group them.
Once you’re satisfied, you can save your workspace and use it to create report, importing data from your runs and saved artifacts (training/validation files).
How can I access my fine-tune details?
Fine-tune details are logged to W&B as artifacts and can be accessed with:
1
import wandb
2
3
artifact_job = wandb.run.use_artifact('USERNAME/PROJECT/job_details:VERSION')
Copied!
where
VERSION is either:- a version number such as
v2 - the fine-tune id such as
ft-xxxxxxxxx - an alias added automatically such as
latestor manually
You can then access fine-tune details through
artifact_job.metadata. For example, the fine-tuned model can be retrieved with artifact_job.metadata["fine_tuned_model"].What if a fine-tune was not synced successfully?
You can always call again
openai wandb sync and we will re-sync any run that was not synced successfully.If needed, you can call
openai wandb sync --id fine_tune_id --force to force re-syncing a specific fine-tune.Can I track my datasets with W&B?
Yes, you can integrate your entire pipeline to W&B through Artifacts, including creating your dataset, splitting it, training your models and evaluating them!
This will allow complete traceability of your models.
📚 Resources
📚
Last modified 26d ago
Copy link
Contents
Check out interactive examples
Sync your fine-tunes with one line!
Optional arguments
🔍 Inspect sample predictions
Frequently Asked Questions
How do I share runs with my team?
How can I organize my runs?
How can I access my fine-tune details?
What if a fine-tune was not synced successfully?
Can I track my datasets with W&B?
Resources