Skip to main content
Hugging Face AutoTrain is a no-code tool for training models for Natural Language Processing (NLP), Computer Vision (CV), Speech, and Tabular tasks. Hugging Face AutoTrain integrates directly with W&B for experiment tracking and config management. You only need a single parameter in the CLI command for your experiments. This page shows you how to enable W&B experiment tracking when you train a model with Hugging Face AutoTrain. You can capture metrics and configuration for every run without writing additional code. This page is for users who are already familiar with AutoTrain and want to add observability to their training workflows.
Experiment metrics logging

​
Install prerequisites

Before you can train a model and log results to W&B, install the AutoTrain CLI and the W&B client library. Install autotrain-advanced and wandb. 
pip install --upgrade autotrain-advanced wandb
To demonstrate these changes, this page fine-tunes an LLM on a math dataset and evaluates pass@1 on the GSM8k Benchmarks.

​
Prepare the dataset

Before training, prepare your dataset so it matches the format AutoTrain expects. Hugging Face AutoTrain expects your CSV custom dataset to have a specific format to work properly. Your training file must contain a text column, which the training uses. The data in the text column must conform to the ### Human: Question?### Assistant: Answer. format. Review an example in timdettmers/openassistant-guanaco. However, the MetaMathQA dataset includes the columns query, response, and type. First, pre-process this dataset. Remove the type column and combine the contents of the query and response columns into a new text column in the ### Human: Query?### Assistant: Response. format. Training uses the resulting dataset, rishiraj/guanaco-style-metamath.

​
Train using autotrain

With your environment and dataset ready, you can now start training. Start training with the autotrain advanced from the command line or a notebook. Use the --log argument, or use --log wandb to log your results to a run. The --log wandb argument enables the W&B integration for this run. Replace <huggingface-token> with your Hugging Face access token and <huggingface-repository-address> with the target repository address (for example, your-username/your-repo). 
autotrain llm \
    --train \
    --model HuggingFaceH4/zephyr-7b-alpha \
    --project-name zephyr-math \
    --log wandb \
    --data-path data/ \
    --text-column text \
    --lr 2e-5 \
    --batch-size 4 \
    --epochs 3 \
    --block-size 1024 \
    --warmup-ratio 0.03 \
    --lora-r 16 \
    --lora-alpha 32 \
    --lora-dropout 0.05 \
    --weight-decay 0.0 \
    --gradient-accumulation 4 \
    --logging_steps 10 \
    --fp16 \
    --use-peft \
    --use-int4 \
    --merge-adapter \
    --push-to-hub \
    --token <huggingface-token> \
    --repo-id <huggingface-repository-address>
Experiment config saving
After training starts, AutoTrain logs your run’s metrics and configuration to W&B, where you can review them alongside any other runs in your project.

​
More resources