> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wandb.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> Parallelize W&B Sweep agents on multi-core or multi-GPU machine.

# Parallelize agents

Parallelize your W\&B Sweep agents on a multi-core or multi-GPU machine. Before you get started, ensure you have initialized your W\&B Sweep. For more information on how to initialize a W\&B Sweep, see [Initialize sweeps](./initialize-sweeps).

### Parallelize on a multi-CPU machine

Depending on your use case, explore the following tabs to learn how to parallelize W\&B Sweep agents using the CLI or within a Jupyter Notebook.

<Tabs>
  <Tab title="CLI">
    Use the [`wandb agent`](/models/ref/cli/wandb-agent) command to parallelize your sweep agent across multiple CPUs with the terminal. Provide the sweep ID that was returned when you [initialized the sweep](./initialize-sweeps).

    1. Open more than one terminal window on your local machine.
    2. Copy and paste the code snippet below and replace `sweep_id` with your sweep ID:

    ```bash theme={null}
    wandb agent sweep_id
    ```
  </Tab>

  <Tab title="Jupyter Notebook">
    Use the W\&B Python SDK library to parallelize your W\&B Sweep agent across multiple CPUs within Jupyter Notebooks. Ensure you have the sweep ID that was returned when you [initialized the sweep](./initialize-sweeps).  In addition, provide the name of the function the sweep will execute for the `function` parameter:

    1. Open more than one Jupyter Notebook.
    2. Copy and paste the W\&B Sweep ID on multiple Jupyter Notebooks to parallelize a W\&B Sweep. For example, you can paste the following code snippet on multiple jupyter notebooks to parallelize your sweep if you have the sweep ID stored in a variable called `sweep_id` and the name of the function is `function_name`:

    ```python theme={null}
    wandb.agent(sweep_id=sweep_id, function=function_name)
    ```
  </Tab>
</Tabs>

### Parallelize on a multi-GPU machine

Follow the procedure outlined to parallelize your W\&B Sweep agent across multiple GPUs with a terminal using CUDA Toolkit:

1. Open more than one terminal window on your local machine.
2. Specify the GPU instance to use with `CUDA_VISIBLE_DEVICES` when you start a W\&B Sweep job ([`wandb agent`](/models/ref/cli/wandb-agent)). Assign `CUDA_VISIBLE_DEVICES` an integer value corresponding to the GPU instance to use.

For example, suppose you have two NVIDIA GPUs on your local machine. Open a terminal window and set `CUDA_VISIBLE_DEVICES` to `0` (`CUDA_VISIBLE_DEVICES=0`). Replace `sweep_ID` in the following example with the W\&B Sweep ID that is returned when you initialized a W\&B Sweep:

Terminal 1

```bash theme={null}
CUDA_VISIBLE_DEVICES=0 wandb agent sweep_ID
```

Open a second terminal window. Set `CUDA_VISIBLE_DEVICES` to `1` (`CUDA_VISIBLE_DEVICES=1`). Paste the same W\&B Sweep ID for the `sweep_ID` mentioned in the following code snippet:

Terminal 2

```bash theme={null}
CUDA_VISIBLE_DEVICES=1 wandb agent sweep_ID
```