> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wandb.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Why is my run showing as crashed?

W\&B marks a run as **Crashed** when it stops receiving heartbeats from the process that called `wandb.init()`, without the process having called `wandb.finish()`. This happens when the training process is killed, exits unexpectedly, or loses connectivity before it can report a clean finish.

**Common causes**

* **Out-of-memory (OOM) error**: The process is killed by the OS or GPU driver when it exceeds available memory. Check `output.log` for `CUDA out of memory` or `Killed` messages.
* **Uncaught exception**: An unhandled Python exception causes the process to exit without calling `wandb.finish()`. The exception appears in `output.log`.
* **Job scheduler preemption**: On SLURM or other cluster schedulers, jobs can be preempted and killed without warning. The run never gets a chance to finish cleanly.
* **Network loss**: In rare cases, a long network outage causes the W\&B backend to time out waiting for heartbeats and mark the run as crashed, even though the process is still running.
* **Process killed manually**: Using `kill -9` or `SIGKILL` bypasses Python's signal handlers, preventing `wandb.finish()` from being called.

**How to debug**

1. In the project sidebar, click **Runs**.
2. Click the name of your run and then click the **Files** tab.
3. Download `output.log` for stdout/stderr. This file usually contains the error that caused the crash.
4. Download `debug.log` and `debug-internal.log` for W\&B-level diagnostics (connectivity issues, upload errors).
5. If the run was on a cluster, also check the scheduler's job log for preemption or OOM signals.

**Data from a crashed run**

Metrics logged before the crash are preserved and visible in the UI. The run's charts, system metrics, and any artifacts that were fully uploaded before the crash are all accessible. Partially-uploaded artifacts may be incomplete.

If steps that were logged locally are missing from the UI (for example, the process kept running after the run was marked crashed), push the buffered data from the local run directory with `wandb sync`. Replace `[TIMESTAMP]` and `[ID]` with your run's values:

```bash theme={null}
wandb sync wandb/run-[TIMESTAMP]-[ID]
```

See [My run's state is crashed on the UI but is still running on my machine](/support/models/articles/my-runs-state-is-crashed-on-the-ui-but-i) for details.

**Preventing crashes from losing data**

Use `wandb.init()` as a context manager so the run finishes cleanly when your script raises an exception. The run is marked as **Failed** (rather than **Crashed**) and buffered data is flushed:

```python theme={null}
import wandb

with wandb.init(project="[YOUR-PROJECT]") as run:
    for step in range(1000):
        loss = ...  # your training step
        run.log({"loss": loss})
```

For run state definitions, see [Run states](/models/runs/run-states). For console logs after a crash, see [Why is console output not captured for my run?](/support/models/articles/why-is-console-output-not-captured-for-my-run).

***

<Badge stroke shape="pill" color="orange" size="md">[Runs](/support/models/tags/runs)</Badge><Badge stroke shape="pill" color="orange" size="md">[Run Crashes](/support/models/tags/run-crashes)</Badge>