An RL training job changes the GPU count and parallelism of its rollout and trainer engines while it trains
each column is one GPU — top: trainer rank, bottom: vLLM worker — brackets mark tensor-parallel groups
what every GPU does at each optimizer step, and the reward of that step — dashed lines mark the changes
state = fp32 master weights + Adam moments, moved GPU to GPU · start-up = new processes, vLLM and trainer, built in the background · pause = gap in generation
| Change | Trainer | vLLM | Procs out / in | State moved | Move + install | Start-up | Pause |
|---|