OmniFlow · recorded demo

One job, four layouts, no restart

An RL training job changes the GPU count and parallelism of its rollout and trainer engines while it trains

Qwen3-1.7B · GRPO on GSM8K · 8× NVIDIA B200 · DeepSpeed + vLLM 0.30

loading…
FIG. 1

The four layouts

replay pauses at the start of each layout · space skips

each column is one GPU — top: trainer rank, bottom: vLLM worker — brackets mark tensor-parallel groups

trainer (DeepSpeed)vLLM worker
FIG. 2

Training continues through every change

hover a step

what every GPU does at each optimizer step, and the reward of that step — dashed lines mark the changes

TABLE 1

Each change in detail

state = fp32 master weights + Adam moments, moved GPU to GPU · start-up = new processes, vLLM and trainer, built in the background · pause = gap in generation

ChangeTrainervLLMProcs out / in State movedMove + installStart-upPause