Long-Horizon RL Challenges? QwenGyre’s Elastic RL Delivers Gains

Training agents that must act for hours and process nearly a million tokens hits two bottlenecks. First, rollout times vary wildly, leaving GPUs idle while the system waits for the slowest trajectory. Second, harnesses such as Claude Code create branching execution trees; naïve expansion duplicates shared prefixes and blows up memory and compute.

QwenGyre solves both with an elastic scheduler and a trajectory processor. GPUs are grouped into cells that can switch instantly between serving rollouts and hosting training replicas. A core cell holds the authoritative weights; satellite cells pull the latest parameters over RDMA and join the optimizer step as soon as they become free. The scheduler monitors a rollout water level – the sum of queued and running executions – and reallocates cells to training once enough capacity remains to cover live work. Switching takes under nine seconds on average, so idle hardware is reclaimed quickly.

The trajectory processor records every token‑in/token‑out pair via a TITO proxy, storing the execution as a prefix‑shared tree. It scores timed‑out runs on the work they completed, discards only evaluator failures, and caps the number of trajectories per execution at five, prioritising main paths over summaries. Shared prefixes are deduplicated, and loss is averaged inside each execution so extra branches do not inflate its weight. Forward‑backward time drops by roughly a quarter.

On benchmarks like NL2RepoBench, DeepSWE and TerminalBench, QwenGyre delivers 1.38‑1.85× end‑to‑end speedup over Async and Colocate baselines while raising the pass rate from ~52 % to ~58 % for the 2.4 T parameter model. The framework needs only a multi‑node GPU cluster and works with existing black‑box harnesses – no code changes to the agent are required.

If you run long‑horizon coding agents and see GPU waste or exploding training data, try elastic scheduling with trajectory pruning; it cuts idle time, bounds sample size, and lifts performance without new model training.

#AI #Product #ReinforcementLearning #ElasticCompute #AgentTraining #LLM