Perplexity Research shows how to turn real‑world agent mistakes into useful training signals without discarding failed sessions. Their method separates each assistant turn into three groups: imitate successful non‑error steps, correct error turns that have a validated hint, and keep the rest as context only. By giving the model a hint that points to the exact mistake—such as a disallowed parameter value—and then running the same checkpoint twice (once with the hint as teacher, once without as student), they create a KL divergence loss that nudges the student toward the correct behavior while still preserving successful patterns through cross‑entropy loss.
In practice this means developers can keep training on the full log of user interactions, including those that end in failure, and still improve reliability. The hint is derived from information the model already had, so it does not require external data or human labeling beyond a simple validation check. The combined loss (CE + λ × KL) lets teams tune how much weight to give correction versus imitation; setting λ to zero recovers standard fine‑tuning, while a positive λ enables learning from mistakes.
Live A/B tests demonstrated a 21.2 % relative reduction in tool‑call failures, dropping from 2.24 % to 1.77 % between two checkpoints. Offline experiments also showed lower error rates when the hint‑guided self‑distillation was added to rejection sampling fine‑tuning. Importantly, the approach does not require releasing new model weights; it can be applied to any existing checkpoint, such as the openly available GLM‑5.2 base, by recording hints during data collection and running the two‑pass training loop.
For teams struggling with brittle tool use, sporadic failures, or costly post‑hoc debugging, this technique offers a concrete way to extract learning signal from every session, improve success rates, and keep the training process simple and scalable.
#AI #Product #MachineLearning #LLM #AIResearch #Tech