Video world models often produce clips that look realistic but violate basic physics—butter spreads like paint, balls pass through walls, and objects behave impossibly. The root cause is that conventional captions only describe what happens, not why it happens, leaving the model without the causal knowledge needed to enforce physical laws.
Physis‑Lang solves this by turning language itself into a physics‑aware signal. Each base caption is enriched with a physics_reasoning field that spells out the entities, causes, governing principles, temporal evolution and effects of a scene. A matching physics_negative_prompt lists implausible outcomes (e.g., a stone floating on water) to steer the model away from them at inference time.
The framework uses a self‑evolving loop: a frozen captioner generates captions for a small development set, a physics‑aware critic scores each caption on precision (claim‑by‑claim verification) and recall (coverage of human‑verified assertions), and an evolution agent rewrites the instruction based on failures. The loop validates each new prompt on PhysCapBench, a benchmark of 246 videos and 3,794 verified assertions, steadily improving caption F1 from 78.64 to 87.82 over nine iterations.
Language‑guided data curation further strengthens the model. A diagnosis agent maps generated‑video failures to physics categories (rigid‑body motion, collision, fluid dynamics) and retrieves relevant clips from a large gallery, boosting training relevance without relying on visual similarity alone. The final set combines filtered WISA data with retrieved clips, yielding consistent gains across backbones—+7.05 on Wan2.1‑14B, +3.24 on Cosmos3‑Edge‑4B, +6.22 on Cosmos3‑Nano‑16B, and +5.02 on Cosmos3‑Super‑64B.
Even without fine‑tuning, adding physics reasoning and negative prompts to a frozen model lifts performance by over five points on PhyGenBench. For teams wary of API costs, a distilled local pipeline (PhysThinker‑C and PhysThinker‑U) delivers most of the benefit at negligible expense.
In short, to stop video models from hallucinating physics, embed explicit causal language in captions, use negative prompts to block impossible outcomes, evolve those captions with a critic‑driven loop, and curate training data by physics relevance rather than appearance alone.
#AI #VideoModels #PhysicsReasoning #PhysisLang #ML #DeepLearning