Many teams building agents that learn from video hit a wall when they need to label every frame with the action that produced it. Collecting those labels is slow, expensive, and often impossible for large‑scale internet video. Induction Labs shows that the labeling step is not required. Their imagination model architecture pretrains on raw video without any action labels and still learns an implicit policy that can predict future states and guide behavior.
The core of the approach is a vision encoder that compresses each frame into 960 discrete tokens using finite scalar quantization. Each token is an eight‑dimensional vector whose components can only be –1, –0.5, 0, 0.5, or 1. This yields a codebook of 5⁸ possible codes and reduces the storage needed for a frame to about 2.2 KB – more than a hundred times smaller than typical OCR or multimodal‑model representations while keeping text, layout and state changes intact. By encoding frame differences instead of absolute contents, a differential latent encoder pushes the compression even further.
The pretrained model, Photon‑1, is a sparse 106‑billion‑parameter mixture‑of‑experts transformer with only 5 billion active parameters. It is trained on 18 years of computer‑demonstration video at one frame per second, using a next‑latent‑token‑prediction objective. On an internal computer‑use benchmark Photon‑1 outperforms Gemini 3.1 Flash‑Lite while consuming roughly 30 times fewer FLOPs and costing about one third as much to serve.
For practitioners this means: you can start pretraining immediately on any video corpus, avoid costly labeling pipelines, cut compute bills, and deploy a smaller, cheaper model that still matches or beats larger labeled baselines. The workflow is simple – collect video, run the FSQ encoder, train the imagination model, then fine‑tune on a few thousand task trajectories if you need explicit action outputs. The result is a scalable, label‑free path to competent video‑driven agents.
#AI #Product #MachineLearning #VideoAI #Efficiency #Innovation