Robots that must navigate unfamiliar indoor spaces often rely on depth sensors, LiDAR or multiple cameras to understand where they are and where to go. These extra sensors increase cost, weight and power consumption, and they complicate integration on small wheeled, legged or flying platforms. Training vision‑and‑language navigation models usually requires months of compute because each time step is processed separately, and the resulting systems still struggle when the target is out of view.
Mistral AI’s Robostral Navigate solves these problems by using a single RGB image as the only sensor input. The model is an 8‑billion‑parameter vision‑language foundation built from scratch for grounding tasks such as pointing, counting and object localization. Once the model can locate objects in the frame, it learns to issue movement commands that bring the robot toward the target.
Training is performed entirely in simulation with a pipeline that generated about four hundred thousand trajectories across six thousand diverse scenes. A prefix‑caching, tree‑based attention masking technique compresses an entire episode into one sequence, allowing the model to learn from every time step in a single forward pass. This reduces the number of training tokens by roughly twenty‑two times, turning what used to be months of training into days.
After supervised learning, an online reinforcement learning stage called CISPO lets the model improve through trial and error, recover from failures and develop exploratory behaviors. This step adds another three point two percent to success rates while reducing the gap between training and real‑world deployment.
On the standard Room‑to‑Room in Continuous Environments benchmark, Robostral Navigate achieves a 76.6 percent success rate on unseen scenes using only one RGB camera, outperforming the best depth‑or multi‑camera systems by several points. Because the approach works on wheeled, legged and flying robots of various sizes, a single fleet can share the same navigation stack, simplifying maintenance and lowering overall system cost.
#AI #Product #Robotics #VisionLanguage #Navigation #EfficientAI