Zyphra Zamba2-VL Slashes Vision-Language Model First-Token Delay

Zamba2-VL addresses three common pain points for developers building vision-language applications: high latency when processing long visual sequences, heavy compute demands that push models to the cloud, and uneven accuracy on tasks like document understanding versus visual counting. The hybrid Mamba2‑Transformer backbone replaces dense attention with state‑space layers that scale linearly with token count, cutting time‑to‑first‑token by roughly an order of magnitude compared with pure‑Transformer VLMs of similar size. This makes real‑time response feasible on edge devices such as smartphones or industrial gateways. The three released sizes—1.2 B, 2.7 B, and 7 B parameters—let teams pick a model that fits their memory and power budget while still benefiting from the same architecture. Benchmarks show strong performance on DocVQA (≈91 % for the 2.7 B model) and counting benchmarks (PixMoCount ≈82 %), which directly help use cases like invoice parsing, receipt digitisation, inventory tracking, and UI grounding. Knowledge‑heavy reasoning (MMMU, MathVista) remains weaker than larger baselines, so teams should complement Zamba2‑VL with retrieval or external knowledge sources when deep world understanding is required. The models are released under Apache 2.0 with weights on Hugging Face and inference code on GitHub; they run best with a CUDA GPU and the provided flash‑attention‑2 and Mamba2 kernels, while CPU fallback is considerably slower. By aligning low latency, modest hardware needs, and solid visual‑language accuracy, Zamba2‑VL gives practitioners a practical path to deploy responsive multimodal assistants on‑device without sacrificing core capabilities. #AI #MachineLearning #VisionLanguage #EdgeAI #LLM #OpenSource