Xiaomi MiMo & TileRT: 1T‑Param Model >1k Tokens/s on Regular GPUs

Ultra‑fast token generation is now a decisive factor for competitive AI services. Xiaomi’s MiMo‑V2.5‑Pro‑UltraSpeed shows that a trillion‑parameter mixture‑of‑experts model can decode more than 1 000 tokens per second on ordinary GPUs, eliminating the need for custom silicon. The breakthrough comes from three tightly integrated layers. First, FP4 (MXFP4) quantization is applied only to the MoE experts, where memory bandwidth is highest, while Quantization‑Aware Training keeps model quality essentially unchanged. Second, DFlash speculative decoding removes the serial bottleneck of traditional draft models by predicting an entire masked block of up to eight tokens in one forward pass; acceptance lengths average 6.3 tokens for coding, 5.6 for math/reasoning, and 4.3 for agent tasks, with lossless output guaranteed by rejection sampling. Third, the TileRT runtime replaces per‑operator launches with a persistent engine kernel and warp specialization, squeezing each step into microseconds and eliminating the gaps that normally throttle throughput at extreme speeds. Together these techniques deliver roughly a ten‑fold speed increase over the baseline MiMo‑V2.5‑Pro model while running on a single 8‑GPU commodity node. Practical benefits include faster parallel reasoning (Best‑of‑N, tree search), reduced latency for coding agents, real‑time decision loops for trading or fraud detection, and interactive prototyping where a simple game can be generated in about ten seconds. The UltraSpeed mode is offered via an API‑only trial (June 9‑23 2026) at three times the standard token price, delivering the speedup without requiring custom hardware. This approach gives developers and enterprises a clear path to low‑latency LLM deployment on existing infrastructure. #AI #LLM #InferenceSpeed #FP4Quantization #SpeculativeDecoding #TileRT