Long‑horizon agents force language models to handle millions of token contexts, which inflates the key‑value cache and overwhelms HBM, SSD capacity and memory bandwidth. Repeated prefills and large contexts make serving expensive and limit the length of useful agent interactions. DeepSeek‑V4.1‑Flash tackles this bottleneck head‑on. Its global KV cache now occupies only 890 bytes per token – roughly a quarter of the previous Flash version and more than 400 times smaller than the original V1 release. This dramatic reduction comes from FP4 quantisation of the main KV store, which cuts storage without sacrificing accuracy.
To further lighten the workload, the model uses a causal encoder‑decoder split. During prompt processing only the 20‑layer encoder runs, halving the prefill compute and activating just 8 billion parameters per token. The decoder rebuilds its sliding‑window attention state by replaying the last 128 tokens, a technique called bounded replay, so it never needs to store a full KV history for each layer.
CSA2 (Compressed Sparse Attention 2) shares the main KV, indexer keys and top‑K indices across layers in three modes – Full, Reindex and Reuse – allowing most layers to reuse already‑computed structures instead of recomputing them. Hierarchical sparse indexing limits the candidate pool to a manageable size, keeping attention both accurate and efficient.
All of these innovations ship under an MIT licence and are ready to deploy via vLLM, SGLang or the Transformers library, with public API tiers for low, high and max reasoning effort. The result is a model that serves long contexts with far less memory pressure and lower latency, enabling agents to work over extended horizons without hitting hardware limits.
#AI #Product #MachineLearning #LLM #InferenceOptimization #DeepSeekV41Flash