Paper Explained: DeepSeek-V4.1-Flash — How Far Can KV Cache Compression Go?
DeepSeek-V4.1-Flash shrinks its global KV cache to 890 bytes per token. We unpack the four ideas behind it (the CED layer split, cross-layer KV sharing in CSA2, FP4 caching, and SWA Bounded Replay) from first principles.
Key points
- DeepSeek-V4.1-Flash is a multimodal MoE model with 552B backbone parameters and a one-million-token context, activating 16B parameters in decode and 8B in prefill.
- Combining cross-layer KV reuse (CSA2) with FP4 caching shrinks the global KV cache in HBM to 890 bytes per token.
- The 890 bytes is roughly 1/4 of DeepSeek-V4-Flash and reported as about 437× smaller than the original DeepSeek-V1.
- SWA Bounded Replay cuts the persistent cache on SSD or host memory to roughly 1/8.
- For long inputs, global KV takes up almost all memory and scales as N × bytes per token, so only bytes per token can be reduced.
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
Primary source — what this article is built on
undefined2026-09-17→undefined2026-09-25same month
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache CompressionDeepSeek-AI, :, Anyi Xu et al. · 2026-09-17 · v1arXiv:2609.19969Paper page·PDFundefined
The widespread adoption of long-horizon agents has made model workloads increasingly input-heavy. Although prior work has substantially reduced the cost of long-context computation, prefill remains computationally expensive, and large KV caches continue to strain HBM and SSD capacity and data-transfer bandwidth. Together, these compute, storage, and bandwidth demands constitute the primary bottleneck to further lowering deployment costs. To address this challenge, we introduce DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens. With its Causal Encoder-Decoder (CED) architecture, the model activates 16B parameters per token during decode but only 8B parameters during prefill, substantially improving cost efficiency for agentic workloads. To push the limits of KV cache compression, DeepSeek-V4.1-Flash combines cross-layer KV cache reuse in Compressed Sparse Attention 2 (CSA2) with FP4 KV caching. These designs reduce its global KV cache footprint (always in HBM) to 890 bytes per token, roughly 1/4 of the corresponding footprint of DeepSeek-V4-Flash. Further, through a dedicated deployment optimization known as SWA Bounded Replay, DeepSeek-V4.1-Flash reduces its persistent KV cache footprint (always on SSD or in host memory) to roughly 1/8 of that of DeepSeek-V4-Flash. Despite its much smaller KV cache footprint, the model delivers substantially better performance than the baseline. In addition, we streamline the DeepSeek-V4 architecture and introduce several efficient architectural extensions. We pretrain DeepSeek-V4.1-Flash on a multimodal corpus comprising 45T tokens and conduct comprehensive post-training, yielding strong performance across diverse text-based and multimodal agentic scenarios. Model checkpoints are available at https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash.
What problem does this paper solve?
This article covers DeepSeek-AI's technical report "DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression" (arXiv:2609.19969).
Here is what the abstract says. Long-horizon agents are now everywhere, so model workloads have become input-heavy. Prior work has already made long-context computation much cheaper, but prefill (reading the prompt) is still expensive. Large KV caches also put pressure on HBM and SSD capacity and on transfer bandwidth. Together, these compute, storage, and bandwidth demands are the main barrier to lowering deployment cost. The authors respond with a multimodal MoE model that has 552B backbone parameters and handles contexts of up to one million tokens. Its Causal Encoder-Decoder (CED) architecture activates 16B parameters per token during decode but only 8B during prefill. By combining cross-layer KV reuse (CSA2) with FP4 caching, the global KV cache that always sits in HBM drops to 890 bytes per token, roughly 1/4 of DeepSeek-V4-Flash. A deployment technique called SWA Bounded Replay then cuts the persistent cache on SSD or host memory to roughly 1/8. Even with far less cache, the model outperforms its predecessor.
If KV caches are new to you, start with KV Cache from Scratch.
An analogy: where do you keep the meeting minutes?
Each time a Transformer writes a word, it looks back at a "key" and a "value" computed from every earlier word. Those saved notes are the KV cache. Think of them as meeting minutes. The longer the meeting runs, the less the minutes fit on the desk (HBM). They get moved to the warehouse (SSD) or a locker (host memory), and fetching them takes time. The paper's point is that sparse attention has already made the compute cheap, so the remaining wall is how thick the minutes are and how often they must be carried around (§1).
V4-family models use two attention branches. A global branch sees the whole context, and sliding-window attention (SWA) sees only a fixed recent span. SWA memory stops growing once the window is full, so for long inputs, global KV takes up almost all of the memory.
In words, the total size of the minutes is the context length multiplied by how much each token's entry takes up. In plain terms, if the context length grows tenfold, so do the minutes. You can't shrink , so the only lever is the bytes per token. The paper gets this down to 890 bytes, which it reports as about 437× smaller than the original DeepSeek-V1 (Figure 1).
Comments
Sign in to comment