#decode

1 articles

01 ·Inference & Serving·★ MEMBER·PAPER·6 min read Paper Explained: Disaggregated Quantization — Quantize LLM Prefill and Decode Differently A from-scratch explanation of disaggregated quantization: giving LLM prefill and decode their own number formats, weights and storage. Covers the prefillers that more than double 1-bit GGUF accuracy, SSD-streamed prefill (ODP), and where the method stops working.