#llama-cpp

2 articles

01 ·Inference & Serving·★ MEMBER·PAPER·6 min read Paper Explained: Disaggregated Quantization — Quantize LLM Prefill and Decode Differently A from-scratch explanation of disaggregated quantization: giving LLM prefill and decode their own number formats, weights and storage. Covers the prefillers that more than double 1-bit GGUF accuracy, SSD-streamed prefill (ODP), and where the method stops working. 02 ·Model Families·FREE·9 min read The Llama Family from Scratch — The Main Line of Open LLMs How Meta's Llama became the main line of open LLMs. The contrarian bet behind Llama 1, the lineage running through Llama 3.x, the llama.cpp / GGUF / Ollama ecosystem that grew around it, and the licensing traps to read before you ship commercially — starting from no prior knowledge.