#lora
12 articles
01
·RAG & Retrieval·FREE·8 min read
RAG vs Fine-Tuning — Which One, and When
The two main ways to make an LLM better, compared on four axes: knowledge freshness, cost, hallucination, and data requirements. From metaphor to math to interactive demos to the mistakes that break production systems.
02
·Inference & Serving·★ MEMBER·PAPER·10 min read
Paper Explained: Compile by Training — Turning a Natural-Language Spec into a Function That Runs Locally
"Sort my email into urgent and later" — a spec that vague, turned into a small function you can run locally after about a minute of training. A ground-up walkthrough of Compile by Training: analogy, mechanism, equations, measured numbers.
03
·Training & Alignment·★ MEMBER·PAPER·12 min read
Paper Walkthrough: StudentSim — Training a Simulator That Is Actually *That* Student
How to build the practice students an AI tutor needs, by training on real learner records. From the two-axis definition of behavioral fidelity and guidance responsiveness, through the pooled-then-specialized pipeline, to using the simulator as a reward model.
04
·Inference & Serving·★ MEMBER·PAPER·8 min read
Paper Walkthrough: Normalized Low-Rank Adaptation — Why Normalizing LoRA's Entry Matrix Works
Rescaling LoRA's down-projection so every column has unit length improves convergence, stability and forgetting resistance at zero extra cost. A ground-up reading of Normalized Low-Rank Adaptation (NoRA) through the lens of a hidden preconditioner.
05
·Time Series·★ MEMBER·PAPER·12 min read
H3-World, Explained — Turning Language Understanding into World Control
How to turn a 33B video generator into a controllable world model without bolting on an action module: express actions as sentences, bind each sentence to a video-latent interval, and use attention routing to stop the crosstalk. Only 0.199% of parameters are trained.
06
·Agents·★ MEMBER·PAPER·14 min read
Paper Walkthrough: Code World Model — Putting a Coding Agent in Charge of the World
A ground-up reading of Code World Model, which hands the *evolution* of a world to executable code and its *appearance* to a video model — including the proxy interface that joins them, and the limits the authors admit to.
07
·Generative Models·FREE·PAPER·11 min read
A Practical Map of Image Generation — SD, ControlNet, and Applying LoRA
A single map for anyone about to touch image generation for the first time: the four boxes of latent diffusion, where the prompt actually takes effect, locking down composition with ControlNet, fine-tuning with LoRA, and the licensing layers that quietly cause the most damage.
08
·Training & Alignment·★ MEMBER·PAPER·9 min read
Continual Learning and Catastrophic Forgetting — Why Models Can't Just Keep Learning
Why a model that learns something new tends to lose what it already knew, explained from a single fact: weights are a shared resource. Walks through EWC, replay, and swappable LoRA adapters with equations and interactive figures, then explains why production teams still just mix the old data back in and retrain.
09
·Linear Algebra·FREE·7 min read
The Linear Algebra Under LoRA and RAG — Eigenvalues, Low Rank and Vector Search, Hands On
A matrix is a deformation of space, an eigenvector is a direction that survives it, SVD generalises the idea, and the dot product is the definition of 'similar'. Four interactive figures and four equations show that LoRA's ΔW=BA and RAG's vector search stand on the same floor. A column meant to be dragged, not just read.
10
·Linear Algebra·★ MEMBER·PAPER·10 min read
Singular Value Decomposition and Low-Rank Approximation — the Math Behind LoRA
Starting from the 'rotate, stretch, rotate' picture, this article builds Singular Value Decomposition (SVD) from zero: matrices as stacks of rank-1 layers, why real-world data needs only a few of them, and how that single fact lets LoRA fine-tune a giant model with 0.4% of the parameters.
11
·Agents·★ MEMBER·PAPER·8 min read
Paper Walkthrough: Macaron-V1 — A Frozen Base plus a Mixture of LoRAs, Built to Keep Learning After Launch
A guided tour of Mind Lab's open agent-model family Macaron-V1, straight from the paper: the Mixture-of-LoRA architecture that stacks four specialist adapters on a frozen 744B base and picks one per turn, and the recursive self-improvement loop that trains the model and its harness as a versioned pair.
12
·Paper Deep-Dives·★ MEMBER·PAPER·12 min read
Paper Deep Dive — LoRA: Low-Rank Adaptation of Large Language Models: Why Low Rank Is Enough
A re-reading of the LoRA paper (Hu et al., 2021) grounded strictly in its own text: what BA actually means, the 21x amplification factor the authors measured, why r=1 sufficed, and the questions they left open.