Paper Walkthrough: Macaron-V1 — A Frozen Base plus a Mixture of LoRAs, Built to Keep Learning After Launch
A guided tour of Mind Lab's open agent-model family Macaron-V1, straight from the paper: the Mixture-of-LoRA architecture that stacks four specialist adapters on a frozen 744B base and picks one per turn, and the recursive self-improvement loop that trains the model and its harness as a versioned pair.
Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA
Primary source — what this article is built on
undefined2026-08-10→undefined2026-08-12same month
Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRAMind Lab, :, Vin Bo et al. · 2026-08-10 · v1arXiv:2608.09819Paper page·PDFundefined
Macaron-V1 is an open agent-model family for experiential intelligence: learning from experience in real environments and continuing to learn after deployment. It is organized around two system goals. Adaptation is pursued through recursive improvement of versioned model-harness pairs, where experience from one configuration is evaluated under an external contract and used to construct its successor. Collaboration is pursued via the Mixture-of-LoRA (MoL) architecture that freezes a base model, composes specialist LoRA adapters, and selects one LoRA per user turn. The flagship Macaron-V1-Venti combines a 744B GLM-5.2 base with four LoRAs for chat, agent, coding, and GenUI; the Qwen3.6-based Macaron-V1-Tall (50B) uses the same design for local deployment. This report presents Macaron-V1 as a co-designed system spanning architecture, algorithms, and infrastructure. The MoL architecture supports continual learning through extensible LoRA specialists. The algorithm combines Model-Harness Co-design and recursive self-improvement loop, including the UI4A component-native GenUI harness, a stateful action substrate, versioned HCP contract, and the agentic RL framework MindForge. The supporting infrastructure includes the post-training platform MinT, the long-context RL method LongStraw, and stability techniques for sparse MoE and DSA base models. We evaluate Macaron-V1 on Personal Intelligence, GenUI, and general capability benchmarks against frontier baselines. Our results validate the current system, while compounding gains from continual learning and collective intelligence remain open questions.
The model that stops aging the day it ships
One structural weakness of large language models is that the world moves on the moment they ship. A map app updates when a road changes; a model's weights stay frozen at the day training ended. Retraining a 744B-class model from scratch every time is prohibitively heavy, and cramming chat, coding, and tool use into one parameter space invites another problem the paper calls out: heterogeneous objectives competing through shared parameters can interfere with each other (§1).
Macaron-V1, an open agent-model family released by Mind Lab, is an attempt to break this "ship it and freeze it" pattern. The paper frames the goal as experiential intelligence — the ability to learn from experience accumulated in a real environment and to keep learning after deployment (§1).
Two bets: adaptation and collaboration
The paper organizes everything around two system goals (§1).
The first is adaptation. Instead of improving a model in isolation, Macaron-V1 versions the pair of model plus harness — the harness being the scaffolding that turns model output into actions: tools, prompts, the runtime (see LLM agents fundamentals). Experience from one versioned configuration is evaluated under an external contract and used to construct its successor, recursively.
The second is collaboration, pursued through the Mixture-of-LoRA (MoL) architecture: freeze a large base model, layer specialist LoRA adapters on top, and select exactly one adapter per user turn.
The family has three members. The flagship Macaron-V1-Venti puts four LoRAs (chat, agent, coding, GenUI) on a frozen 744B GLM-5.2 base, with 748B as the release-facing label. Macaron-V1-Tall ships the same four-adapter design on a Qwen3.6 base at roughly 50B, aimed at local deployment. Macaron-V1-Coding-Venti is the single-specialist exception, with the coding LoRA merged directly into the base (§1, §2.2).
Mixture of LoRA: swappable branches on a frozen trunk
LoRA is the standard trick of training a small low-rank delta instead of touching the giant weight matrices themselves (background in our LoRA paper walkthrough). MoL's design principle fits in one line (§2.1):
Cluster tasks that share skills and thinking patterns into one LoRA, and keep tasks whose skills diverge sharply in separate LoRAs.
Two properties fall out. Because the base is frozen, a new capability is added by training and registering another adapter — later specialization can never overwrite what the base already knows. And because the base is shared, specialists trained by different teams, or personalized for different users, can coexist on the same runtime (§2.1).
The similarly named MoE (Mixture of Experts) has a softmax gate distribute weight across experts for every token. MoL instead makes a hard selection — one adapter per turn. In the figure below, lowering the temperature makes the distribution collapse onto a single spike; MoL is that limiting case adopted by design, at turn granularity.
So far, so architectural. The interesting questions are operational: who decides which adapter takes each turn, what that decision costs in latency, and why switching specialists doesn't wreck the KV cache. That is where the paper earns its keep.
The three-step routing loop, once per turn
MoL trains no separate router model. The chat adapter L0 itself reads the incoming request and decides which specialist should handle it (§2.3). Each user turn runs three stages:
- Route: L0 emits exactly one canonical label (L0–L3) under a constrained-decoding grammar with a 24-token budget.
- Answer: the selected specialist generates the response.
- Summary: the specialist writes a recap of what it just did, capped at 192 tokens, stored server-side only.
Comments
Sign in to comment