Paper Walkthrough: CodeNib — A Multi-View Data System That Serves Repository Context to Coding Agents

Coding agents grep their way through the same repository over and over. CodeNib (UC San Diego et al.) reframes this as a database problem — lexical, dense, and structural views over an immutable commit — and measures the whole lifecycle, caveats included.

CodeNib: A Multi-View Data System for Serving Repository Context to Coding Agents

Coding agents show up to their first day of work — every day

Imagine a brilliant new hire who loses their memory every night. Their skills are intact, but each morning they redraw the office map before getting anything done.

That is the daily life of a coding agent — an LLM that operates on a repository through tools like grep and file reads. Every task starts with "where is that function again?", and everything the agent learns while exploring is thrown away when the task ends. The paper describes the status quo bluntly: disconnected indexes, language servers, and task-local histories "force repeated discovery and obscure lifecycle costs" (Abstract).

CodeNib, published on 2026-07-28 by a team centered at UC San Diego, attacks this not as a machine-learning problem but as a database problem.

The reframe: a repository is a database

The paper's core move is a mapping (§1): a commit is immutable base data; chunks, postings, embeddings, symbol occurrences, and relationships are derived views; agent requests are view-specific queries; and the context that lands in a prompt is a bounded delivery result.

Databases have a classic tool for exactly this shape: the materialized view — precompute a result, pay upfront, answer queries fast, and refresh when the base data changes. A code-search index is a materialized view over a commit. From that intuition, the paper derives three coupled challenges (§1). C1: lexical, dense, and structural views need different physical layouts, yet their results must map back to the same commit and source ranges. C2: a single edit breaks each view differently, so each needs its own update path. C3: precomputed evidence must reach the model while build, load, query, and history costs stay visible.

Three views and one shared "address"

CodeNib slices source code into a hierarchy (§4.1): files are $L_0$, type-like scopes such as classes are $L_1$, and callable definitions — functions and methods — are $L_2$. Every unit carries a repository-relative path plus line range, and three views are built on top (§5):

• Lexical view: BM25 postings, optionally a Zoekt trigram index — lookup by keywords and identifiers
• Dense view: embeddings of $L_0$ or $L_2$ units stored in FAISS — lookup by semantic similarity
• Structural view: a graph of files, scopes, and definitions connected by typed edges like "contains" and "references" — go-to-definition and find-references

Dense search embeds the query into the same vector space and returns units with the largest inner product — a measure of how closely two vectors point in the same direction (for how code and prose become vectors in the first place, Embeddings from Scratch is the groundwork). You can get a feel for it here:

The views are built independently, but every result is normalized to the same address format: repository-relative path plus line range. A manifest $M_c$ acts as the catalog, recording each view's location, status, and capabilities per commit (§5.4). If the vector build fails, BM25 stays usable — per-view independence plus catalog-based discovery is the heart of the design.

The agent-facing side compresses to a few lines (condensing the paper's Listing 1):

M = compile(checkout, ["bm25", "vector", "graph"])  # build offline, record in manifest
views = load_views(M, needed)   # at runtime: open existing views, never build
runner = AgentRunner(views, M)  # exposed as search / definition / reference tools

When an edit arrives: repair, don't rebuild

Repositories keep moving, and rebuilding every view per commit is expensive. CodeNib combines Git diffs with LSP (the language-server protocol behind editor features) to classify graph symbols as deleted, affected, shifted, unchanged, or added, then repairs only what broke (§6.4). A function whose lines merely shifted just gets its position rebased; on the vector side, embeddings whose content hash is unchanged are reused.

The admirable part is that the paper attaches a correctness condition to every speed claim. Each incrementally updated view is compared offline against an independently rebuilt "ground truth" (§4.2), and only transitions with an exact match count toward the reported speedup, defined as (§4.3):

In words: "full-rebuild time divided by (incremental-update time plus an amortized share of server startup)." Here $T_f$ is the rebuild cost, $T_u$ the update cost, $T_s$ the language-server startup and warmup cost, shared across $n_{\mathrm{share}}$ updates (5 in the experiments). Above 1, the incremental path wins.

Strip the symbols away and the ratio asks one plain question: how many times faster is patching only the broken part than throwing the index away and building it again? The honest touch is the $T_s/n_{\mathrm{share}}$ term, which says that spinning up a language server is not free — instead of quietly ignoring that cost, the paper charges it to the updates that share it.

Results (§9.5): symbol-level graph repair matches the independent rebuild on 15 of 33 transitions, with a median 8.67× speedup on those matches; vector updates match on 28 of 31 (90.3%) at a median 25.44×. The flip side is just as clear: not a single Rust or TypeScript/JavaScript graph repair passes every check — edge F1 hovers near 99%, but near-perfect is not exact — and the paper repeatedly stresses that offline comparison is not a runtime guarantee.

Quality versus cost: the millisecond–second wall

Measured over 100 snapshots derived from SWE-Bench Verified/Multilingual (§9.2–9.3):

• Dense search averages 26–295 ms, with File Recall@10 (how often target files appear in the top ten) of 0.705–0.820
• Adding a reranker — an LLM that rescores candidates — lifts recall to 0.858 but costs 4.29 s, about 46.6× the latency of plain dense search (0.812 at 92 ms)
• Graph expansion (widening results one hop along reference edges) is inconclusive: every 95% confidence interval crosses zero
• Approximate nearest neighbor (HNSW) makes the FAISS search step 33.9× faster, but the heavier build only amortizes after roughly 2,300 searches. At this scale, a plain exhaustive scan (Flat) is fine

That last point is a complexity intuition: Flat is a linear scan over the number of stored vectors, while HNSW pays upfront at build time to make each search cheap.

What about go-to-definition (§9.4)? The static index returns the same locations as a live language server on 632 of 1,000 requests (63.2%), and on that matched subset it is a median 4.72× faster (0.62 ms versus 2.26 ms). But the split is lopsided — 87.4% for definitions, only 39.0% for references — and the paper says it plainly: 4.72× is "a conditional opportunity, not an achieved workload speedup." That is why CodeNib keeps a live LSP path instead of routing everything to the static index.

Serving the agent: grep/read, eager delivery, and a one-time rewrite

Finally, C3 — how evidence actually reaches the model (§7.4). Three policies are compared:

1. Grep/read: no candidates; the model hunts with tools on its own (the status quo)
2. Eager: the top-10 retrieved $L_2$ code blocks are injected into the prompt upfront
3. Eager+Compact: same injection, plus a one-time history rewrite right after the first successful file read

Compact's rewrite is deterministic — no summarization model. It drops the candidate dump and the exploration transcript, keeping only the deduplicated read paths, the latest successful read in full, and a short cue from the last assistant message. Every later turn extends that short prefix, so the KV cache can be reused (§7.4).

Results (§9.7): for each of five models (Claude Haiku 4.5, Qwen3.5-9B/27B, Gemma 4-12B, Gemini 2.5 Flash), some policy meets the quality margin while using 49.9% down to 12.9% of the paired grep/read tokens — a 50–87% reduction with localization quality (AnswerRecall@5) preserved. And compaction is honestly reported as non-universal: on Haiku, Compact increases tokens to 123.3% of Eager, while on Gemma it shrinks them to 27.9%. The right serving policy is model-dependent.

What the paper does not claim

The paper spends unusual effort marking its own boundaries; take them at face value.

• Evaluation stops at localization inside the repository — patch generation and issue resolution are not measured (§9)
• The manifest is not a transaction: after an edit, there is no guarantee all views are simultaneously consistent; concurrent updates are future work (§3.4, §10)
• Static navigation is not an LSP replacement — reference lookups match only 39% of the time (§9.4)
• The update-correctness check is offline, not a runtime oracle (§4.2)
• The quality margin (ε = 0.05) is "a reporting threshold, not a pre-registered equivalence test" (§4.3)

How this plays out on the job

Who, and when: platform and developer-productivity teams running coding agents as a product or internal service, at the moment they design the agent's search tooling. It is also the best cost-measured ammunition yet for the recurring debate "do we need RAG over the codebase, or is grep enough?"

Knobs and tools you will actually touch: FAISS index types (IndexFlatIP / IndexIVFFlat / IndexHNSWFlat with nprobe and efSearch), retrieval widths (the paper's retrieve_top_k and rerank_candidate_top_k play different roles), tree-sitter, SCIP-family indexers (scip-python, scip-go, rust-analyzer; clangd needs a compile_commands.json), and exposing the operators as MCP tools.

Pitfalls that cause real incidents:

• Don't reach for HNSW by reflex. At the paper's scale it takes ~2,300 searches to pay off the build. Measure Flat first
• Don't call a reranker inside every loop turn. It turns milliseconds into seconds and dominates perceived agent latency
• Don't silently substitute a static index for LSP. Reference-resolution mismatch rates are high in practice
• Track index freshness in a catalog. Silently serving a stale index is the scariest failure mode; CodeNib's manifest and schema checks (a stale graph refuses to load rather than being misread) are the guardrail
• Measure history compaction per model. As the Haiku case shows, rolling it out uniformly can backfire

Takeaway

CodeNib's contribution is not a flashy new model — it is framing repository context as a measurable serving problem. Build views once, unify their addresses, repair only what broke, deliver only what is needed — and attach to every speed claim the exact conditions under which it holds. More than the numbers, that discipline of verification is what is worth taking home.

For the basics underneath this paper, see LLM Agents from Scratch and RAG Fundamentals.