Paper Deep Dive: AskChem — Changing the Unit of Search from Papers to Provenance-Carrying Claims

An NYU-led systems paper that swaps the retrieval unit of chemistry literature search from papers to claims grounded by a DOI and a verbatim quote. We walk through the 2.4M-claim index, evidence graph, and hybrid search design, the evaluation that took DOI resolution from 88.3% to 100%, and the paper's honest limitations.

AskChem: Claim-Centered Infrastructure for Chemistry Literature Synthesis

The problem this paper solves

"What electrocatalysts have been reported for CO2 reduction to CO, and at what Faradaic efficiency?" — the answer to a question like this doesn't live in any single paper. Small assertions about catalysts, reaction conditions, measurements, and mechanisms are scattered across dozens of publications (§1). Yet existing literature search returns a ranked list of documents and nothing more. A human has to open each paper, find the relevant passage, verify the numbers, and assemble the answer by hand.

This inconvenience became a real hazard once LLM agents started doing literature surveys. The paper points out that agents inherit the limitations of their retrieval tools — and when asked to answer from parametric memory instead, an LLM may fabricate plausible-looking citations (§1).

The answer from the NYU group (with Kyunghyun Cho among the authors) is simple: change the unit of retrieval from the paper to the provenance-carrying claim. AskChem currently indexes 2.4 million claims from 147K papers and is live not only as a web interface but also via REST, SDK, and MCP for AI agents (§1, Abstract).

A metaphor: the library that hands you books vs. the one that hands you index cards

A conventional search engine is a library whose answer is "it's somewhere in one of the books on this shelf" — and then hands you the pile. Reading is your job.

AskChem is a library where the librarian has already read every book and written one index card per fact, each card noting not just the assertion but exactly which book and which sentence it came from. Ask a question and you get a stack of cards, not books — and any suspicious card can be traced back to the original line instantly. Searching and verifying become the same motion.

What exactly is a "claim"?

In the paper's definition, a claim is an atomic (indivisible), typed scientific assertion, always grounded by a source DOI and a verbatim quote — or, where no contiguous quote exists, an explicit evidence locator (§1, §2). It also carries structured fields such as reactants, conditions, and measurements, plus an extraction confidence score (§2).

Here is a real record from the index (abbreviated, from Appendix B):

{"claim_type": "reaction",
 "source_doi": "10.1002/anie.201914977",
 "reaction_type": "electrocatalytic CO2 reduction (to CO)",
 "reactants": [{"name": "CO2", "role": "substrate"},
               {"name": "Ni SA-N2-C", "role": "catalyst"}],
 "outcomes": {"selectivity": "CO Faradaic efficiency 98%"},
 "verbatim_quote": "the Ni SA-N2-C catalyst ... achieves very high
   CO Faradaic efficiency (98%) and turnover frequency (1622 h-1),"}

Right next to the number "98% efficiency" sits the original sentence it was taken from — that is the backbone of the whole system. Under the same claim identity hang the paper metadata (Source), the claim's positions in the taxonomy (TreeNode), and its relations to other claims (Edge), so search, hierarchical browsing, and graph traversal all return the same provenance-bearing objects (§2). The implementation is refreshingly unglamorous: SQLite with FTS5 full-text search plus a vector index, served through FastAPI (§2).

Retrieval runs on two legs: "do the words match?" and "is the meaning close?" The foundation of the second leg is nothing more than the vector dot product we covered in the linear algebra article.

How claims get made

Extraction runs as two complementary pipelines (§3). A high-throughput extractor processes abstracts at scale, while a deeper extractor reads full-text PDFs and captures claim types that rarely appear in abstracts — hypotheses, limitations, surprising findings. Per Appendix B, the former uses GPT-5-mini and the latter Gemini 3.1 Pro with native PDF input via Vertex AI batch; every output must pass JSON schema validation (required provenance fields, numeric ranges, chemistry-specific fields).

The paper is honest about what this validation means. In the current index, 100% of the 2.4M claims carry a claim type, a source DOI, and a verbatim quote — but the authors state plainly that this establishes traceability, not proof that every extraction semantically interprets its source correctly (§3, §7 RQ1).