◉ CLOSE READING
ReAct: Synergizing Reasoning and Acting in Language Models
ReAct: Synergizing Reasoning and Acting in Language Models
Abstract
While large language models (LLMs) have demonstrated impressive performance across tasks in language understanding and interactive decision making, their abilities for reasoning (e.g. chain-of-thought prompting) and acting (e.g. action plan generation) have primarily been studied as separate topics.
A concessive opening — the standard first move of an abstract: grant the field its wins, then name the gap in the same breath. Everything the paper will do is compressed into four words, “as separate topics”: reasoning and acting each have a literature, and neither reads the other. Note the hedge in “primarily”, which leaves room for the prior work Section 5 has to acknowledge; the claim is a gap, not a vacuum. The parenthetical examples are ballast — skip them on a first pass and the sentence still carries.
interactive decision making
A setting where the agent receives an observation, returns an action, and repeats until the goal is met — as opposed to answering once and stopping.
chain-of-thought prompting
Asking for the intermediate reasoning in prose before the answer. Named this early so the abstract has something to pick a fight with.
In this paper, we explore the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with and gather additional information from external sources such as knowledge bases or environments.
Everything after the colon is the mechanism, and it is deliberately symmetric: reasoning maintains the plan, acting supplies the facts. That two-way arrow is the contribution — a paper with only one direction would be planning, or tool use, but not this. “In an interleaved manner” is the method itself rather than a stylistic flourish; read past it and the figures later will not parse. Note also how modest the verbs are — explore, allowing for — an abstract still warming up before it quotes a number.
interleaved
Two kinds of output alternating in a single stream. The word is the method; nothing else in this sentence separates ReAct from ordinary tool use.
reasoning trace
The sequence of thoughts a model emits — the process, treated as an object in its own right rather than as a route to the conclusion.
synergy
The claim that the combination beats either part alone. It is the noun behind the title's “Synergizing”, and a claim the experiments are obliged to earn.
We apply our approach, named ReAct, to a diverse set of language and decision making tasks and demonstrate its effectiveness over state-of-the-art baselines in addition to improved human interpretability and trustworthiness.
Where the method acquires its name, in the standard slot: our approach, named X. The comparison class always sits after “over”, so read that phrase before the adjectives — “state-of-the-art baselines” is doing more work here than “effectiveness”. The tail, “in addition to improved human interpretability and trustworthiness”, opens a second axis with no number attached to it anywhere in the abstract. It is also, in hindsight, most of why this paper is cited.
state-of-the-art baseline
The strongest existing method at the time of writing. Always check whether the one being beaten was prompted or trained; this paper compares against both, and the distinction changes what the win means.
Background — Prompting and in-context examples
No model is trained anywhere in the main experiments. The weights stay fixed, and the only lever is the text placed in front of the question: a handful of worked examples, called in-context examples, which the model continues by pattern.
A few examples is few-shot, one is one-shot, none is zero-shot. For ReAct the examples are hand-written traces shaped Thought → Action → Observation, so what the model imitates is a format rather than an answer.
Fine-tuning — actually updating weights — appears once, as a supplementary experiment late in Section 3. Keep the two apart while reading the tables: almost every result in this paper was bought with a page of text, not a training run.
Concretely, on question answering (HotpotQA) and fact verification (Fever), ReAct overcomes prevalent issues of hallucination and error propagation in chain-of-thought reasoning by interacting with a simple Wikipedia API, and generating human-like task-solving trajectories that are more interpretable than baselines without reasoning traces.
“Concretely” is the hinge where an abstract stops asserting and starts naming datasets; “Specifically” does the same job. The sentence has the shape overcome A by doing B — cut it at “by” and you have the problem (hallucination, error propagation) and the remedy (a Wikipedia API) without reading anything else. “Prevalent” is a softener that charges a whole line of work with a defect while sounding like an observation. Note whose defect it is: chain-of-thought reasoning, the method these authors build on.
hallucination
Output asserted as fact that is not. The specific failure a retrieval loop is meant to prevent, and the one this paper measures by hand rather than by metric.
error propagation
One wrong step contaminating everything downstream. The reason a long reasoning chain with no external check is fragile in a way a short one is not.
Furthermore, on two interactive decision making benchmarks (ALFWorld and WebShop), ReAct outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples.
“An absolute success rate of 34%” — absolute, not relative, and that single word is the difference between an honest number and a press release. Whenever you meet “respectively”, stop and recount the list it points back at: 34% belongs to ALFWorld, 10% to WebShop. The real claim is in the concessive tail. One or two examples, against imitation- and reinforcement-learned baselines that consumed tens of thousands of trajectories — spending nothing on training while beating systems that spent everything is the asymmetry being sold.
absolute success rate
A gap measured in percentage points rather than as a ratio. Stating it as absolute forestalls the reader inflating 45→71 into a much larger-sounding relative gain.
benchmark
A fixed task and dataset that lets competing methods be compared under identical conditions — and, over time, quietly defines what the field counts as progress.
1 Introduction
A unique feature of human intelligence is the ability to seamlessly combine task-oriented actions with verbal reasoning (or inner speech, Alderson-Day & Fernyhough, 2015), which has been theorized to play an important role in human cognition for enabling self-regulation or strategization (Vygotsky, 1987; Luria, 1965; Fernyhough, 2010) and maintaining a working memory (Baddeley, 1992).
An AI paper opening on developmental psychology is a motivation move, not evidence: the citations license the design rather than support the results. “Which has been theorized to play an important role” is a careful passive — the authors are reporting someone else's claim and are not on the hook for it. What actually matters downstream is the verb phrase “combine … with”. The architecture is that phrase, restated in tokens.
verbal reasoning
Thinking in language rather than in action. Treated here as interchangeable with inner speech, borrowed from psychology to motivate a design choice.
working memory
The short-term store holding what you are working on right now. Cited as one function inner speech is thought to serve — and, not coincidentally, what a growing context window does for a model.
Consider the example of cooking up a dish in the kitchen. Between any two specific actions, we may reason in language in order to track progress (“now that everything is cut, I should heat up the pot of water”), to handle exceptions or adjust the plan according to the situation (“I don’t have salt, so let me use soy sauce and pepper instead”), and to realize when external information is needed (“how do I prepare dough? Let me search on the Internet”).
A homely example doing structural work. The three quoted thoughts are not decoration — track progress, handle an exception, notice that outside information is needed — and each reappears as a thought type in Section 2 and again as an annotation instruction for ALFWorld in Section 4. When an introduction enumerates with parallel infinitives (to track / to handle / to realize), the author is counting something they intend to reuse. Mark the list.
handle exceptions
Recovering when the situation departs from the plan. One of the three thought types this cooking example quietly introduces before the method section formalises them.
On one hand, properly prompted large language models (LLMs) have demonstrated emergent capabilities to carry out several steps of reasoning traces to derive answers from questions in arithmetic, commonsense, and symbolic reasoning tasks (Wei et al., 2022).
“On one hand” promises an “on the other hand”: the introduction is about to split prior work into two camps so that it can complain nobody joined them. “Properly prompted” quietly concedes that the ability is not reliably present — it belongs to the prompt as much as to the model. “Emergent capabilities” was the term of the moment in 2022 and has been argued over since, since much of the apparent discontinuity depends on how the metric is defined. Read it as a citation to Wei et al., not as settled fact.
emergent capabilities
Abilities that appear with scale without being trained for. A contested label — sharp thresholds often turn out to be artefacts of all-or-nothing metrics — but standard usage at the time of writing.
symbolic reasoning
Reasoning by manipulating symbols under fixed rules, such as concatenating final letters of words. One of the task families where chain-of-thought was first shown to pay off.
Background — Chain-of-thought, and the CoT-SC variant
Instead of asking for the answer, you ask the model to work the problem through in prose and take the answer at the end. Wei et al. (2022) showed that this alone lifts accuracy on arithmetic and commonsense tasks, with no change to the model.
The interesting part, for this paper, is what it cannot do. Every intermediate step is produced from the weights alone, so nothing in the loop can check a step against the world — which is exactly the opening ReAct attacks with the phrase “static black box”.
CoT-SC appears throughout the results tables and is the self-consistency variant: sample 21 chains at temperature 0.7 and take the majority answer. It is a strictly stronger baseline than plain CoT, and costs 21 times the inference to run.
However, this “chain-of-thought” reasoning is a static black box, in that the model uses its own internal representations to generate thoughts and is not grounded in the external world, which limits its ability to reason reactively or update its knowledge. This can lead to issues like fact hallucination and error propagation over the reasoning process (Figure 1 (1b)).
The paragraph that names the opponent, and worth reading slowly, because two of its adjectives become axes for the entire evaluation. “In that” introduces grounds for the assertion just made — narrower than “because”, and a signal that what follows is definitional rather than causal. Static pairs with the dynamic reasoning claimed three paragraphs on; grounded pairs with the “more grounded, fact-driven” verdict in Section 3.3. Hold those two oppositions and the results section reads as a scorecard on them.
static black box
The paper's label for reasoning that neither exposes its inputs to inspection nor admits new ones mid-flight. Two complaints packed into one metaphor; only the second is what ReAct fixes.
grounded
Anchored in something outside the model — an observation, an environment, a retrieved page. An old term in AI, and one pole of this paper's central opposition.
Beyond such simple embodied tasks to interact with a few blocks, there have not been studies on how reasoning and acting can be combined in a synergistic manner for general task solving, and if such a combination can bring systematic benefits compared to reasoning or acting alone.
“There have not been studies on …” is the load-bearing sentence of any introduction: it is where a paper earns the right to exist. Two questions are posed, joined by “and if” — can the two be combined, and is the combination actually worth anything — and the experiments answer them in that order. The closing phrase, “compared to reasoning or acting alone”, is a promise about method: it commits the authors to CoT-only and Act-only baselines, which is why Section 3.2 builds baselines by deleting parts of ReAct instead of importing other systems.
embodied task
A task involving action on an environment — a robot, or a simulator standing in for one. The qualifier “simple” before it is what makes the gap claim survive the prior work being dismissed.
ReAct prompts LLMs to generate both verbal reasoning traces and actions pertaining to a task in an interleaved manner, which allows the model to perform dynamic reasoning to create, maintain, and adjust high-level plans for acting (reason to act), while also interact with the external environments (e.g. Wikipedia) to incorporate additional information into reasoning (act to reason).
The definition, and the two parentheticals are the whole paper in four words each: reason to act, act to reason. They recur verbatim later, so they are worth memorising rather than parsing twice. Grammatically the sentence is hard because the subject of “allows” is not any noun in view but the entire preceding idea — generating both kinds of output interleaved — a construction that turns up whenever a method sentence has to refer to itself. “Pertaining to” is a stiffer “relating to”; no meaning rides on it.
dynamic reasoning
Reasoning that can change course as new observations arrive. Defined here purely as the negation of static, which is why it needs the closed loop to mean anything.
high-level plan
A coarse sequence of subgoals rather than individual operations. Maintaining one across many steps is the job thoughts are being asked to do.
Figure 1 (1) of the original runs all four styles on the same HotpotQA question, side by side.
Redrawn from the text describing Figure 1 (1) of the original. Read the four as one family rather than as four systems: Section 3.2 builds the baselines by stripping elements out of a ReAct trajectory, so all that separates the rows is what has been deleted.
§
Members-only from here
The rest of the close reading and the glossary are for members. $4.99/mo, cancel anytime.
Comments
Sign in to comment