Paper explained: DEFT-RLVR — show a driving VLM the future trajectory too early and it fabricates its reasoning

Let a teacher model peek at the ground-truth trajectory while writing chain-of-thought annotations for driving, and severe hallucinations double from 29% to 50% — the paper calls this trajectory anchoring bias. This article walks through AD-MCQ, which turns planning into a verifiable multiple-choice task, and DEFT-RLVR (arXiv:2608.01755), which reveals trajectories only after the model commits to a decision, using nothing but the paper itself as the source.

Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs

A teacher who sees the answer first invents the explanation

Imagine asking a driving instructor: "The correct move here is a right turn. Now explain why." Instead of retracing an actual decision process, the instructor will construct a justification for the turn — and in the worst case, invent a "mandatory right-turn sign" that was never there.

That last part is not hypothetical. Figure 1 of this paper (arXiv:2608.01755) shows a real example: a teacher model that was shown the correct trajectory fabricated a mandatory-turn sign absent from the scene to make its explanation work (§1).

Mainstream VLA (Vision-Language-Action) models for autonomous driving pair a large VLM (vision-language model), which handles high-level reasoning and decision making, with a much smaller action expert specialized for geometric trajectory prediction (§1). Think of it as the classic end-to-end driving stack (End-to-End Driving from Scratch), which pushes pixels to control through one network, with a language layer bolted on so the decisions can be explained and checked. To sharpen the VLM's reasoning, the standard recipe is to have a teacher model write chain-of-thought (CoT) rationales as training data. The problem: most existing annotation pipelines show the teacher the ground-truth (GT) future trajectory before it writes the rationale. The paper argues, and then demonstrates, that this induces trajectory anchoring bias — the cognitive-psychology phenomenon where information supplied up front disproportionately shapes the judgment that follows (§1, §2).

The evidence: severe hallucinations double from 29% to 50%

The paper's controlled study is meticulous (§2, §B.1). The authors selected 100 causally demanding scenes — hard braking, stopping from motion, sharp turns — and used the same teacher (Qwen3.5-397B-A17B), the same 12 video frames, a byte-identical system prompt, and temperature 0. The only difference between the two arms is one contiguous block: the 10 raw waypoints of the logged future trajectory, present in one arm and absent in the other.

Two human annotators, blind to which condition produced each output, scored all 200 CoTs on grounding, absence of hallucination, specificity, and causal coherence (§B.2):

• Severe hallucination rate: 29.0% without the trajectory, 50.0% with it (Table 1)
• Pairwise preference: rationales written without the trajectory win 60.5% of judgments; with it, only 24.0% (Table 1)

Handing the model the answer up front doesn't improve its explanations — it pushes the model toward fabricating scene evidence to justify a foregone conclusion. And the degradation concentrates exactly where reliable reasoning matters most: the causally hard scenes (§1).

The obvious fix — hide the trajectory and let the model decide from the scene — creates a new problem. Open-ended trajectory generation entangles high-level decision making with millimeter-scale geometric synthesis and low-level vehicle dynamics in a single task (§1), which is a poor match for what language models are good at.

Tool number one: turn driving decisions into a multiple-choice question — AD-MCQ

So the paper converts planning into a multiple-choice question. AD-MCQ (Autonomous-Driving Multiple-Choice Question) presents each scene with a small set of explicit candidate trajectories and asks the model to pick one (§3). A choice can be verified exactly, by machine.

Where do candidates come from? The authors take roughly 490,000 logged 5-second trajectories (10 waypoints at 0.5-second intervals), run K-means, and obtain a codebook of $K{=}8192$ motion prototypes. Any trajectory is discretized by assigning it to its nearest prototype (§3):

Put in words: walk through all ten time steps, measure how far your trajectory's waypoint $\mathbf{p}_t$ sits from prototype $k$'s waypoint $\mathbf{c}_{k,t}$, add those gaps up, and keep the prototype number $z$ whose total is smallest. It is a nearest-neighbour lookup — "which of the 8,192 stock motion shapes is closest to the one actually driven?" The choice of $K{=}8192$ balances reconstruction fidelity (0.290 m average error even on unseen trajectories) against prototype utilization (92.0%) (§3, §C.5).

Each question mixes the one correct prototype with hard negatives — wrong answers whose similarity $\rho$ to the oracle falls in a controlled band — into a six-option set. Because the options preserve trajectory-level distinctions in braking time, speed profile, and lateral geometry, the task probes far finer judgment than coarse meta-actions like "turn left / go straight" (§3).

The core idea: DEFT — reveal trajectories only after the decision

There's a catch, though. If the candidates are visible from the start, the GT-trajectory anchor is simply replaced by a candidate-set anchor: the model spends its reasoning comparing options against each other instead of reading the scene (§1).

DEFT (Deferred Exposure of Future Trajectories) breaks that shortcut by splitting the interaction into two turns (§4.1):

• Turn 1 (causal decision reasoning): the model sees only the scene video, ego state, and navigation instruction. It traces how the evidence leads to driving implications and commits to a high-level decision (HLD) — before any candidate exists.
• Turn 2 (trajectory grounding): only now are the six candidates revealed. The recorded decision is treated as binding, and candidate geometry is used solely to find its closest explicit realization.

The future trajectory is transformed from a pre-decision premise into a post-decision verification target — that is what the paper's title means. The two turns stay separate at inference but are optimized jointly as a single rollout with GRPO (Group Relative Policy Optimization) (§4.2).

Reward design: a correct answer alone earns no praise

If the reward is just answer correctness $R^{\mathrm{MCQ}}$ (1 if right, 0 if wrong), reinforcement learning happily reinforces lucky guesses and shortcut reasoning that happened to land on the right option (§4.3). DEFT-RLVR therefore grades the reasoning process too — but only for rollouts that got the answer right.

For each question, a rubric is generated once, offline: a VLM looks at the scene and writes 6–10 atomic, weighted criteria describing the concrete evidence a good CoT should mention (§4.3). During training, a text-only grader checks the Turn-1 reasoning trace against those criteria, and the weighted pass rate becomes the rubric reward:

which says the final reward is a product of two things: "was the choice correct (0 or 1)" times "what weighted fraction of the criteria $c_{i,k}$ did the reasoning actually satisfy" — each criterion ticked off in $\mathbf{b}$ counts for its weight $w_{i,k}$, and dividing by the sum of the weights keeps the score between 0 and 1. Because the two factors are multiplied, a wrong answer scores zero no matter how elegant the reasoning. Crucially, the GT trajectory is never given to the rubric generator, the policy, or the grader (§4.3). The grader sees no images and no candidates — only text. This offline-rubric, text-only-grading design makes each training step 41.1% faster than an online image-conditioned grader, while adding just 0.5% overhead over correctness-only training (§5.4, Table 4).

Results: better driving reasoning without sacrificing general vision

Headline numbers on Qwen3-VL-8B-Instruct (§5.2, Table 2):

Merely restructuring the prompt into two turns jumps accuracy from 28.1% to 56.6%; RLVR pushes it to 77.9%. Under the identical correctness-only reward, the ablation that shows candidates up front (JEFT+RLVR) stalls at 61.1% — exposure order alone accounts for a 15.3-point gap (§5.4). Qwen3.5-4B tells the same story: 34.0% → 82.2% (Table 2).

The general-capability column is the remarkable part. Distilling teacher responses via SFT reaches the highest raw accuracy (84.1%) but drags the general vision average from 54.81% down to 51.80%, and cold-start SFT starts eroding general capability from the very first epoch (§5.3). DEFT-RLVR instead nudges it up, 54.81% → 56.09%. The paper attributes this to RL's fine-grained updates — merely raising or lowering the probability of tokens the policy itself sampled — and to Turn-1 causal reasoning exercising visual-spatial skills shared with the general benchmarks (§5.3). The gains also transfer out of domain: on 500 nuScenes scenes, accuracy rises from 39.6% to 49.5% (Table 3).

Why not just generate trajectories directly?

"Why not add 8,192 trajectory tokens to the vocabulary and generate them?" The paper kills this alternative with a diagnostic experiment (§5.5, §D.6). After SFT on 100,000 examples, training loss keeps falling, but held-out trajectory error bottoms out around epoch 8 and then climbs; the best result, 1.963 m, sits far above the codebook's own quantization floor of 0.279 m — so token inference, not quantization, is the bottleneck. Worse, general visual capability collapses: without CoT the 12-benchmark mean falls from 53.49% to 0.67% by epoch 8, and adding CoT only slows the decay to 21.50% (§D.6).

It is the failure mode you miss whenever you read the training curve alone and conclude that learning is going well (Overfitting and Evaluation Design). Forcing a large trajectory-token inventory into the model's vocabulary perturbs its pretrained distribution. That is why AD-MCQ uses the codebook only to retrieve candidates and externalizes generation as selection — the heart of the design (§5.5).

Limitations and what remains unverified

• Selection accuracy is not driving quality. The paper explicitly does not assume the logged future is the unique safe trajectory (§D.1). There is no real-vehicle or closed-loop simulation evaluation, and the authors position AD-MCQ as closer to a verification interface than a replacement for a downstream continuous planner (§A.2).
• Distillation still wins on raw OOD accuracy. On nuScenes, mixed-target distillation reaches 55.8% versus DEFT-RLVR's 49.5% (Table 3). RLVR's selling point is improving while preserving general capability, not topping every leaderboard.
• CoT quality metrics are LLM-judged. CFS and HLD are scored by Qwen3.5-397B-A17B. Agreement with humans is validated (Spearman 0.78 for CFS, Cohen's κ 0.84 for HLD, §E), but they remain automatic proxies.
• Long-tail quantization error persists. Even large codebooks leave rare maneuvers poorly reconstructed — 99th-percentile ADE stays above 1 m in the best configuration (§C.4).

How this is used in practice

The lesson here reaches well beyond autonomous driving: it applies to every team that generates CoT training data with a teacher model.

• If you build annotation pipelines: audit your prompt templates today. If the GT label, reference answer, or gold code appears in the prompt that asks for a rationale, that's exactly the anchor this paper describes. Restructure to solve-then-verify: let the teacher answer first, and use the ground truth only to filter or verify the generated reasoning afterward.
• If you run RLVR post-training: the knobs are all in the paper — GRPO group size 16, KL coefficient 0.001, rollout temperature 1.0 / top-p 0.95, and difficulty is controlled by candidate count $M$ and the similarity band $[\rho_{\min},\rho_{\max}]$ (§5.1, §D.3, §D.7). If you add rubric rewards, generate rubrics offline and grade with text only; online image-conditioned grading makes each training step 1.7× slower (Table 4).
• Pitfalls that become incidents: (1) task-specific SFT starts destroying a general-purpose VLM's broad capability from epoch one — monitor general-benchmark regression alongside your training curves, or you'll discover post-release that unrelated features broke. (2) An evaluation that shows options before reasoning measures skill at comparing options, not scene understanding. (3) Internalizing action tokens via vocabulary expansion is a double bottleneck: you lose both trajectory precision and general capability (§5.5).

Read next

• DAPD — breaking the teacher's "cheat-sheet illusion" in distillation — a sibling paper on the same pathology, teachers corrupted by seeing the answer, in the LLM distillation setting
• BEV representation — the bird's-eye view behind self-driving — the spatial representation downstream of the multi-camera input this paper's VLM consumes
• Instruction tuning and RLHF — the post-training landscape where RLVR lives