#vlm

20 articles

01 ·Time Series·★ MEMBER·PAPER·11 min read Paper Explained: Holi-World — Turning First-Person Recordings into a Ledger of the Room, Then Teaching a VLM to Reason over It Holi-World (arXiv:2610.26549) automatically turns first-person recordings into a record of room geometry, furniture, and the trajectories of objects people pick up, then converts that record into about 80K spatial and temporal QA pairs to train a 4B vision-language model. A from-scratch walkthrough of the pipeline, the numbers, and the limits. 02 ·★ MEMBER·PAPER·7 min read Paper Explained: SpatialBlock — Teaching Vision-Language Models Spatial Sense with 15,000 Block-Stacking Puzzles A paper that improves spatial reasoning in vision-language models using nothing but 15,000 synthetic block-stacking questions — no real-scene 3D labels. We walk through how the data is built, the two training recipes, what actually helped, and where it falls short. 03 ·★ MEMBER·PAPER·11 min read Paper Explained: VDiff-Bench — Even Frontier MLLMs Miss the Low-Level Changes in a Spot-the-Difference Puzzle VDiff-Bench asks a multimodal LLM what changed between two near-identical images, as a four-way multiple choice. We walk through how its 1,756 questions across 10 change categories were built, what 11 models scored, why several of them answer 'no difference' when noise or texture changed, and what this means if you use an MLLM to grade image edits. 04 ·★ MEMBER·PAPER·13 min read Paper Explained: Reason Through the Latent! — Visual information being in a latent state doesn't mean the answer uses it Several latent visual reasoners keep their accuracy even when you corrupt the latent state — the answer is not actually flowing through it. CVRR (arXiv:2609.06746) starts recurrence from the question state after the image has been absorbed, then closes every other image-conditioned route before decoding, making latent computation necessary while keeping performance. A walkthrough grounded only in the paper's text, including its causal interventions. 05 ·★ MEMBER·PAPER·13 min read Paper Walkthrough: ENEAS — Telling Real Instances From Their Doppelgangers Ask for 'person' and you get the statues too. This paper fixes the failure where a mask is geometrically right and semantically wrong, using an embedding filter plus a conditional VLM judge — explained from first principles. 06 ·Agents·★ MEMBER·PAPER·11 min read Editable Visual Design, explained — demoting image generation to a sketch and shipping the artifact as HTML The image model stops being the deliverable and becomes an imagination engine plus an asset factory, while a coding agent rebuilds the piece in HTML/CSS. A walk through the paper behind posters whose layers come apart and whose text is real text. 07 ·★ MEMBER·PAPER·12 min read Paper walkthrough: Qwen-Drive-1.0 — bolting 3D perception and planning onto a VLM without touching its architecture Leave the pretrained VLM exactly as it is, attach a BEV perception head and a Planning Expert, and you get 3D detection, occupancy, maps and trajectory generation in one model — general vision-language ability nearly intact, 90.7 PDMS on NAVSIM. A ground-up walkthrough, straight from the paper. 08 ·Agents·★ MEMBER·PAPER·13 min read Paper Walkthrough: Code as Worlds — An Agent That Writes the World Down as Runnable Code A close read of Code-as-World, which represents physical worlds as executable code and searches for world hypotheses through a propose–execute–render–verify loop. The three parts of an EWR, the agentic discovery loop, the QuantiPhy results, and the limits the paper itself admits. 09 ·VLMs & Multimodal·★ MEMBER·PAPER·8 min read Video Understanding from Scratch — From a Pile of Frames to a Sense of Time A video is not just a lot of images. This piece splits video understanding into three questions — which frames to take (sampling), how to mix them (temporal attention), and how to shrink a long video (compression) — and works through each with equations, interactive figures, and code. 10 ·Agents·★ MEMBER·PAPER·12 min read Paper Walkthrough: Embodied-Navigator (TAMP-Nav) — Let the VLM Just Point, and Navigation Gets Both Faster and Better Instead of asking a VLM for 3D coordinates, have it point at a 2D pixel; think and store memory only at key nodes; align it all with two-level GRPO. A ground-up walkthrough of the design that hits 66.2% SR on R2R-CE at 16.58s per task using only 90k training trajectories. 11 ·VLMs & Multimodal·★ MEMBER·PAPER·10 min read Document AI and OCR Today — How an LLM Ends Up Reading Your Invoices How machines came to read invoices and scanned PDFs, from first principles: text detection and CTC, how errors are measured (CER), LayoutLM's trick of embedding coordinates alongside words, the OCR-free Donut line, and today's habit of handing the page straight to a VLM — plus the walls that matter in production: tables, handwriting, and hallucination. 12 ·FREE·PAPER·11 min read Paper Walkthrough: SemComp-Bench — Asking Video Generators 'So Did It Actually Get Done?' SemComp-Bench moves video-generation evaluation from 'does it look good' to 'did it reach the outcome I asked for'. A walkthrough of the task definition, the four-stage data pipeline, the AND-vs-average scoring, and the measured numbers — all from the paper itself. 13 ·FREE·PAPER·12 min read Paper Review: S²VOPD — Don't Strengthen the Teacher, Blur the Student's View A walkthrough of S²VOPD, which improves vision-language models without labels, rewards, or a stronger teacher. The trick is inverting where the asymmetry comes from: degrade the student's view instead of privileging the teacher's. 14 ·Agents·★ MEMBER·PAPER·12 min read Paper Explained: Co-RL — Reasoning Without Labels, Emerging From a Diverse Cohort Grade your own answers long enough and the model collapses. Co-RL breaks that loop by rewarding each agent against a peer's majority vote, matching supervised training without touching a single ground-truth label. The mechanism, the dynamics, the numbers, and the traps — straight from the paper. 15 ·VLMs & Multimodal·★ MEMBER·PAPER·8 min read How VLMs Came Together — Wiring a Vision Encoder into an LLM A vision-language model is three parts: a vision encoder, a projection layer, and an LLM. From the bridge CLIP built, through the three schools of connector design, to resolution strategies and the implementation constraints that bite in practice. 16 ·CNNs & Image Recognition·★ MEMBER·PAPER·10 min read Paper Walkthrough: PhiZero — A World Model That Reasons in a Language of Physics Before It Renders A close read of PhiZero, which reasons about how the world will change in a discrete 'physical language' learned self-supervised from video, and only then renders the footage — instead of predicting pixels directly. The two-stage tokenizer and reasoner, the top spot on Physics-IQ Verified, zero-shot motion transfer, and the limitations the paper itself admits. 17 ·Agents·FREE·PAPER·10 min read Paper Explained: OSReward — Can You Trust the AI That Grades AI? Remeasuring Rewards for Computer-Use Agents The VLM judges that decide whether an agent driving a PC or a phone succeeded turn out to share a leniency bias that reads failure as success — a paper that exposes it with 1,019 human-verified trajectories and closes the gap with OS-Shepherd, an open reward model 30–60× cheaper, explained from zero background. 18 ·FREE·PAPER·8 min read Paper Walkthrough: Hunyuan3D-Buffalo 1.0 — One Brain for 3D Understanding, Generation, and Editing Describe a 3D shape in words, generate it from text, edit it by instruction, carve out a named part — Tencent Hunyuan's Hunyuan3D-Buffalo 1.0 does all four in a single model. We walk through the 87M-sample data factory behind it, the Nano3D-v2 editing-pair pipeline, and the paper's most practical finding: training generation makes editing better. 19 ·FREE·PAPER·8 min read Paper Walkthrough: GST-Bench — Can VLMs Build a Global Map of a Scene from Video? Show a VLM a walkthrough video of a house, then ask 'from where you're standing now, which way is the sofa?' — even the strongest model scores barely half of what humans do. A walkthrough of GST-Bench from ByteDance Seed: the shortcut-proof benchmark design, results across 22 models, and the training data that closed 27 points of the gap. 20 ·FREE·PAPER·9 min read Paper explained: DEFT-RLVR — show a driving VLM the future trajectory too early and it fabricates its reasoning Let a teacher model peek at the ground-truth trajectory while writing chain-of-thought annotations for driving, and severe hallucinations double from 29% to 50% — the paper calls this trajectory anchoring bias. This article walks through AD-MCQ, which turns planning into a verifiable multiple-choice task, and DEFT-RLVR (arXiv:2608.01755), which reveals trajectories only after the model commits to a decision, using nothing but the paper itself as the source.