Students Professional
JA EN

#vlm

14 articles

01 ·★ MEMBER·PAPER·12 min read Paper walkthrough: Qwen-Drive-1.0 — bolting 3D perception and planning onto a VLM without touching its architecture Leave the pretrained VLM exactly as it is, attach a BEV perception head and a Planning Expert, and you get 3D detection, occupancy, maps and trajectory generation in one model — general vision-language ability nearly intact, 90.7 PDMS on NAVSIM. A ground-up walkthrough, straight from the paper. 02 ·Agents·★ MEMBER·PAPER·13 min read Paper Walkthrough: Code as Worlds — An Agent That Writes the World Down as Runnable Code A close read of Code-as-World, which represents physical worlds as executable code and searches for world hypotheses through a propose–execute–render–verify loop. The three parts of an EWR, the agentic discovery loop, the QuantiPhy results, and the limits the paper itself admits. 03 ·VLMs & Multimodal·★ MEMBER·PAPER·8 min read Video Understanding from Scratch — From a Pile of Frames to a Sense of Time A video is not just a lot of images. This piece splits video understanding into three questions — which frames to take (sampling), how to mix them (temporal attention), and how to shrink a long video (compression) — and works through each with equations, interactive figures, and code. 04 ·Agents·★ MEMBER·PAPER·12 min read Paper Walkthrough: Embodied-Navigator (TAMP-Nav) — Let the VLM Just Point, and Navigation Gets Both Faster and Better Instead of asking a VLM for 3D coordinates, have it point at a 2D pixel; think and store memory only at key nodes; align it all with two-level GRPO. A ground-up walkthrough of the design that hits 66.2% SR on R2R-CE at 16.58s per task using only 90k training trajectories. 05 ·VLMs & Multimodal·★ MEMBER·PAPER·10 min read Document AI and OCR Today — How an LLM Ends Up Reading Your Invoices How machines came to read invoices and scanned PDFs, from first principles: text detection and CTC, how errors are measured (CER), LayoutLM's trick of embedding coordinates alongside words, the OCR-free Donut line, and today's habit of handing the page straight to a VLM — plus the walls that matter in production: tables, handwriting, and hallucination. 06 ·★ MEMBER·PAPER·11 min read Paper Walkthrough: SemComp-Bench — Asking Video Generators 'So Did It Actually Get Done?' SemComp-Bench moves video-generation evaluation from 'does it look good' to 'did it reach the outcome I asked for'. A walkthrough of the task definition, the four-stage data pipeline, the AND-vs-average scoring, and the measured numbers — all from the paper itself. 07 ·★ MEMBER·PAPER·12 min read Paper Review: S²VOPD — Don't Strengthen the Teacher, Blur the Student's View A walkthrough of S²VOPD, which improves vision-language models without labels, rewards, or a stronger teacher. The trick is inverting where the asymmetry comes from: degrade the student's view instead of privileging the teacher's. 08 ·Agents·★ MEMBER·PAPER·12 min read Paper Explained: Co-RL — Reasoning Without Labels, Emerging From a Diverse Cohort Grade your own answers long enough and the model collapses. Co-RL breaks that loop by rewarding each agent against a peer's majority vote, matching supervised training without touching a single ground-truth label. The mechanism, the dynamics, the numbers, and the traps — straight from the paper. 09 ·VLMs & Multimodal·★ MEMBER·PAPER·8 min read How VLMs Came Together — Wiring a Vision Encoder into an LLM A vision-language model is three parts: a vision encoder, a projection layer, and an LLM. From the bridge CLIP built, through the three schools of connector design, to resolution strategies and the implementation constraints that bite in practice. 10 ·CNNs & Image Recognition·★ MEMBER·PAPER·10 min read Paper Walkthrough: PhiZero — A World Model That Reasons in a Language of Physics Before It Renders A close read of PhiZero, which reasons about how the world will change in a discrete 'physical language' learned self-supervised from video, and only then renders the footage — instead of predicting pixels directly. The two-stage tokenizer and reasoner, the top spot on Physics-IQ Verified, zero-shot motion transfer, and the limitations the paper itself admits. 11 ·Agents·★ MEMBER·PAPER·10 min read Paper Explained: OSReward — Can You Trust the AI That Grades AI? Remeasuring Rewards for Computer-Use Agents The VLM judges that decide whether an agent driving a PC or a phone succeeded turn out to share a leniency bias that reads failure as success — a paper that exposes it with 1,019 human-verified trajectories and closes the gap with OS-Shepherd, an open reward model 30–60× cheaper, explained from zero background. 12 ·★ MEMBER·PAPER·8 min read Paper Walkthrough: Hunyuan3D-Buffalo 1.0 — One Brain for 3D Understanding, Generation, and Editing Describe a 3D shape in words, generate it from text, edit it by instruction, carve out a named part — Tencent Hunyuan's Hunyuan3D-Buffalo 1.0 does all four in a single model. We walk through the 87M-sample data factory behind it, the Nano3D-v2 editing-pair pipeline, and the paper's most practical finding: training generation makes editing better. 13 ·★ MEMBER·PAPER·8 min read Paper Walkthrough: GST-Bench — Can VLMs Build a Global Map of a Scene from Video? Show a VLM a walkthrough video of a house, then ask 'from where you're standing now, which way is the sofa?' — even the strongest model scores barely half of what humans do. A walkthrough of GST-Bench from ByteDance Seed: the shortcut-proof benchmark design, results across 22 models, and the training data that closed 27 points of the gap. 14 ·★ MEMBER·PAPER·9 min read Paper explained: DEFT-RLVR — show a driving VLM the future trajectory too early and it fabricates its reasoning Let a teacher model peek at the ground-truth trajectory while writing chain-of-thought annotations for driving, and severe hallucinations double from 29% to 50% — the paper calls this trajectory anchoring bias. This article walks through AD-MCQ, which turns planning into a verifiable multiple-choice task, and DEFT-RLVR (arXiv:2608.01755), which reveals trajectories only after the model commits to a decision, using nothing but the paper itself as the source.