Paper Explained: FreeFlow — Optical Flow with a Hierarchical Transformer and No Flow-Specific Parts

FreeFlow drops correlation volumes, warping, and iterative refinement, and reaches the top of Sintel, KITTI-2015, and Spring using only window, shifted-window, and low-resolution global attention. We walk through it from zero.

FreeFlow: A Bias-free Hierarchical Transformer for Optical Flow Estimation

What this paper claims

The paper is "FreeFlow: A Bias-free Hierarchical Transformer for Optical Flow Estimation" (Bargatin, Yakovenko, Abud, Vatolin, September 2026, arXiv:2609.11486, accepted at ECCV 2026).

Here is the abstract in plain terms. Optical flow is the task of finding, for every pixel, where it moved between two video frames. To get high accuracy, flow methods have leaned on task-specific machinery (inductive biases) such as correlation volumes, feature warping, and iterative refinement. These work, but they lock the model into predefined heuristics, which can limit what it can express, make pipelines more complex, and add compute. The authors build FreeFlow, a hierarchical transformer with none of these flow-specific components, just a single feed-forward encoder–decoder. It mixes three kinds of attention: window attention for local processing, shifted-window attention to pass information across windows, and global attention at reduced resolution. The design scales cleanly, getting consistently better from small to large variants. Despite having no standard inductive biases, it reaches state-of-the-art results on Sintel (0.68/1.48 EPE on Clean/Final), KITTI-2015 (3.23 Fl-all), and Spring (3.192 1px), while staying memory-efficient at 1080p.

In short: throw away the purpose-built motion toolkit, use a general transformer, and it ends up doing best.

An analogy: spot-the-difference on every pixel

Optical flow is like laying two nearly identical photos side by side and answering "where did this point go?" for every single pixel. At 1080p that is roughly 2.07 million pixels, each getting an arrow like "3.2 pixels right, 0.5 down."

A person doing this would first compare the immediate neighborhood (small motions). For a ball that flew across the frame, they would step back and scan the whole picture to get a rough idea. For thin poles and object edges, they would zoom in. FreeFlow's three attention types map neatly onto "look nearby," "peek into the next block," and "shrink the whole image and scan it."

The specialized parts earlier methods accumulated

Before deep learning, flow was solved as an optimization problem under assumptions like "brightness stays constant as things move" and "motion is smooth" (§2.1). Early deep learning simplified things: FlowNet showed a network could regress flow directly. But later high-accuracy methods brought the classical ideas back in as hard-wired modules (§1):

• Correlation volume: a table of similarities between every pixel in frame 1 and candidate positions in frame 2
• Warping: pull frame 2 toward frame 1 using the current estimate, then solve only for the leftover offset
• Iterative refinement: nudge the estimate over many passes (popularized by RAFT)
• Convex upsampling: enlarge a low-resolution estimate with learned weights

Table 1 in the paper lists which of these PWC-Net, RAFT, FlowFormer, WAFT, GeoViT, CroCo-Flow and others use. FreeFlow is the only row with "no" in every column, and its output head is just a few convolutions.

Meanwhile, in detection, segmentation, and depth estimation, general-purpose Vision Transformers have been replacing bespoke pipelines (§1, §2.2). For ViT basics, see Vision Transformers explained. The authors ask: can optical flow shed its specialized parts too and still reach the top?

The big picture: read both frames, cross-reference, draw arrows

The skeleton follows CroCo/DUSt3R (§3.1):

1. Cut each image $I_1, I_2$ into $8\times8$ pixel patches and turn them into token sequences
2. Encode both with a weight-shared encoder into features $F^1, F^2$
3. A decoder alternates self-attention with cross-attention in which $F^1$ queries $F^2$, producing flow tokens $Z$
4. A small head maps $Z$ back to a full-resolution flow field

The token count is the image size divided by the patch size along each axis:

Put simply, the number of tokens is the number of 8×8 tiles you need to cover the image. At 1080p (1920×1080) that is $240\times135=32{,}400$ tokens (our arithmetic, not a figure from the paper).

Attention cost grows with the square of the token count, so comparing all 32,000 tokens with each other is too expensive. Drag $n$ in the figure below to see how steeply the quadratic curve climbs.