Symmetry and Equivariance — How Group Theory Shapes Network Design

How to bake "a shifted cat is still a cat" into the architecture itself — from the four axioms of a group, through the definition of equivariance, to why convolution is the only translation-equivariant linear map and how AlphaFold handles 3D geometry. Including the cases where imposing symmetry backfires.

Group Equivariant Convolutional Networks

A Cat Is Still a Cat, Upside Down

Shift a photo of a cat ten pixels to the right and it is still a cat. Tilt it ninety degrees and it is still a cat. Obvious to us, not remotely obvious to a neural network. Shift the pixels and the input vector becomes an entirely different list of numbers — to a plain fully-connected layer, an unrelated input. The knowledge that "these are the same thing" has to come from somewhere.

There are two places it can come from. You can teach it with data: show enough shifted and rotated photos that the model learns, empirically, to answer the same way (augmentation). Or you can build it into the structure: make "shift the input and the answer just shifts with it" a property of the layer itself, true before training starts.

The language for the second option is group theory, and its two central words are invariance and equivariance. Why CNNs dominate images, why Transformers need positional encoding, why AlphaFold could handle atomic coordinates at all — these all sit on one map.

"Doesn't Change" vs. "Moves Along With"

Start by separating the two words. A group photo makes it concrete.

• Invariant: everyone steps one pace to the right, and the number of people in the frame is unchanged.
• Equivariant: everyone steps one pace to the right, and each face's position shifts one pace right too.

Head count is the invariant; face position is the equivariant quantity. Equivariance is not "nothing happens" — it is "things change in a way you can predict." That distinction drives design: intermediate layers should be equivariant, and usually only the very end should be invariant.

The reason is that going invariant early throws information away. Collapse to "position-free features" in layer one and you can never recover where the eyes sit relative to the ears. Stay equivariant and the positional information flows onward, available until you choose to discard it. Stack equivariant layers, then make it invariant at the end — geometry-aware networks are almost all built on that single template.

The most familiar invariant of all is the dot product. Rotate two vectors together by the same angle and the dot product doesn't budge. Every individual component changes; only the combined quantity survives.

Groups — Naming a Collection of Operations

"Shift," "rotate," "reorder": strip those down to their shared structure and you get a group. The definition is four lines. A set of operations $G$ is a group when:

1. Doing two operations in a row lands you back inside $G$ (closure)
2. Chaining three or more gives the same result however you bracket them (associativity)
3. There is a "do nothing" operation $e$ (identity)
4. Every operation has an "undo" (inverse)

Rotations make it click. Ninety degrees followed by one hundred eighty is two hundred seventy (closure), zero degrees is do-nothing (identity), and two hundred seventy undoes ninety (inverse). So the four quarter-turns form a genuine group, written $C_4$.

The groups you actually meet in practice are a short list.

A group is an abstract collection of operations, but to act on data it needs to be matrices. Assign each element $g$ a matrix $\rho(g)$ such that $\rho(g)\rho(h) = \rho(gh)$, and you have a representation. From here on, "acting with the group" means multiplying by one of these matrices.

Writing Equivariance Down

A map $f$ — a whole network, or a single layer — is equivariant with respect to a group $G$ when:

Here $x$ is the input, $\rho_{\text{in}}(g)$ is the matrix that performs $g$ on the input side, and $\rho_{\text{out}}(g)$ performs the same $g$ on the output side. All the equation says is that "move first, then process" and "process first, then move" give the same answer. You may swap the order. That's it.