biases

Definition

Two distinct senses share the word. The social sense is skew in data or model behavior, the subject of fairness work. The technical sense is an additive term: the bias parameter in a linear layer, or an attention bias that adds a distance-dependent offset to attention logits, the mechanism behind ALiBi-style position handling. Papers about position and attention almost always mean the second.