encoding

Definition

Turning information into a form the model can consume, and sometimes the resulting representation itself. The most frequent use is positional encoding: attention is order-blind on its own, so position has to be injected explicitly, either added to the input or applied inside the attention computation. Related to but not the same as an encoder — encoding names the scheme, the encoder is the module that runs it.