units

Definition

The elementary computational elements of a network — neurons, or the count of them in a layer, which sets the model's width. The word is also borrowed for larger functional blocks, as in Gated Recurrent Unit, where a unit is an entire gating mechanism rather than a single neuron. Which sense applies depends on whether it is being counted or named.

Explainers using this term

• Build Your Own BPE Tokenizer — Learning Merge Rules, and Getting Punished by JapaneseNeural Machine Translation of Rare Words with Subword Units
• Tokenizers from Scratch — The Unit an LLM Cuts the World IntoNeural Machine Translation of Rare Words with Subword Units

In the textbook

• From Logic Gates to Arithmetic Units