word

Definition

A word in the linguistic sense, historically central through word embeddings, which mapped each word to a dense vector whose geometry encoded meaning. Modern models do not operate on words: they tokenize into subword units, so a single word may span several tokens and a token may span a word boundary. Several well-known failure modes — arithmetic on long numbers, character-level tasks, rare spellings — trace back to that mismatch.

Explainers using this term

• Embeddings from Scratch — from word2vec Intuition to Contextual EmbeddingsEfficient Estimation of Word Representations in Vector Space