sampling

Definition

Drawing a value according to a probability distribution. In generation it names the decoding step itself — each token is sampled from the model's predicted distribution, with temperature and top-p shaping how adventurous that draw is, and greedy decoding as the degenerate case. The statistical sense of drawing observations from a population is the same idea, but in ML papers the word usually means the generative procedure.