Sampling

Sampling is the general name for how a language model turns its per-step probability distribution into a concrete next Token — it answers the question 'given these probabilities, which one do we actually write down?' The simplest form, 'argmax' or greedy decoding, always picks the most likely token; reliable but creatively flat. In practice, real systems mix techniques like Temperature, Top-K and Top-P to balance fluency and diversity. Path-search strategies like Beam Search also belong to this family, typically used for tasks like machine translation where one 'best' answer is expected. Because sampling settings shape the character of an LLM-powered product, the same model with the same prompt can feel like a different system under different sampling regimes.