Top-K sampling is a Sampling strategy that, when an LLM picks the next Token, restricts the choice to the K highest-probability candidates and zeroes out the rest. Popularised by Fan et al.'s 2018 work on open-ended generation, it cuts down on the incoherent or 'wild' tokens that pure random sampling sometimes produces. The downside is that a fixed K is a blunt tool — sometimes 50 candidates are reasonable, sometimes only 5, which is why most modern APIs offer Top-P (nucleus) instead of, or alongside, top-k. Tuned together with Temperature, it lets you fine-tune just how 'free-wheeling' the output should be.
MEVZU N°124ISTANBULYEAR I — VOL. III
Glossary · Intermediate · 2018
Top-K Sampling
A sampling strategy that picks the next token from only the K most likely candidates.
- EN — English term
- Top-K Sampling
- TR — Turkish term
- Top-K Örnekleme