A decoder is the generative half of a Transformer: at each step it produces a new Token conditioned on the ones before, repeating this loop until it stops. Decoder-only models — the GPT family, Llama 3, Claude — are the dominant architecture of the modern generative-LLM era; that is exactly what 'Autoregressive language modelling' means. During training they use masked Self-Attention so each position can only attend to the context on its left, never peeking ahead. The decoder is also where Sampling choices like Temperature, Top-K and Top-P take effect.
MEVZU N°124ISTANBULYEAR I — VOL. III
Glossary · Intermediate · 2017
Decoder
The Transformer component that generates the next token conditioned on what came before.
- EN — English term
- Decoder
- TR — Turkish term
- Çözücü (Decoder)