An eval is the test suite — or framework that runs it — that quantifies a model or AI system's performance on a given task. OpenAI's "Evals" library, released in 2022, helped popularize the term, and every serious AI product now writes its own. The umbrella covers academic Benchmarks like MMLU, HumanEval, and GSM8K, as well as domain-specific evals against "our actual user requests." The line "an LLM product without evals is software without tests" has become a shared refrain in the AI engineering community.
MEVZU N°124ISTANBULYEAR I — VOL. III
Glossary · Beginner · 2022
Eval
A test suite that scores a model or system against predefined criteria.
- EN — English term
- Eval
- TR — Turkish term
- Eval — Değerlendirme