In AI systems a “good answer” is often a personal judgement. To go to production, that judgement must be turned into a measurable evaluation set, real user tasks and acceptable error limits.
Measurement starts with data
Prepare a small but representative evaluation set of real questions, hard examples and unexpected inputs. Selecting only the examples the model likes shows quality better than it is.
Which dimensions to measure?
- Correctness: does the answer match the source?
- Relevance: does it truly answer the user’s question?
- Safety: does it produce sensitive information or unauthorised actions?
- Stability: does it behave consistently on similar inputs?
- Cost and speed: does it run in the workflow’s accepted time and budget?
How should human evaluation be used?
Human scoring can be valuable, but the scoring rubric, example answers and disagreement resolution must be defined. Instead of mixing human feedback directly into production data, keep it controlled for version comparison.
Every model change must be re-measured
When the model, prompt, retrieval setting or source document changes, result behaviour can change too. Connecting the evaluation set to CI or release checks prevents a “small” change from silently degrading quality.