1 | Name what is being evaluated

Identify whether the object is retrieval, generation, routing or page experience. Different objects need different criteria; visual impression and operational reliability are not one conclusion.

2 | Fix inputs and conditions

Retain questions, source versions, user roles and evaluation dates. Results after input changes form a new run rather than being compared indiscriminately with older conditions.

3 | Define acceptable outcomes

Correct answers, inspectable references, correct scope and appropriate hand-offs are distinct requirements. Record why each matters and which user action is affected by a failure.

4 | Classify problems

Separate content errors, version errors, missing references, interaction failures and visual problems. Record reproduction steps and evidence so each issue can be acted upon.

5 | Recheck after changes

After a fix, repeat the relevant checks under the same conditions and examine adjacent flows. Record the result, not merely that a change was made.

6 | Keep the conclusion within its scope

Passing a set of examples means those examples passed under the stated conditions, not that every situation has been proven. A well-scoped conclusion supports further iteration.

Open / print working paper