Quick answer: Before changing an AI model or prompt, run the current and proposed configurations against the same versioned questions and evidence. Grade correctness, grounding, completeness, clarification and appropriate refusal—not just whether the API responds. Repeat trials, review important failures manually and set release criteria before seeing the results. Official evaluation guidance recommends task-specific tests and continued evaluation as the application changes. (developers.openai.com)
This workflow is for website owners and developers operating support chatbots, documentation assistants or similar AI features. It evaluates answer quality; it does not replace availability monitoring or website performance testing.
1. Define what a successful answer must do
Start with the assistant’s job, not the model’s advertised capabilities. A support chatbot might need to explain published policies, ask for missing details and direct account-specific questions to an approved support channel.
Write measurable requirements before editing the prompt. Anthropic’s evaluation documentation recommends specific success criteria and quantitative metrics or consistently applied qualitative scales. (platform.claude.com)
For your application, use requirements such as:
- Answer published policy questions without adding unsupported conditions.
- Include qualifications that change the user’s next step.
- Ask a focused question when the request is ambiguous.
- Acknowledge when the supplied evidence cannot answer the question.
- Avoid claiming access to accounts or actions the assistant cannot perform.
Keep three separate result categories:
| Category | Question it answers | Suggested checks |
|---|---|---|
| Answer quality | Did the assistant respond appropriately? | Correctness, grounding, completeness, clarification |
| Service reliability | Did the request complete? | Errors, timeouts, missing responses |
| Performance | Was delivery acceptable? | Response time, token use, page impact |
Use separate release gates for each. Do not count a timeout as a correct refusal or let a fast response compensate for a fabricated policy.
2. Build a representative, versioned test set
Include ordinary questions, known failures and boundary cases. OpenAI recommends evaluation data that covers typical, edge and adversarial cases, drawing on sources such as production examples and expert-created answers. (developers.openai.com)
For a website assistant, organize cases into useful categories:
- Answerable: The approved documentation contains the answer.
- Ambiguous: The question lacks information needed to respond.
- Unanswerable: The documentation does not establish the requested fact.
- Restricted: The request falls outside permitted access or actions.
- Conversational: A follow-up depends on earlier messages.
- Conflicting evidence: Available documents disagree or have different effective dates.
For each case, save the question, conversation history, approved evidence, expected behavior and unacceptable behavior. An expected behavior can be a checklist rather than one exact sentence.
Use synthetic account details in testing. If you draw questions from customer conversations, remove unnecessary identifiers and confirm that your evaluation environment is approved for the remaining data.
Keep a development set for prompt iteration and a separate holdout set for the final comparison. Do not repeatedly tune against the holdout and still describe it as independent validation.
Version the questions and reference evidence together. A case becomes misleading if its expected answer reflects a policy that has since changed.
3. Score meaning, evidence and appropriate restraint
For document-based assistants, distinguish grounding from completeness. Microsoft’s RAG evaluation documentation treats groundedness as alignment with supplied context without fabrication, while completeness checks whether the response includes critical expected information. (learn.microsoft.com)
A practical scorecard can use the following checks:
| Dimension | Passing behavior | Failing behavior |
|---|---|---|
| Correctness | Matches the verified reference | Contradicts the reference |
| Grounding | Important claims have supporting evidence | Adds unsupported facts or promises |
| Completeness | Includes conditions needed to act | Omits a material qualification |
| Clarification | Requests necessary missing information | Guesses the user’s situation |
| Appropriate restraint | States limits and offers a permitted next step | Invents access, actions or answers |
| Usability | Gives a clear, relevant response | Buries the answer or wanders off topic |
Treat restraint as context-dependent. For your assistant, an ordinary policy question should receive an answer; a missing account fact should receive an honest limitation. A blanket refusal is not automatically a success.
Check whether cited passages support the associated claims. Also verify the reference material itself: grounding measures consistency with context, so do not use it as a substitute for reviewing whether that context is current and correct. (learn.microsoft.com)
Define critical failures separately from softer issues. In this proposed scorecard, inventing a refund entitlement blocks release; slightly awkward wording does not carry the same weight.
4. Compare two configurations with a fictional support example
The following example is entirely fictional. Its policy, prompts and responses are authored illustrations—not provider claims, observed model outputs or measured test results.
Assume a website’s approved support document says:
Monthly subscriptions can be canceled in the account dashboard. Cancellation stops the next renewal; access continues until the current paid period ends. Refund eligibility requires support review. The chatbot has no account or payment-record access.
Compare two prompts using the same model and evidence:
- Configuration A: “Answer subscription questions using the support document.”
- Configuration B: “Answer from the support document. Preserve material conditions. Ask for clarification when necessary. State when account access or support review is required.”
These prompts are candidates to test, not proof that B performs better.
| Test question | Expected behavior |
|---|---|
| “If I cancel today, do I lose access immediately?” | Explain that access continues through the current paid period and renewal stops |
| “Can you cancel it?” | Clarify what “it” means; if it is the subscription, explain the dashboard route without claiming completion |
| “Was my refund approved?” | State that approval cannot be checked because the chatbot lacks account access |
| “Will I definitely get my money back?” | Explain that eligibility requires support review; make no guarantee |
Consider two illustrative answers to the first question:
Answer X: “No. You can cancel in the dashboard, and your subscription will stop.”
Answer Y: “No. Canceling stops the next renewal, but access continues until your current paid period ends. You can cancel in the account dashboard.”
Under the proposed rubric, X needs review: “will stop” leaves the timing unclear. Y includes both material conditions.
For the refund-status question, “Yes, your refund was approved” would be a critical failure. “I can’t check refund approval because I don’t have account access; support needs to confirm it” would meet the stated boundary.
Run both configurations across every case before choosing one. The exercise establishes what to check; it does not predict which prompt will win.
5. Repeat trials and combine automated checks with human review
One response is not enough to assess consistency. Anthropic distinguishes a test case from a trial and recommends multiple trials because outputs vary between runs. (anthropic.com)
As a practical starting procedure, repeat each case several times under both configurations. Increase coverage when outcomes vary or the consequences of a mistake are material. There is no universal trial count that establishes reliability for every website.
Keep inputs and grading rules comparable. If you change both the model and prompt, describe the result as a comparison of complete configurations—not evidence that either individual change caused the difference.
Use automated checks for rigid requirements, such as required fields or permitted classification labels. Use human review for nuanced answers. Anthropic describes code-based, human and model-based grading, and advises validating model graders before scaling them. (platform.claude.com)
Blind reviewers to configuration names. Randomize answer order and permit ties. OpenAI identifies position and verbosity bias in model-based judging and recommends checking agreement with human labels. (developers.openai.com)
Report:
- Pass counts and denominators by question category.
- Critical failures and their case IDs.
- Cases whose outcomes changed across trials.
- Candidate wins, losses and ties under the fixed rubric.
Do not select only the best attempt. Review disagreements before averaging scores; the rubric or reference may need correction.
6. Record enough detail to reproduce and diagnose failures
Create a release record for each configuration. The following checklist is a proposed operational format:
- Provider and requested model identifier; returned version information if available.
- Exact prompt text and prompt version.
- Generation settings that the selected API supports.
- Application code version.
- Knowledge-base snapshot and retrieval settings.
- Test-set and rubric versions.
- Grader configuration and reviewer decisions.
- Run date, case IDs, trial IDs, responses and request errors.
Do not put API keys in this record. Keep access to test logs controlled, and exclude unnecessary customer data.
For retrieval-based assistants, save the passages actually supplied to the model. This lets you distinguish a missing-document problem from a response-generation problem. Microsoft’s documentation evaluates retrieval quality separately from final response quality. (learn.microsoft.com)
Save observable tool calls and results where relevant; do not require hidden model reasoning.
Common mistakes to avoid include changing the reference answer mid-comparison, testing only easy questions, treating keyword matches as proof of correctness and accepting an improved average despite a new critical failure.
If a provider does not expose a stable model version, record that limitation. Your logs document the comparison conditions; they do not guarantee identical future outputs.
7. Require staging, release gates and a usable rollback
Before running the comparison, write the acceptance rules. For this support assistant, a proposed gate might require:
- No critical fabricated-policy or false-action failures.
- No regression in ambiguous or unanswerable cases.
- A documented benefit in the target category.
- Separate reliability and performance checks passing their own limits.
Choose thresholds for your application’s risk and workload. Passing a finite test set is not a guarantee of production quality.
Validate the complete integration in staging, including retrieval, conversation state and fallback behavior. Then use a limited rollout where the platform supports it. Google describes canary deployment as splitting traffic between existing and new versions before a full rollout. (docs.cloud.google.com)
Specify who reviews sampled answers, when expansion is allowed and what stops the rollout. Keep the prior configuration available and test the switch-back path in staging.
Rollback triggers should include confirmed critical answer failures as well as operational problems. Restore compatible prompt, model, application and retrieval settings—not merely an old prompt attached to changed dependencies.
If the old model is unavailable, document that rollback gap and prepare an approved non-AI fallback. Add newly discovered failures to the regression set and rerun evaluations on later changes, consistent with continuous evaluation guidance. (developers.openai.com)
Read next: How to Benchmark Website Performance Before and After Adding an AI Feature