Source: [AI-drafted interview guides: rehearse the rating criteria first — Skills for Business]() [← All articles](https://skillsforbusiness.ai/blog/) HR & people · Guide # AI-drafted interview guides: rehearse the rating criteria first Before using an AI-drafted interview guide, check whether its questions elicit relevant evidence and whether interviewers can apply the rating criteria consistently. [Skills for Business](https://skillsforbusiness.ai/blog/editorial/)2 November 20267 min read On this page 1. [Interview structure includes how answers are evaluated](https://skillsforbusiness.ai/blog/ai-interview-guides-scoring-rehearsal/#interview-structure-includes-how-answers-are-evaluated) 2. [Give the assistant a bounded drafting brief](https://skillsforbusiness.ai/blog/ai-interview-guides-scoring-rehearsal/#give-the-assistant-a-bounded-drafting-brief) 3. [Replace vague rating labels with observable distinctions](https://skillsforbusiness.ai/blog/ai-interview-guides-scoring-rehearsal/#replace-vague-rating-labels-with-observable-distinctions) 4. [Rehearse with responses that challenge the guide](https://skillsforbusiness.ai/blog/ai-interview-guides-scoring-rehearsal/#rehearse-with-responses-that-challenge-the-guide) 5. [Keep follow-up questions inside the intended assessment](https://skillsforbusiness.ai/blog/ai-interview-guides-scoring-rehearsal/#keep-follow-up-questions-inside-the-intended-assessment) 6. [Use rehearsal as a diagnostic, not a validation claim](https://skillsforbusiness.ai/blog/ai-interview-guides-scoring-rehearsal/#use-rehearsal-as-a-diagnostic-not-a-validation-claim) 7. [Sources](https://skillsforbusiness.ai/blog/ai-interview-guides-scoring-rehearsal/#sources) What to take away - Draft interview questions and rating criteria together from an approved account of the work. - Rehearse with fictional responses and compare independent ratings before interviewing candidates. - Agreement in a rehearsal can reveal clearer instructions; it does not validate a selection method or an AI scoring system. Before using an AI-drafted interview guide, rehearse the questions and rating criteria with the people who will conduct the interview. Check whether a question draws out the evidence the job requires, and whether interviewers can explain their ratings using that evidence. A polished question and a numbered scale are a draft, not proof of a sound assessment. This guide proposes a preparation exercise for hiring managers and HR teams. The example responses below are fictional materials for testing the guide. They are not candidate assessments, observed model outputs or evidence that an AI product can select suitable employees. The starting point is an approved description of the work. Our guide to [justifying AI-drafted job requirements](https://skillsforbusiness.ai/blog/ai-job-descriptions-justify-requirements/) covers that earlier decision. Here the question is how to turn an agreed capability into interview materials that people can actually use. ## Interview structure includes how answers are evaluated The U.S. Office of Personnel Management’s [Structured Interviews: A Practical Guide](https://www.opm.gov/policy-data-oversight/assessment-and-selection/structured-interviews/guide.pdf), dated September 2008, describes developing questions alongside rating scales and behavioral examples, piloting the interview and preparing interviewers. It also describes independent panel ratings before discussion. This is established U.S. federal assessment guidance, not a test of generative AI or a universal legal procedure. [CIPD’s selection-methods factsheet](https://www.cipd.org/en/knowledge/factsheets/selection-factsheet/), dated 22 September 2026, similarly recommends pre-agreed questions and criteria, preparation for consistent interviewing, and independent scores before group discussion. These recommendations concern the selection process; they do not establish that an assistant’s proposed criteria are job-relevant or that its scores are accurate. The [2014 review by Levashina and colleagues](https://onlinelibrary.wiley.com/doi/10.1111/peps.12052) treats rating scales, follow-up questions and reactions to structure as distinct research topics. Only its abstract was accessible for this article; we do not use it for effect estimates or detailed findings. It predates current generative-AI tools and cannot validate an AI-drafted guide. Our editorial recommendation is to inspect the whole question-and-rating package. Giving everyone the same wording does not settle what counts as good evidence or how an interviewer should handle an incomplete answer. ## Give the assistant a bounded drafting brief Suppose a fictional team is hiring a service coordinator. One approved capability is preparing a useful handover when an issue remains unresolved. The person should identify what is known, what still needs checking, who owns the next action and when the matter needs attention. The role does not authorize them to promise an unconfirmed delivery date. Give the assistant that account, the intended interview format and the boundaries of the role. Ask it to draft a question, a description of the evidence sought, possible follow-up prompts and provisional rating criteria. Ask it to mark assumptions rather than invent company policy, escalation deadlines or authority levels. A proposed situational question might be: “An order remains unresolved at the end of your shift. The delivery date is unconfirmed, and another team needs to investigate. What would you put in the handover, and what would you do next?” This is an illustrative draft for review, not a validated interview item. The hiring manager should check whether the situation resembles the actual job and supplies enough information to answer. If a strong answer depends on a company-specific procedure that candidates have not been given, decide whether the question should provide it or assess something else. Do not silently reward knowledge that the organization intends to teach after hiring. ## Replace vague rating labels with observable distinctions A generated scale might use labels such as “poor,” “competent” and “excellent.” Those labels leave the interviewer to supply the meaning. Adding adjectives such as “confident” or “professional” may introduce another judgement without clarifying the work evidence. For the fictional handover question, the team could instead inspect these provisional distinctions: | Draft distinction | Evidence the guide should help the interviewer identify | Question for the guide’s reviewer | | --- | --- | --- | | Missing operational detail | The answer leaves the unresolved issue or next action unclear. | Is the missing information actually required by the role brief? | | Usable handover | The answer distinguishes known facts from uncertainty and identifies an owner and next action. | Could different wording demonstrate the same capability? | | Additional relevant judgement | The answer also explains how urgency or a failed handoff would be handled within the role’s authority. | Is this required on entry, or an optional strength? | These are fictional design notes, not score bands or a hiring cutoff. A responsible assessment owner still needs to determine the appropriate method, rating scale and use of the results. Separate evidence from inference. “Names an owner” is something an answer can contain. “Will always be reliable under pressure” is a much broader conclusion. The guide should not turn a limited response into an unsupported personality judgement. ## Rehearse with responses that challenge the guide Create a small set of fictional responses for preparation. Include different ways of expressing the same work evidence, an incomplete response, an answer that oversteps authority and one that exposes an ambiguity in the question. An assistant can propose examples, but the role owner should check their plausibility and relevance. For the illustrative handover question, consider two brief response outlines. One uses polished language about taking ownership and keeping stakeholders informed but omits the unresolved fact and the next owner. Another uses plain language to identify the unknown delivery date, the investigating team and the next check. Neither outline is a complete candidate response. Their purpose is to ask whether the guide directs attention to the intended evidence rather than presentation alone. Give the preparation materials to interviewers without an AI-supplied “correct score.” Have them independently note the evidence they found, the provisional rating they would give and the part of the guide they used. Then compare reasons, rather than only the final numbers. If ratings differ, identify the source of the disagreement. Did interviewers interpret the question differently? Did one assume a deadline that the scenario never supplied? Did the criterion mix two capabilities? Did the fictional response genuinely lack enough information? Each diagnosis calls for a different revision. Avoid resolving every difference by adding more text. A shorter question, a separated criterion or a clearer statement of permitted follow-up may be more useful than a larger rubric. After a material revision, try fresh example responses so the group is checking the revised instructions rather than remembering the discussion. ## Keep follow-up questions inside the intended assessment Decide in advance how interviewers should seek clarification. For example, a neutral prompt might ask what the person means by “follow up.” A leading prompt might supply the missing owner, deadline and escalation path, then credit the answer for repeating them. The distinction matters in the preparation exercise: are interviewers eliciting evidence, or giving different people different amounts of help? Ask the assessment owner to define the permitted approach. Consistent assessment also needs to work with the organization’s arrangements for access needs; identical delivery is not a reason to ignore those arrangements. Keep candidates’ questions about the employer distinguishable from assessed responses. An interview also gives applicants information about the job. The guide should help interviewers explain when they are assessing a response and when they are answering a candidate’s question, without treating ordinary requests for clarification as evidence of inability. ## Use rehearsal as a diagnostic, not a validation claim Agreement among interviewers on a few fictional answers can reveal clearer instructions. It does not show that the interview predicts job performance, that it is free from bias or that it supports a particular hiring decision. Interviewers can agree while sharing the same mistaken assumption. Nor does asking the assistant to score its own fictional answers provide independent validation. That exercise may expose inconsistencies in the draft, but it remains generated material checked against generated expectations. Keep the accountable assessment owner responsible for the selection design and its intended use. A small business can begin with the hiring manager and another person familiar with the work. A larger organization can use its existing assessment specialists and interviewer training process. In either case, retain the approved guide version, the reason for substantive revisions and the unresolved questions. Avoid improvising new rating criteria after hearing actual candidates; refer necessary changes through the established selection process. For the next AI-drafted guide, choose one consequential question and rehearse it before expanding the exercise. The useful output is a clearer link between the job, the question and the evidence interviewers will consider—not a claim that the assistant has validated the interview. ## Sources & further reading Sources checked for this article. Research findings and vendor claims are distinguished in the text. 1. [Structured Interviews: A Practical Guide](https://www.opm.gov/policy-data-oversight/assessment-and-selection/structured-interviews/guide.pdf)U.S. Office of Personnel Management · Accessed 2 November 2026 2. [Selection methods](https://www.cipd.org/en/knowledge/factsheets/selection-factsheet/)CIPD · Accessed 2 November 2026 3. [The Structured Employment Interview: Narrative and Quantitative Review of the Research Literature](https://onlinelibrary.wiley.com/doi/10.1111/peps.12052)Levashina et al.; Personnel Psychology · Accessed 2 November 2026 Produced with AI-assisted research and writing. Examples and recommendations are our editorial synthesis unless attributed. This article does not claim independent product testing or human expert review. [Read our editorial policy.](https://skillsforbusiness.ai/blog/editorial/) ## Keep reading [AI pilot recruitment: who do your results represent?](https://skillsforbusiness.ai/blog/ai-pilot-recruitment-results-represent/)[AI skills inventories: separate inference from demonstrated ability](https://skillsforbusiness.ai/blog/ai-skills-inventories-inference-demonstration/)