How AI scoring works
An AI reviewer grades each AI practice session and certification, and gives a reason for each grade. Use this page to understand your feedback and to find a grade that needs a correction.
What the AI grades
Section titled “What the AI grades”The AI grades only the learner. It compares what the learner said with the content of the step. The content is the only source of facts. A statement that the content does not mention is not an error.
An AI Training or an AI Roleplay has four criteria. Each criterion has a grade from 1 (Needs work) to 4 (Excellent).
| Criterion | What the AI looks for |
|---|---|
| Content Engagement | Quick responses and references to the content. |
| Content Expertise | Accurate, detailed answers with the correct terms. |
| Application (AI Training) | The content applied to real situations or new scenarios. |
| Customer Communication (AI Roleplay) | Discovery questions, and objections handled as the content teaches. |
| Effort and Realism | Full answers, as to a real customer. One-word answers have a low grade. |
The session score is the total of the four grades, out of 16. Each grade has a reason, usually with examples from the transcript.
Score and approval
Section titled “Score and approval”Only an approved session completes a step. Arcade approves an AI Training or an AI Roleplay only when all these conditions are true:
- The session is complete, or it stopped in the final wrap-up.
- The learner took part. A grade of 1 for Effort and Realism stops the approval.
- The total score is 6 or more. Arcade marks a total of 4 or 5 as off-topic.
A session without approval keeps its grades. The grades of a session that ended early, or that the learner did not take part in, do not count in averages.
How the AI grades a certification
Section titled “How the AI grades a certification”A certification has no score out of 16. The AI gives each question Pass or Fail, with a reason. The question sets what the answer must cover. Its grading context, which the learner does not see, sets which facts are correct.
By default, an answer passes when all these conditions are true:
- It answers the main point of the question.
- It covers about 60% of the key ideas.
- It states no essential fact incorrectly.
An answer can pass without a small detail, unless the question asks for that detail. When the evidence is not clear, the answer passes. The certification passes when the number of passed questions is equal to or more than the Pass Criteria.
Why two sessions have different scores
Section titled “Why two sessions have different scores”The AI grades each session alone, without earlier sessions. These items change the result:
- First answers count more. An answer that repeats what the AI explained in its last turn shows engagement, not expertise. A correct first answer, or a point applied to a new scenario, shows expertise.
- Early attempts are safe. In an AI Training, a wrong answer before the coach teaches a point does not lower a grade.
- Only the parts that the session covered count. The parts after an early stop do not lower a grade.
Skill scores
Section titled “Skill scores”After an AI Training, an AI Roleplay, or a coaching session, the AI also rates skills. It rates the skills in the skill focus of the pathway. If there are none, it rates the skills of the person’s role. A skill score goes from 1 to 100. It is separate from the session score.
- A skill that the session gave no chance to show does not change.
- In an AI Roleplay or a coaching session, a skill that the scenario clearly needed but the learner did not use can go down.
- By default, a skill score shows after the AI rates the skill in three sessions. Then one session moves it by 15 points or less.
- An acceptable use of a skill never lowers its score.
- A certification or a lesson does not change skill scores.
What a manager can and cannot change
Section titled “What a manager can and cannot change”A Team Manager or a higher role can change the grades and their reasons, and set a certification or one question to Pass or Fail. A manager can also approve a session with the Status switch.
A manager cannot edit their own assessment. A change to a grade or to a Pass or Fail result does not change the approval or the skill scores. A new AI grade never replaces a change from a manager.
Good to know
Section titled “Good to know”- An AI session that is shorter than 20 seconds, or that has fewer than 20 words from the learner, has no score.
- The AI writes its reasons and the session summary in English, also for a session in a different language.
- A change to the result of one question does not change the result of the certification.