Can AI Assess Therapy Skills? Why Clear Rubrics Matter

“You were very empathic: 9/10.” It is an easy comment to understand and a pleasant one to receive. But how useful is it for learning?

Without explicit criteria, dialogue evidence and an opportunity to question the result, an automated score can create an illusion of precision.

From impressions to observable behaviour

A rubric describes the criteria used to judge a performance. Rather than saying “good empathy,” for example, an assessor might consider whether a response:

• Acknowledges emotional cues in what the client has said.
• Avoids premature advice.
• Leaves room for the client to confirm or correct the interpretation.

These criteria are more open to scrutiny than an overall impression. They still do not capture the entire therapeutic relationship.

What can be assessed more readily?

A written transcript may allow relatively observable behaviours to be reviewed: open versus closed questions, repeated advice-giving, collaborative goal language or adherence to an agreed structure.

Tone of voice, nonverbal communication, cultural context and the meaning of a pause are harder to judge. A well-phrased sentence in isolation may have been poorly timed during the session.

What does the research suggest?

A 2026 study evaluated a multiagent AI system that scored synthetic motivational interviewing and CBT encounters against explicit criteria. The framework distinguished prompt-defined competence levels in its artificial scenarios and showed preliminary alignment with human-rater benchmarks.

Importantly, the researchers called for prospective testing with human trainees, standardized or real patients and formal psychometric assessment. The work should not be taken as evidence that an AI tool can certify clinical competence.

What does actionable feedback look like?

Instead of “Low score in listening,” imagine receiving this comment:

“When the character said, ‘I'm afraid I will fail,’ you immediately asked about the timeline. One alternative would have been to acknowledge that fear before moving into practical details. Try the exchange again with that priority.”

The feedback shows where, why and what to practise next. It is also open to review and discussion with a supervisor.

Three limitations to make visible

Automated scoring may be inconsistent, may prefer particular ways of speaking and may disagree with experienced supervisors. Moreover, feedback for private learning is not equivalent to high-stakes assessment affecting academic progression or professional entry.

The strongest use of an AI score is therefore to open an evidence-based learning conversation, not to settle it.

From theory to practice: In TheraMentor, reflection is most meaningful when it relates to concrete choices within the conversation.