An AI Coaching Buyer Guide for What the Demo Cannot Prove

A good AI coaching demo is often the least useful evidence. It can show fluency, speed, a pleasant tone, and a few neat interaction patterns. It cannot show what happens when a user becomes distressed, manipulates the system, shares sensitive information, asks for advice that crosses a coaching boundary, or brings in a culturally complex workplace conflict. The demo shows the sales version of the conversation. Enterprise buyers need the pressure version.

In the Speexx Exchange Podcast, I talked to Anke Paulick, President of the ICF Germany Chapter, about AI Coaching Governance and the International Coaching Federation’s perspective on quality. The live setting was useful because I could ask the questions buyers ask when the product looks good but the responsibility still feels unclear. Anke’s answer was consistent: AI coaching quality cannot be assessed from the interface alone.

That sentence deserves to sit at the center of any AI coaching buyer guide. HR and L&D leaders want scale, democratization of coaching, more cases for the same budget, personalization, continuity, and stronger evidence. Those are legitimate expectations. They still need a buying process that tests the full system, not the charm of the first conversation.

12 Questions for enterprise buyers to ask about AI Coaching

Here is Anke Paulick’s ICF buyer’s guide for AI Coaching:

1. What exactly are we trying to scale?
2. Is it clear to the user that they are interacting with AI?
3. Are the system limits explained before use?
4. What coaching methodology underpins the system?
5. Is there evidence from realistic coaching scenarios, not only a product demo?
6. Is there a documented crisis prevention and escalation approach?
7. Who sees the data?
8. Can managers or HR access individual coaching conversations?
9. Has the system been tested for bias?
10. Are the quality claims externally tested?
11. Who is accountable if the AI causes harm?
12. How does the provider improve the system after launch?

The first buying question is what you are trying to scale

Anke’s buyer checklist started with a deceptively practical question: what exactly are we trying to scale? Some organizations want wider access to reflection. Others want better preparation for difficult conversations, more leadership practice, support between human coaching sessions, faster onboarding for new managers, or coaching-like development for employees who rarely receive live coaching. A few mainly want reporting. The answer changes the risk profile.

Scaling access is different from scaling behavioral change. Scaling role play is different from scaling emotional support. Scaling coaching capacity is different from scaling management data. Buyers who skip this distinction end up with a vague business case: more coaching, more users, more activity. Activity does not prove development. A coaching system has to be evaluated against the developmental job it is being asked to do.

The research Anke referred to from Nicky H.D. Terblanche helps keep the access question grounded. The 2022 comparison of artificial intelligence and human coaching goal attainment efficacy found that AI coaching was effective in a narrow goal-attainment application and could help democratize coaching, while still recognizing human coaches as irreplaceable where empathy and emotional intelligence are central. That is the right level of claim. It supports scale without pretending that scale erases professional limits. The paper is available in PLOS ONE, with related background through Coach Vici.

A buyer guide therefore starts before the vendor meeting. The L&D team needs to define whether the desired outcome is access, reflection, behavior change, practice, coaching capacity, or a better way to connect development with business communication challenges. When the goal is precise, the evaluation becomes more useful. When the goal is vague, the vendor’s narrative fills the gap.

Transparency is not a user experience detail

One of Anke’s clearest criteria was transparency. Users need to know they are interacting with AI before the conversation starts. They also need to understand the system limits in plain language. This is not a legal footnote. It is part of the coaching agreement.

The ICF AI Coaching Framework and Standards treats AI coaching as a system, including documentation, functionality, processes, management, and security. The framework also asks providers to explain system functionality, data privacy, compliance, evidence base, methods, and the governance that surrounds the coaching interaction. A buyer who asks only about features will miss much of the quality system.

Transparency matters because coaching involves trust. If users discover later that the system has hidden data flows, unclear escalation rules, or ambiguous human access to their content, adoption will suffer. Worse, people may continue using the tool with the wrong assumptions. In L&D, trust is not a soft value. It determines whether employees bring real problems or stay at the level of safe, generic practice.

The EU context makes this more concrete. The EU AI Act centers trustworthy AI, risk classification, transparency, and human oversight. The European Commission’s high-risk AI guidelines clarify classification principles for providers and deployers. Even where AI coaching does not make employment decisions, the workplace context requires care because users may disclose sensitive professional and personal information.

Methodology needs evidence, not theatre

AI coaching vendors often describe their systems as evidence-based, aligned with coaching models, or grounded in professional practice. A buyer needs to ask what that means in product behaviour. Which coaching methodology underpins the interaction? How does the system keep a non-directive stance? How does it avoid advice-giving when the user asks for a shortcut? How does it support accountability without becoming performance management? Which humans review the coaching logic?

The ICF perspective is useful because it connects coaching craft to testable product questions. An AI coach is not a coach because it has a warm tone. Good coaching depends on contracting, active listening, powerful questioning, values, client agency, and accountability. In a product, these qualities appear in question design, boundary handling, summary behavior, escalation rules, feedback loops, and testing evidence.

A strong provider will be comfortable explaining the methods. A weak provider will hide behind language like “our model is trained on best practices” without showing how those practices appear in the user experience. Buyers need to press for evidence: coaching advisory input, documented methodology, diverse test cases, bias evaluation, crisis protocols, user-facing limits, and a process for improvement.

Josh Bersin’s 2026 research on corporate learning gives this buyer discipline more urgency. His organization reported that 74% of companies say they are not keeping up with demand for new skills, despite a $400 billion corporate training market. That pressure helps explain why buyers look to AI. It also warns against buying AI as a relief mechanism before defining what quality looks like. The research is summarized by Josh Bersin.

Pressure testing separates coaching systems from chat interfaces

The most useful buyer evidence comes from realistic scenarios. In our podcast conversation, Anke described the difference between reviewing documentation and testing real coaching interactions. The point was not to embarrass a product. It was to see where coaching quality becomes visible.

A buyer evaluation needs cases that include feedback conversations, conflict resolution, leadership pressure, cultural sensitivity, manipulation attempts, commercial steering, emotional distress, and crisis escalation. A clean demo rarely tests these conditions. A serious assessment does. The goal is to understand whether the coaching stance holds under pressure or whether the system drifts into advice, reassurance, therapy, compliance language, or irrelevant output.

This is where personalization also needs scrutiny. Personalization can make AI coaching useful because the system adapts to context. It can also create false confidence if the AI sounds more certain than it is. Buyers need to know whether personalization stays inside the coaching purpose. The system can reflect the user’s goals and context. It cannot become a hidden manager, therapist, adviser, or evaluator.

Data protection belongs in the same pressure test. Individual coaching conversations must not become manager dashboards. Aggregated usage and program insight can help L&D improve the service, but personal content needs a clear boundary. This is especially important in global enterprises where works councils, local privacy rules, and cultural expectations around speaking openly can differ widely.

A practical buyer checklist for AI coaching

The buyer checklist from the conversation can be turned into selection criteria without turning the article into procurement paperwork. The first criterion is clarity of purpose: the organization needs to know whether it is scaling access, reflection, behavior change, practice, or coaching capacity. The second is transparency: users need to know they are speaking with AI and where the system limits sit. The third is safeguarding: distress detection and escalation need to be documented and tested.

Methodology comes next. A provider needs an evidence base, a coaching design logic, and credible human expertise behind the system. Data privacy is equally central. Managers and HR should not have access to individual coaching conversations. Quality credentials need scrutiny because “ICF-aligned” is only meaningful when a provider can explain who assessed what, when, and how. Bias testing needs evidence across cultures, languages, demographics, and real users. Accountability must have a name or body attached to it.

This is the point where standards and certifications become relevant for enterprise buyers. Governance is easier to trust when a provider can show a broader culture of external standards, security, accessibility, and compliance. The Speexx awards record is useful market context, while the decision still rests on whether the AI coaching system itself has been tested against professional criteria.

Speexx AI Coaching and Speexx Coaching™ sit in this buying logic as part of a wider development environment. Speexx supports 1:1, group, team, and AI coaching, with AI used for reflection, practice, and continuity while human coaching remains central for depth and complexity. That mix matters because buyers do not need one format to solve every development problem. They need the right format for the right context.

The buyer’s final test is accountability

Anke’s most practical question for buyers was simple: who is accountable if the AI causes harm? Many vendors can answer technical questions. Fewer can answer accountability questions clearly. In enterprise AI coaching, accountability cannot sit vaguely with the model, the user, the manager, or the product team. It needs defined ownership, escalation paths, governance review, and evidence that the provider learns from incidents, edge cases, and audit findings.

This is where the buyer conversation becomes mature. A strong AI coaching provider will welcome scrutiny because scrutiny improves the system. It will publish enough about methodology, security, governance, and human oversight for buyers to make an informed decision. It will avoid exaggerated promises. It will show how it handles failures, not just how it performs when everything goes well.

The demo still has a place. It helps buyers understand the user experience. It can show adoption potential, accessibility, and conversational flow. It just cannot carry the buying decision. AI coaching at enterprise scale is a system-level decision about quality, trust, access, cost, data, and human development.

The practical buyer standard is therefore clear. Buy the system that behaves responsibly when the conversation becomes difficult, not the one that sounds most impressive when the conversation is easy.