For decades, contact center QA programs have been designed around human agents.
Did the agent verify the customer's identity? Did they provide accurate information? Did they demonstrate empathy? Did they follow the required process? Did they use appropriate language and successfully resolve the customer's issue?
Organizations built scorecards around those behaviors, reviewed a sample of conversations, and used the results to coach employees and improve performance.
Now there's a new agent entering the contact center.
And it isn't human.
AI agents are beginning to answer questions, resolve issues, navigate workflows, and interact directly with customers. As their responsibilities expand, organizations will have to confront an important question:
Who's quality-checking the AI?
Customers Don't Care Who Made the Mistake
From the customer's perspective, the distinction between a human mistake and an AI mistake isn't particularly meaningful.
If a human agent provides incorrect information about a refund policy, that's a poor customer experience. If an AI agent provides the same incorrect information, the outcome is exactly the same.
The customer received the wrong answer.
The same is true for compliance. If a human employee fails to make a required disclosure, organizations generally recognize that as a QA issue. If an AI system fails to make that disclosure, it shouldn't somehow fall outside the quality program simply because software conducted the interaction.
As AI becomes another customer-facing representative of the organization, quality standards need to follow the conversation, not the employee.
AI Creates Entirely New QA Questions
Many of the traditional measures of a good customer interaction still apply to AI.
Was the information accurate? Was the customer's problem resolved? Was the appropriate process followed? Were required disclosures provided? Was sensitive information handled correctly?
But AI introduces additional questions that traditional agent scorecards were never designed to answer.
Did the AI recognize when it didn't know the answer? Did it escalate appropriately rather than confidently providing incorrect information? Did it remain within approved policies and knowledge sources? Did it clearly identify itself as AI when required? Did it preserve context when transferring the customer to a human?
These aren't theoretical concerns. As governments and regulators develop frameworks for AI, evaluation, transparency, oversight, and accountability are becoming increasingly important components of responsible deployment. The EU AI Act, for example, establishes transparency requirements for certain AI systems interacting directly with people and places broader obligations around monitoring and oversight on covered systems. EU AI Act Service Desk
The more autonomy organizations give AI, the more important it becomes to verify what that AI is actually doing.
One Customer Journey May Involve Multiple Agents
The challenge becomes even more interesting when a conversation isn't exclusively human or AI.
Imagine a customer begins with an AI agent. The AI gathers information, attempts to resolve the issue, and eventually determines that a human needs to intervene. The interaction transfers to a live representative who uses AI-powered guidance while completing the conversation.
Who owns the quality of that interaction?
The answer should probably be: everyone involved.
Organizations need visibility across the entire journey. The AI portion should be evaluated. The handoff should be evaluated. The human interaction should be evaluated. And the organization should understand whether the complete experience ultimately delivered the right outcome.
That requires moving beyond the traditional concept of an "agent scorecard."
The Future of QA Is Conversation QA
A better model may be to evaluate the conversation itself.
Instead of beginning with "How did this agent perform?" organizations can begin with "Was this a good customer interaction?"
That shift sounds subtle, but it fundamentally changes the role of QA.
A conversation-level scorecard might evaluate accuracy, compliance, resolution, customer effort, escalation, empathy where appropriate, and adherence to organizational policies. Some criteria may apply equally to humans and AI, while others will be specific to one or the other.
The important thing is that every interaction is evaluated against a consistent definition of quality.
This also creates a more useful picture of what's actually happening across the contact center. Leaders can compare where humans excel, where AI performs well, which situations consistently require escalation, and where either side needs improvement.
Automated QA Makes This Possible at Scale
Traditional manual QA was already limited by sampling.
When organizations could only review a small percentage of human conversations, adding thousands or millions of AI interactions to the QA workload would have been nearly impossible.
Automated QA changes that equation.
With accurate transcription and conversation intelligence, organizations can evaluate interactions at scale and automatically identify whether important behaviors occurred. That creates the possibility of continuously monitoring both human and AI conversations rather than treating AI performance as a separate technical exercise reviewed by a different team.
At MosaicVoice, we believe this is an important evolution of automated QA. The goal shouldn't simply be to score human agents more efficiently. It should be to give organizations visibility into the quality of the customer conversations happening across their business.
As the definition of "agent" changes, QA needs to change with it.
Humans and AI Can Learn From Each Other
There is another interesting opportunity here.
Once organizations evaluate humans and AI against common outcomes, the QA data itself becomes a learning system.
Perhaps human agents consistently outperform AI when customers are emotional or situations become ambiguous. Maybe AI performs particularly well on certain highly structured workflows. Perhaps the best human agents use language or approaches that could improve automated experiences. Or AI interactions may reveal common customer questions that agents need better training to handle.
Instead of maintaining separate improvement programs for humans and technology, organizations can create a continuous feedback loop between the two.
QA becomes the mechanism that identifies what works, regardless of who did it.
The Goal Isn't to Catch AI Making Mistakes
It's tempting to think of QA for AI primarily as a risk-management exercise.
And risk absolutely matters.
But the same principle that applies to human QA should apply here too: the objective isn't simply to find mistakes. It's to continuously improve the customer experience.
Organizations should use QA to understand where AI succeeds, where it struggles, when it should ask for help, and how it can work more effectively alongside human employees. Those insights can inform AI training, knowledge management, workflow design, agent coaching, and escalation strategies.
The result is a contact center where humans and AI aren't competing for the same work.
They're continuously learning how to divide it better.
The Bottom Line
The contact center is changing faster than the scorecards used to evaluate it.
As AI begins interacting directly with customers, organizations can no longer assume that quality assurance is exclusively about human agent performance. Every customer interaction represents the brand, regardless of whether the voice on the other end belongs to a person or an AI system.
The next generation of QA won't simply ask, "How did the agent do?"
It will ask, "How did the conversation go?"
And increasingly, the answer may depend on how well humans and AI worked together.