Understanding the Dynamics 365 Quality Evaluation Agent: A Complete Breakdown

There is another member of the Dynamics 365 Customer Service/Contact Center agents that I haven’t had a chance to introduce to you yet. You’re probably already familiar with the Case Management Agent, Intent Agent and Customer Knowledge Agent which I have written articles about previously. In this article I am going to discuss the Quality Evaluation Agent. In customer service organizations it’s important measure the quality of service provided to customers because this can significantly impact customer retention, revenue and brand reputation. Higher service quality typically leads to better customer satisfaction, stronger loyalty, and increased lifetime value. Poor service can quietly drive customers away without obvious warning signs. Organizations that are not measuring service quality have a harder time improving because they don’t know where improvement is possible. They might think their processes are efficient, but customers could be frustrated, confused, or worse. Quality measurement helps organizations catch those issues early, improve customer service rep performance, and ensure consistency across the team. Many organizations have quality assurance (QA) programs where Supervisors review customer interactions like calls, chats, and emails against a standard scorecard. Grading usually includes things like empathy, accuracy, adherence to process, and communication clarity. Historically this was a manual job, usually done by supervisors but we can now use the Quality Evaluation Agent (aka Quality Assurance Agent) to help with this.

How does the Quality Evaluation Agent work

The Quality Evaluation Agent is built around a framework with three core pieces working together. One of the most important parts is the evaluation criteria. This is where you define what ‘good service’ actually looks like for your organization. Here you configure specific questions, possible answers to those questions, scoring rules and guidance for how those should be interpreted. Think of these as a benchmark the agent uses to evaluate the service that was provided during conversations and cases. This is where you define what quality means to your organization, whether that’s empathy, accuracy, compliance, or all of the above.

Evaluation Criteria

There are two different pieces you need to consider when defining the criteria the agent should use. There is the evaluation criteria(parent) and there is the ability to extend (child) existing evaluation criteria. Let me explain what the difference is here. Many organizations have strict company-wide compliance standards that every team must follow. For example, a global bank may have the need for all customer interactions to be reviewed for things like fraud prevention, customer verification, and data privacy handling while insurance companies may enforce compliance around policy disclosures, claims handling procedures. These are standard questions that always need to be asked during evaluations. Using the evaluation criteria, organizations can create their own standard set of questions and instructions that become the foundation for all the evaluations across the business.

At the same time, different lines of business may have their own unique processes. By adding extended criteria, teams can build on top of the baseline criteria mentioned above. The extended criteria is where they add their own questions and instructions while still inheriting the original questions from the (parent) baseline evaluation criteria. This gives organizations the flexibility to tailor evaluations for different teams or departments without losing consistency across the business. The inherited questions from the baseline criteria can’t be modified within the extended criteria. However when the baseline (parent) criteria is updated, all extended criteria associated to the baseline (parent) criteria will be updated as well. This will help keep evaluation standards consistent across the organization, while still letting individual teams make their own adjustments.

Evaluation Plans

Next, we have evaluation plans. These are all about which interactions (cases, conversations) are being evaluated, when this is happening and which evaluation criteria or extended criteria should be used. Evaluation plans are used to run evaluations in bulk. We can target specific customer service scenarios, for example high-priority cases or certain channels and automate those reviews. Please note that you don’t have to use evaluation plans, as users can also run evaluations on demand.

Evaluations

And finally, the evaluation results, which are captured in the evaluations. This is where everything comes together. When an evaluation is completed, we can we can review the results here. This shows all the questions (from the evaluation criteria and/or the extended criteria), answers to those questions (from the quality evaluation agent), the score it assigned to each question/answer (if scoring is enabled) and details on the reason why the answer was selected. There is also a summary and recommended actions section. So instead of just identifying that something went wrong, we can actually see where the gaps are and what to do next. That’s how the agent improves opportunities for coaching and better customer experiences. Here is an example of a question from the evaluation criteria and how the QEA could answer that question: Question: ‘Did the CSR communicate effectively?’. Possible answers to that question could be:

• Confusing and/or lengthy
• Didn’t follow requirements when asking for customer feedback
• Didn’t paraphrase to confirm understanding of issues
• Incorrect spelling or grammar
• On track

If the CSR only asked follow-up questions without rephrasing the customer’s concern the agent would select the ‘Didn’t paraphrase to confirm their understanding of the issue’ answer and might only give a score of 1 or less for that question/answer. I hope that after reading this article you now understand how all the components of the agent work together. I hope you enjoyed this article! In my next article I will discuss how to enable and configure the agent so be sure to check in again soon or subscribe here to never miss another post!