Customer Support Quality Assurance: How to Build a QA Scorecard

Response times and satisfaction scores tell you how fast a support team works and how customers felt afterward. Neither tells you whether the answers were actually correct, complete, and consistent with policy. That’s the job of quality assurance (QA): regularly reading a sample of real conversations and scoring them against a shared standard.

This guide covers how to set up a support QA process from scratch — what to score, how to build the scorecard, how to sample tickets, and how to turn reviews into coaching.

Why QA Catches What Metrics Miss

A ticket can be answered quickly, closed on the first reply, and rated “satisfied” — and still be wrong. A customer might be happy because an agent granted a refund outside policy, or might not notice that the workaround they were given will break again next month. On the other side, a ticket can get a poor rating because the customer disliked a policy, even though the agent handled it perfectly.

QA separates the agent’s work from the outcome. It answers questions such as:

• Did the agent correctly understand what the customer was asking?
• Was the information accurate and complete?
• Was internal process followed — correct tags, notes, escalations?
• Was the tone appropriate for the situation?

Used well, QA is also one of the most direct sources of insight into where your documentation, training, or product is causing confusion.

What to Include in a QA Scorecard

A scorecard is the list of criteria every reviewed ticket is graded against. Most teams group criteria into four or five areas. A reasonable starting set:

1. Understanding

• Identified the customer’s actual problem, including anything implied but not stated.
• Asked clarifying questions only when genuinely needed — not as a stalling tactic.

2. Accuracy and resolution

• Information given was correct and current.
• The solution addressed the root issue, not just the symptom.
• The customer was told clearly what happens next.

3. Process

• Ticket was tagged, prioritized, and categorized correctly.
• Internal notes were left where another agent would need context.
• Escalation followed the defined path when required (see our guide to ticket escalation workflows).

4. Communication

• Reply was clear, scannable, and free of unnecessary jargon.
• Tone matched the situation — more care for an upset customer, more brevity for a quick question.
• Any canned response used was personalized to the ticket.

5. Critical errors (pass/fail)

Some mistakes matter more than everything else combined: sharing one customer’s data with another, skipping identity verification on a sensitive account change, or promising something the company can’t deliver. Treat these as automatic fails regardless of the other scores, so a well-written reply can’t mask a serious problem.

Choosing a Scoring Scale

Simpler scales produce more consistent results between reviewers. Three common options:

• Yes / No / Not applicable for each criterion. Easiest to apply consistently, and the “N/A” option stops reviewers penalizing agents for steps a ticket didn’t need.
• A three-point scale (doesn’t meet / meets / exceeds) when you want to recognize standout work.
• Weighted scores, where accuracy counts for more than formatting. Useful once the scorecard is stable, but harder to explain to agents.

Whatever you choose, write a one-sentence definition for each criterion with an example of what “meets” looks like. “Tone was appropriate” means different things to different reviewers until it’s defined.

How Many Tickets to Review

There’s no universally correct number. The goal is a sample large enough to spot patterns for each agent without turning QA into a full-time job. Some practical guidelines:

• Review a fixed number per agent per week or month, rather than a percentage of total volume, so agents handling fewer tickets still get meaningful feedback.
• Mix random and targeted samples. Random tickets show typical quality. Targeted tickets — those with low satisfaction ratings, reopened tickets, long resolution times, or escalations — show where things go wrong.
• Review whole conversations, not single replies. A ticket that took five back-and-forths to resolve may contain individually fine messages that collectively missed the point.

A team that’s just starting might review a handful of tickets per agent each week and adjust once they see how long reviews take.

Who Should Do the Reviewing

Small teams often have the support lead review everything. As the team grows, common approaches include:

• A dedicated QA specialist who reviews full-time and keeps scoring consistent.
• Peer review, where agents score each other’s tickets. It spreads knowledge across the team, though it works best with clear criteria and some oversight.
• Self-review, where agents score a few of their own tickets against the scorecard. Useful as a reflection exercise alongside other methods.

Calibrate regularly

If two reviewers score the same ticket differently, the scores stop meaning anything. Hold a short calibration session — say, monthly — where reviewers independently score the same few tickets, compare results, and agree on how criteria should be applied. Update the criterion definitions whenever a disagreement reveals ambiguity.

Turning Reviews Into Coaching

QA that only produces a number tends to make agents defensive. QA that produces specific, actionable feedback makes them better.

• Share the reviewed ticket alongside the score, with comments on specific lines.
• Point out what went well, not only what didn’t. Agents learn as much from seeing which of their habits are worth repeating.
• Look for patterns before acting. One missed tag is a slip; the same missed step across five tickets is a training gap — and if several agents make the same mistake, the problem is likely the process or documentation, not the people.
• Keep QA scores separate from punitive measures, at least at first. The same principle applies to SLA targets: metrics that are used to punish get gamed.

Feeding QA Findings Back Into the Operation

The most valuable output of QA is often not about individual agents at all. Reviews regularly surface:

• Knowledge base gaps — agents explaining the same thing from scratch because no article exists, or linking an article that’s out of date.
• Unclear policies — agents interpreting the same rule in different ways.
• Product issues — recurring confusion that a design or copy change would eliminate.
• Broken macros or automations — templates that consistently lead to follow-up questions.

Logging these alongside the score, and reviewing them monthly with whoever owns documentation and product feedback, turns QA from a grading exercise into an improvement loop. Our guide to how customer support software helps businesses covers how support data more broadly informs product and operations decisions.

FAQ

What is a good QA score in customer service?

It depends entirely on your scorecard and how strictly it’s applied, so scores aren’t comparable between companies. What matters is the trend over time within your team and the consistency between reviewers. Set a baseline in your first month, then track whether it moves.

What’s the difference between QA and CSAT?

CSAT (customer satisfaction) measures how the customer felt about the interaction. QA measures whether the agent did the work correctly against your internal standards. The two often agree, but when they don’t — a happy customer with a wrong answer, or an unhappy customer with a correct one — the disagreement is usually where the most useful insight is.

Do small support teams need a formal QA process?

A team of two or three people may not need a formal scorecard, but even then, periodically reading each other’s tickets against a short shared checklist helps keep answers consistent as the team grows. It’s easier to introduce QA early than to retrofit it once inconsistent habits have set in.

Can QA be automated?

Partly. Some help desks and add-on tools can flag tickets for review automatically — low ratings, reopened tickets, negative sentiment — or score simple criteria like whether a ticket was tagged. Judging accuracy, tone, and whether the customer’s real problem was solved still generally needs a human reviewer.