Once a week we would sit in a conference room and listen to recorded calls — calibration we called it. If you like Russian roulette it was quite the thrilling ride. We listened to some calls so painful I would feel the need to reach out to the customer myself to apologize. Other times it would be a gem — we would all high five and want to kiss the representative who delivered that experience.
One spring afternoon in Oklahoma we were in luck. We listened to a call and heard this representative orchestrate a spectacular call — and boy was I proud! I was ready to anoint this representative the next superstar, but then we brought out our quality assurance (QA) forms and started scoring. A few marks knocked off here or there and before I knew it we had labeled this a “bad” call. Initially, I was defensive; I tried to argue for this call. Truth is, however, even I had to agree with the room — according to the standards we had laid out the rep did not comply. This was the moment I realized QA was broken and it needed to be fixed.
So we went out and tweaked the guidelines for QA to make sure they weren’t so prescriptive — and that they allowed for “good” calls to happen outside the guidelines. Things got a little better, but too often we listened to terrible calls (from my perspective) that received good marks because we wanted to be objective. By the letter of the law, the representative did all the things that were asked of him/her, even though we all knew the call stunk.
After some tweaks, things began to look a little better but it wasn’t nearly enough. QA was like a patient who needed open-heart surgery but we kept giving it aspirin.
Let’s go back to the mission of QA. From the start, QA within contact centers has had two distinct missions. Mission one is compliance — making sure the calls follow your internal or legally mandated guidelines. The second mission involves measuring the customer experience impact — did the rep make the customer feel good about the interaction.
In regard to ensuring compliance, it is dubious at best as to whether QA is the best way to accomplish that goal in the 21st century. On the second mission, it has in my mind always failed woefully. However, let’s start with the compliance charge. Yes, it sounds good in theory that if we listened to enough calls we would get a sense for how compliant our reps are with company policy, but this fails to pass the smell test for a couple reasons:
1. Sample size: Assuming your reps handle 50 interactions per day, and you are listening to five of those interactions per month, is that a big enough sample size? If you could even swing that it only accounts for 0.25%. I am not sure what conclusions you can draw on a topic as important as compliance based on that small of a sample size.
2. Accuracy: Any exercise that involves human judgment is rife with subjectivity. The very reason “calibration sessions” exist is because as you might try to take subjectivity out of things, when human beings get involved they bring their biases.
3. Costs: Even if you wanted to scale it so your sample size for compliance was better, it gets more and more expensive. Once you begin to put some serious thought beyond the hard costs, the models start to fall apart.
So if QA isn’t exactly lighting the world on fire in terms of compliance, then what about its impact on the customer experience. QA is a terrible gauge of measuring the customer experience for the following reasons:
1. There is no correlation between QA scores and customer loyalty (as defined by repeat business or increased spending). I have vetted this in more than a dozen organizations and the relationship is very weak at best and non-existent in most cases.
2. It drives the wrong behavior. Not only is the correlation weak, but also the mere presence of a checklist QA process creates bad outcomes. Reps will do all they can to score well and ensure they get rewards or avoid punishments, and will do so even if it creates a disjointed experience.
3. There is already a better way to measure the customer sentiment — in fact there are several different ways that don’t involve QA. Hint: They all involve asking customers directly how their interaction went.
If QA fails at both compliance and improving the customer experience, why does it go on? Simply put, many organizations have not embraced alternatives that deliver both in terms of effectiveness and efficiency.
For compliance, analytics is the answer. Voice and text analytics allow you to check every interaction for compliance and dramatically reduce the error rate. On the customer experience front, making sure you have post-interaction surveys that are guaranteed to get responses from customers is the way to go.
— Amas Tenumah is the author of The Curated Experience: Engineering Customer Service to Build Loyalty. He is also the founder of BetterXperience, a firm that specializes in Customer Experience strategy and operations. He has spent more than a decade managing Customer Experience programs for numerous companies including DirecTV, Teleflora, Coca-Cola, and Convergys Corporation. Tenumah brings a unique perspective to the world of Customer Experience that he shares regularly on his blog at http://amastenumah.com and as a speaker at many events.
