Love It or Hate It: The NPS Approach Works

Brant Cruz, Chadwick Martin Bailey, Inc.

Customer loyalty and your bottom line: Why NPS works.

NPS, or Net Promoter® Score, has generated quite a bit of controversy since its introduction in 2003. And it’s easy to see why. Proponents point to measurement of a single question, the likelihood that a customer will recommend a firm’s product or service, as the key to growing “good profits” and the bottom line. Skeptics dispute this claim, citing a lack of proof; saying that its relative simplicity disguises the more complex issues behind loyalty.

Admittedly, I was one of those skeptics when we began implementing a similar program for all of the reasons others have cited (plus I’ve never met a form of regression I didn’t like). One year later, I’m a strong advocate (a Promoter if you will) of the concepts behind NPS. And I think that this year’s journey has been an eye-opener for me and has added to an already-strong foundation regarding what works, and what doesn’t, when it comes to measuring customer loyalty.

Here are a few of the keys to success around measuring customer loyalty that are hallmarks of Reichheld’s NPS philosophy:

Listen to Your Customer — What THEY Want to Say
No questionnaire, no matter how long and arduous for the respondent, can cover everything. So let your customers tell you what matters, in their own words, then group them into common themes and read every word they say. And when masses of customer comments are read – really read – there is a far better chance of finding inspiration for a solution than when the data provides only higher “attribute” level detail. We’ve learned firsthand about how important this is recently with one of our clients: verbatims picked up an important pain point (and improvement opportunities) related to a business change that would have been completely missed in a more traditional tracking instrument.

Ask Fewer Questions of More People
Keep it short so you can hear from a big sample and broad cross-section of your customers (and prospects). It’s no secret that industry-wide response rates are problematic for a variety of reasons. One thing we can control is the length of our questionnaires, and we know that as interview length goes down, the percent of people who complete goes up. No sampling methodology is perfect and it is still impossible to talk to everyone. But given how hard it is to get someone to pick up the phone or click on a link, don’t we want to ensure that nearly all of them tell us what we need to know?

Link Attitudes to Behavior — Your Transaction Data
Forget asking a ton of behavioral questions – it’s easier than ever for many companies to link individual customer behavior via unique account numbers (e.g., financial accounts, loyalty program membership, unique customer IDs on every eCommerce site) to data collected through a research study. Linking attitudes with behaviors lets you see if how they feel synchs with how they behave – or if how they feel impacts how they behave! Even better, you can create predictive models to find others who “look like” your best Promoters via the data warehouse, and reach out to them to reinforce their (likely) behavior.

So, those are three pretty solid reasons to like NPS. In my opinion, research about them trumps our inborn, researcher need to analyze everything. Flexibility trumps vigilantly tracking exactly the same thing (often, everything) every month. Real behavior trumps self-reported behavior. And meaningfully better response rates trumps just about everything else.

Is the NPS measure materially better than any other single (or multi) measure at predicting behavior than anything else our industry has tried? I have seen several other researchers try to prove that NPS is not a better predictor of behavior than other traditional outcome measures – but all I’ve really seen is that all of this post-survey behavioral analysis is conclusively inconclusive. Given this, I’ll take my chances with a measure that is simultaneously all about the customer and focused on driving and managing an action that we can all agree is good for long-term business success.

In short, I buy Reichheld’s “good profits” argument. But you need more proof, don’t you?
I’ve also seen firsthand the quality of the verbatim responses the advocacy measure elicits. We’ve tested the traditional NPS measure against “likelihood to buy” in a concurrent study and have been able to read and compare more than 12,000 English language verbatims for each. Subjectively, the verbatims based on advocacy are in general more passionate and give better feedback about what is great and what needs work with my client’s brand. More objectively, nearly twice as many advocacy-based verbatims have been coded as “very insightful” (one variable in our rigorous coding scheme is about the “quality” of the verbatim and we have set the bar quite high) – and this translates into nearly 5,000 additional “very insightful” customer comments for our clients executives to read annually.

Despite this ringing endorsement, I do have two suggestions around how to improve upon the original NPS framework that Reichheld proposes and practices:

1. Buyer Beware! Invest in due diligence on your company’s “ultimate question.”Despite my strong belief in the power of advocacy, (heck, we’ve been involved in programs to help foster it for 15 years), I strongly advise conducting piloting the NPS metric versus one or more “challengers” and analyzing post survey behavior over several months. I have seen one instance where the traditional question didn’t work (it was negatively correlated with post-survey financial volume) for a certain segment for a client of mine. And we’ve had to test several versions of the question to find the right one. My philosophy is this: in the absence of compelling empirical evidence of a better question, focusing on likelihood to recommend is the best single question to use to measure and manage loyalty. But given the investment required in these large scale trackers – you owe it to your company to make sure the question works in your setting.
2. Voluntary Amnesia? Don’t forget everything you’ve already learned! Unlike Reichheld, I recommend that you continue tracking a short (no more than 10) set of previously-identified key drivers in addition to your ultimate question. For most companies, an NPS-like methodology won’t be the first brand or performance tracking research you have ever conducted. You’ve been focusing on some outcome(s) and conducting some form of key driver analysis to determine how to move that/those needle(s). Why stop measuring a short set of attributes that you know matter if you can still keep interview length really short and therefore not materially reduce cooperation rates? While verbatims tell the best and most complete story, by tracking a short set of attribute-level measures you can diagnose some of the reasons for NPS shifts quickly and consistently.

- June 2008

This content was provided by Chadwick Martin Bailey, Inc. Visit their website at www.cmbinfo.com.

[Feb 25, 2010]