User Experience Metrics and the Net Promoter Score (NPS) | September 21, 2007

I had the pleasure if being interviewed on the .Net podcast yesterday, on the subject of user experience design. During the discussion, Paul Boag asked how it was possible to measure the affects of good user experience design. I mentioned that Clearleft always try to encourage our clients to outline their goals and define their success criteria. This could be anything from increasing conversion or retention rates, through to reducing customer service calls or complaints.

At this point, Peter Merholz mentioned something called the Net Promoter Score and said that this metric was currently proving popular in the US. I have to admit my ignorance of this, so went off and did some research. It turns out that the Net Promoters Score (NPS for short) is a really simple, yet quite powerful metric for measuring customer satisfaction. Simple because it’s derived from asking a single question, powerful because it appears to have a direct correlation to the growth and profitability of a company.

To calculate your Net Promoters Score, you ask your customers “how likely they would be to recommend you to a friend”, and get them to grade their answers on a scale of zero to ten. Zero would be extremely unlikely while ten would be highly likely. Those who answer nine or ten are considered promoters, and are the most likely people to evangelise your services. Those who answer between zero and six are considered detractors and are the type of people who will spread negative views about your services. People who score between seven and eight are passive. They are generally happy with your product or service, but are likely to switch if something better came along.

To work out your Net Promoters Score, you simply subtract the percentage of detractors from the percentage of promoters. A good score would be in the range of 50-80%, while an average score would be 5-10%. A poor score would be in the negatives, and I can think of quite a few companies that would fit into that category.

I can definitely see the value of using this metric to help judge the success of a site redesign. You could survey all of your customers just before the redesign, and again a few months later, once the new site has bedded down. The greater the change, the more successful the project.

Posted at September 21, 2007 12:09 PM

Comments

Nate Klaiber said on September 21, 2007 1:48 PM

The challenge is to get everyone involved and taking your survey. Do you think those that fall in the 0-6 would take the time to help you by filling out a survey? Some - but the majority? I can bet the 7-10 range would be willing to help. Wouldn’t it take a large sampling for this to prove effective?

I like the principle/theory, thanks for the reference. I hadn’t heard of this, either.

Harry Brignull said on September 21, 2007 1:54 PM

Since Paul’s question was about metrics, you got an answer about metrics.

NPS sounds good, but a better question would have been something along the lines of “What’s the difference between qualitative and quantitative user experience evaluation? What method should I use in what context?”

It sounds like NPS would only work on live sites that your customers have at least some experience with. Not so good for testing your ideas prior to launch. Back to the old ‘fingers crossed’ approach then?

Ryan Adams said on September 21, 2007 2:40 PM

The BBC use the NPS on it’s sites. I’m not sure I can reveal actual scores for sites, but sites which have a low NPS (say something like 15 or 20) are likely to be threatened with removal. The problem is that sites which don’t have much content and are really pointers to other content tend to get low scores.

Something like www.bbc.co.uk/scotland/ gets a low NPS because people wouldn’t recommend other people go see it, but it’s a good place to get access to lots of Scotland related content.

Tim Beadle said on September 24, 2007 3:12 PM

@Ryan Adams: if (say) bbc.co.uk/scotland/ is basically an aggregation of other sites’ content, is it fair to say that it’s probably relatively low-cost to run?

Thus you could use an NPS/Cost ratio to determine whether a site’s worth keeping or not.

Amy Madsen said on October 1, 2007 9:58 PM

Hi Andy. Thanks for the overview on Net Promoter For any of your readers interested in learning more, I’d like to highlight the official site: http://www.netpromoter.com. We have plenty of general info, blogs, discussion forums, conferences, a job board, monthly newsletter, and more.