Effortless Experience: Weak & Flawed Research Model
Summary: The book The Effortless Experience claims that the Customer Effort Score (CES) is a good predictor of customer loyalty. This part of the review addresses the shortcomings of the research model. Since the research model does not include measurements of actual customer loyalty behaviors, that greatly weakens claims that CES is a good predictor of loyalty.
~ ~ ~
1. The Weak Research Model
The research design behind the book’s findings has a few shortcomings. First and foremost, for a book about predicting loyalty, they never actually measure loyalty.
Lack of Actual Behavior in the Model. If you read the book, The Effortless Experience, you will constantly read the authors’ claim for the Customer Effort Score as a “predictor of loyalty.” Occasionally, the authors will use the phrasing “intended future loyalty.” For example, on pages 17 and 157, respectively, they write,
In our global survey, we found virtually no statistical relationship between how customer rates a company on a satisfaction survey and their future customer loyalty (see figure 1. 4).
And that’s where the Customer Effort Score really shines — It helps us understand the actual impact of the service experience (and only the service experience) on customer loyalty. (emphasis added)
A world of difference lies between the intended and actual loyalty behaviors. When the authors omit “intended future” in the book and in blog posts, they are misleading and overstating the value of their research findings.
Here’s their research design, as I have inferred it. They posed a survey to a great number of people. This one survey had questions that (attempted to) measure the type and level of effort customers experienced in a service transaction, among other things. It also had questions at the end of the survey that measured what surveyors call “attitudinal outcomes.” Those are the summary questions — e.g., overall satisfaction, likelihood to recommend, likelihood to buy again in the future, and likelihood to buy other products from the company. These summary questions they say describe customer loyalty. Higher scores means higher loyalty; lower scores mean the inverse.
Their statistical analysis compared the scores from the CES question to the scores given for the attitudinal outcome questions on the same survey. However, they never compare scores from service transactions today to actual future customer loyalty behaviors. That better model is what’s depicted in the other nearby figure.
How well do the intended loyalty measures predict actual loyalty behavior? We don’t know. And they never raise that distinction as a critical assumption to their findings. The assumption here is that what people say matches what people do in the future. Professional researchers make clear their assumptions that affect interpretation and application of the findings.
When you read their book, every time they claim predictive abilities for customer loyalty, remember the text should read “intended future customer loyalty.”
Compare this research model to the research behind Net Promoter Score® (NPS). Reichheld et al. compared scores on the recommendation question to actual future industry profitability. Shortcomings exist in that model too, but its longitudinal nature is far, far superior to the point-in-time research done in Effortless.
Service and Only Service in the Model. Notice in that above quotation from page 157 and the one below from page 12 the authors make explicit that the model also treats only data about service interactions.
[W]e intentionally limited the study to service center transactions and their impact on customer loyalty. Obviously, customer loyalty is a product of all the interactions a company has with the customer — its brand reputation, their friends’ and family’s perceptions of the company, the value and the quality of the products, and of course customer service, among other things.
This approach games the result. If you only ask questions about service interactions, that’s what’s going to be shown to be key predictors of the customer loyalty outcome in the regression models (I believe) they ran. If they had included those other causal factors, the statistical significance and level of impact of the service causal factors may have plummeted.
Does Effort Drive Loyalty or Disloyalty? Okay, let’s accept the research model as valid for the moment. The book is still not about its subtitle: Conquering The New Battleground For Customer Loyalty. More accurately, the book is about Conquering The New Battleground For Minimizing Customer Disloyalty. (But that’s not a title to sell a book — or a consulting practice.) As the authors write on page 130:
The argument of this book… is to mitigate disloyalty by reducing customer effort. (emphasis in the original)
If the prescriptions in the book are accurate and you implement them, you will reduce customer disloyalty, not generate loyalty. In a blog post about the research, the practice manager writes,
[W]e recommend focusing on moving customers to at least a “5” on the 7-point CES 2.0 scale (or “somewhat agree”). Moving a customer from a “1” to a “5” boosts their loyalty by 22%, but there are diminishing returns after that — moving a customer from a “5” to a “7” only boosts their loyalty by 2%.
So, reducing effort moves a customer from a disloyal to a neutral position, but reducing the effort further leads to no loyalty improvement. When you’ve reduced customer effort, have you increased customer loyalty? Not if a competitor matches your effort reduction — and then works on the drivers of positive loyalty. The lesson here — never mentioned in the book — is: Once you’ve reduced effort, then delight your customers. That is exactly the argument behind the Kano Model, which has been around for decades. The authors claim to have broken new ground in theory development. They have not.
Or to put it in my own phrasing, “separate the bow-wow from the wow.” (SM) First, eliminate the bow-wow moments, then focus on the wow moments. (If you use that quote, you’d better give me attribution.)
See the related articles:
“The Effortless Experience” Book Review
2. Questionnaire Design & Survey Administration Issues
3. Statistical Errors