Sears IVR Customer Satisfaction Survey

in

You would probably think that mistakes in survey design would be made by small companies with limited resources and knowledge. Yet, some of the best examples of bad survey design practice I find in big companies. In this article, I’ll illustrate the mistakes made in the IVR  survey to measure Sears customer satisfaction. (Home Depot’s store visit survey is another glaring example of bad survey design.) Read my related article regarding to learn the strengths and weaknesses of IVR survey administration method.

Note: If you have landed on this page because you ran a search on “Sears Survey,” please note that you are NOT on the Sears website. We get people contacting us thinking we’re Sears looking for all kinds of help. We’re not unsympathetic, but please contact Sears. By the way, it appears that entering the Sears survey gets you on all kinds of mailing lists. Beware.

In a nutshell, the Sears IVR customer satisfaction survey:

  • Didn’t allow me, the customer, to fully express my feelings about my entire transaction with Sears
  • Forced me to answer irrelevant (to me) survey questions
  • Force fit a suvey question to a scale to the point where the data generated are not interpretable (that is, they generate garbage data)
  • Asked me for comments — and then ignored me.
  • Most importantly, lead to greater customer dissatisfaction and destroyed any customer loyalty I had to Sears

The last bullet should startle you. Customer surveys are typically meant to provide customer measurements to increase customer satisfaction. So, how could a survey program design actually lead to greater customer dissatisfaction? The answer lies in a poorly conceived survey project and program design that leads the customer to come away with a more negative attitude toward the company in addition to generating survey data of questionable value.

Before I explain all of these survey design problems in detail and how to avoid them — hint: the answer lies in having a good customer survey program design — some background on my interaction with Sears customer service is needed.

My Sears Service Interaction

I bought a hot water heater in 1997 from Sears. It had a lifetime warranty. In September of 2007 the heater failed; it cracked. My transaction with Sears evolved into the following series of interactions.

  1. Claim Initialization. I called the Sears call center at 800-4MyHome to inquire how I would invoke my warranty protection for the failed hot water heater. This interaction was great. They actually had my purchase on record, so I didn’t even need my receipt, which I did have. They arranged for a repair tech to come to my house two days later to verify whether the tank failure was covered under warranty. I wasn’t thrilled with the 2-day wait, but I had backup hot water capabilities. The agent told me that should the technician verify that the failure was covered by my warranty, then I would not be levied the $69 service charge for the technician visit. The explanation was poorly worded, so I restated what I thought the agent said to get verification.
  2. Technician Verification of My Claim. The technician came to my house, verified my claim, gave me the paperwork to get a new hot water heater at my local Sears store — and charged me the $69 service charge. Foolishly, I paid it (with my credit card). I figured getting my refund from a company of Sears’ stature would be no problem. Boy was I wrong.
  3. Fulfillment of my claim. The field tech gave me a form to take to my local store to get my replacement heater. My day was shot so I went to the Marlboro Sears store to get my new heater, but they told me the paperwork would take 24 hours to get into the IT system. Why didn’t the technician tell me this? I was told I would get a call the next day. I didn’t. I had to call them. I did eventually get my new heater, got it installed, etc.
  4. Resolving the Improper Charge. When questioning the charge with the field tech at my house, he told me to call the local office in Danvers, Mass. rather than call the service center. He said I could talk to a “customer relationship specialist” there. The person with whom I spoke was the most useless, aggravating person I have ever engaged. He cut me off in mid sentence, refused to let me explain the situation and basically told me whatever the tech said is right. He was completely unconcerned about the discrepancy between the service center and the tech.  He was so exasperating, I hung up on him. If he is a “customer relationship specialist,” then Sears is in big trouble.I called 1-800-4MyHome again and verified that I should not have been charged. They told me to get my refund by working through my local Sears store. I tried unsuccessfully for months. I wrote to the Sears headquarters about this likely illegal practice in the hopes I could avoid going through my state’s consumer affairs division.A month after my letter was delivered I got a call from a troubleshooter at Sears headquarters. She told me the charge was legitimate. The service agents at 800-4MyHome gave me wrong information and Sears is not responsible if its agents give a customer wrong information that is the basis for establishing a contractual arrangement between Sears and the customer. Tough luck. End of story. I was flabbergasted. My jaw dropped.

Soon after the technician’s visit, I received an automated phone message asking me to take an IVR survey. I should have transcribed the survey, but I can do it good justice. The 6 or 7 question survey — it was one question longer than stated in the introduction — posed all its questions on a satisfaction scale, where 1 represented Very Dissatisfied and 5 represented Very Satisfied. All of the questions pertained to the technician’s visit to my home, which, I’ll argue, was a glaring mistake. But the biggest mistake was what Sears did with my survey data.

What Lessons Can We Learn from the Sears IVR Survey?

Keep the Survey Design Focused. I teach in my Survey Design Workshop to keep a tight focus on the survey contents and not let other departments meddle their way into the survey design process. This I believe happened at Sears. The first 6 questions were about the technician’s performance, but the last question, asked about my “satisfaction with the technician making me aware of Sears’ products and services.” That’s not something about which I give two hoots, and I suspect the operational folks in Sears service don’t either. But the marketing folks do. Clearly, the technician is tasked to promote Sears products, which is one of those strange — and inappropriate roles — for service technicians. Survey focus is achieved by having a good Statement of Purpose or Research Objectives — and sticking to it.

Word Questions to the Scale. The question mentioned immediately above also displays a classic problem in survey question design: wording a question to fit the scale. Look at the question. It was posed on a 1 to 5 satisfaction scale. I had no idea how to respond. The technician didn’t make me aware of any Sears products. I didn’t know he was supposed to, and I was glad he didn’t. So, does that mean I should have scored it a 5 indicating Very Satisfied with the technician, but not because he nicely made me aware of Sears products but because he didn’t waste my time with a sales pitch that I would at best find annoying? Or should I score it a 1 indicating Very Dissatisfied because… well, I don’t know why I would. But I am sure the marketing folks who pushed for this question would want me to score it a 1.  I didn’t. I did the cop-out, middle-of-the-road, 3. Regardless, the marketing folks at Sears are interpreting the results from this survey question that is simply generating garbage data.

Develop Comprehensive Response Options. That last question really needed a Not Applicable option. I tried just not answering, but the system would not let me do that. So, as said, I entered a 3. I had a sense I would soon get the opportunity to answer an open-ended question, so I had to get beyond this question.

Develop Comprehensive Question Set. While we want to keep any survey, but especially an IVR transactional survey, short and sweet, you still need to pose all the needed questions. Usually, my criticism of surveys is that they go too far a field. The Sears survey was too narrow. The technician gave me factually incomplete information about how to get my replacement heater, and I was charged improperly. Those critical flaws in the transaction went unaddressed in the survey questions. Trust me; those factors have made the deep impressions on me about Sears on-site service.

Think Beyond the Interaction to the Transaction. A service transaction is a chain of interactions, and the weak link needs to be identified. The Sears survey only asked me about my experience with the technician. My transaction with Sears went well beyond the technician’s visit, including my interaction with the service center and the store. No one ever asked me about those interactions, and thus did not allow me to express my full feelings about my experience. (You’re probably thinking, “but wasn’t there an open-ended comment question?” Hold that thought!) Yet, I’m sure those who designed this awful survey think it’s great. I, as the customer, came away very frustrated since I had had a poor experience and did not get to say my piece.

Notice how this issue relates to the need for a good Statement of Purpose. While the typical customer (survey respondent) won’t think in these terms, they will know if a survey is comprehensive and well designed. The Sears IVR survey wasn’t. Why? My guess is that Sears has very stovepipe functional groups that do not interact.

Need for a Service Recovery Program. The above mistakes are bad but not egregious. The most disastrous error in the Sears survey is what they did NOT do with my data. The last question presented the opportunity to make a free-form comment about my experience. I was waiting for that opportunity.  I even developed notes to be sure I covered everything. I listed the litany of mistakes that Sears made. You would think that the person who analyzes that textual data would flag me for a follow-up call. A well-run company with a well-designed survey program would have done that. Sears did not. They ignored my cry for help, making me angrier than I already was. Yes, the survey — designed to measure customer satisfaction and identify needed operational improvements — actually made the situation worse!

Need for an Integrated Feedback Management Strategy. Even if someone had read and responded to my comments, it might not have mattered. I recounted in the background on my Sears transaction that Sears headquarters basically told me to get lost. Sears sees no ethical or legal obligation to honor the verbal contract established by its service agents. This is a pretty startling indictment of the strategic role of customer service at Sears. But why does Sears go to the bother and expense of having this one feedback survey that sits like an isolated island disconnected from its potential business value?

Will I ever shop at Sears again? You know the answer. That $69 savings is quite the Pyrrhic victory for Sears.

Addendum #1: Missed Value in a Service Recovery Act. After the headquarters’ troubleshooter told me to get lost, I started looking at my legal options, including initiating a grievance through my state’s attorney general office. Then, 6 weeks after phone call with Sears, a check for $69 shows up from Sears. No accompanying letter. No accompanying phone call.

Part of the purpose of a service recovery act is to win back the hearts and minds of the customer. The check partially won back my mind, but my heart? I’m still seething at the process and how I was treated. Compare this to how Subaru handled its service recovery of a flawed service recovery. Sears missed an opportunity here to win me back.

Addendum#2: Some People at Sears Do Take Ownership. Some Sears headquarters’ folks came across this article and one bought my Customer Survey Guidebook. And in Fall 2009, this person and a colleague came to my Voice of the Customer conference. We had some nice and interesting chats. I give them credit for not blaming the messenger. I actually have shopped in a Sears store with a little more confidence that customer service can be a trait at Sears.

~ ~ ~

Here’s some of the emails we’ve received from people thinking we’re Sears… unedited but anonymous.

  • sears tool #9 43664 rachet wrench reversable angle 13mmx14mm, “NO ONE” can find this tool!!.. please call
  • hello my name is XXX  we have been along time customer of sears but these days i really wonder if you really do think about taking care of your customers and i dont mean just a single store i have been in contact with the store  and every single phone number i can possibly find on or about sears  you people just seem to want to give lip service and nothing else i guess when sears changed owners you really have  a problem helping customers with their problems or you just run a big scam how your customers are really number one well im here to tell you the word of mouth is still a time tested and very effective way of comunicating even in this day of age  and believe me i know a lot of people  common and business men and women alike an old man once told a car dealer rather large that did  a friend of his  wrong  he said to them if you do people right  i will tell everyone i know and if you do people wrong i will tell everyone i know and he did the business went bank rupt after on
  • I was in the sdtore shopping and on my slip is says to go to your web site and enter for a 4,000 sears gift card, i did not liket he site all it did was ask me question about college and buyin other things, this is faluse addvertising. now all these people are calling me. what about the gift cert you said about, never did get any information on it. I am a sears customer  not all this other junk.
  • Today, I called the number for Sears repair and service in my local phone book. The automatic answering machine did NOT give me any option to speak with the people at the service center in Melrose Park, where I left my vacuum cleaner for repairs a week ago. And there is no contact number on the receipt I received when I dropped off the appliance. So can you help me reach this repair center???? The woman to whom I was transferred merely hung up on me when I told her what I need.