What’s the point? An Unactionable Transactional Survey

Summary: Transactional surveys are a vital cog for operational improvement and in customer retention. The HomeAway customer experience transactional survey doesn’t generate actionable data, including identifying customers in need of a service recovery event. This article reviews the shortcomings of this customer support transactional survey.

One of the great things about writing articles on customer experience survey practices is that examples of good and bad survey practice fall into one’s lap. As a customer of HomeAway and its sibling VRBO, both sites for advertising vacation homes, I recently had an as-yet unresolved issue that required me to contact their support organization. Some time after my initial email exchange with a support representative, I got an invitation to take a customer experience survey.

I was pleasantly surprised to get this survey invitation since every other feedback survey invitation I had received from this company was a misrepresentation. While the invitations had said they were feedback surveys, they were really market research surveys. I don’t like to be lied to.

My initial customer experiences were positive in this support incident, so I gave a positive review, and — foolishly in hindsight — put some thought into giving some specific comments. A day or two later, I sent a follow-up email to support about my issue, and got no response. I sent yet another email and got no response. That made me want to amend my previous survey submission. So, I dug out the invitation email and clicked on the link. There was the survey web form, but I wasn’t amending the original survey submission. I was just back at the original blank survey web form.

Had I looked more closely at the initial invitation and the survey web form, I would have realized that this was just a web page to which anyone could navigate. I did not see any key structure in the URL or anything else that would control access to the page. Heck, they didn’t even plant a cookie on my computer to prevent me from a resubmission. So, I was able to submit another survey response.

Why is this bad transactional survey design practice?

First, the folks who run this support operation I do not believe have thought through their support model. What triggered the invitation to take the survey? Is it based on each email I send to support? (I think that’s the case since I got a second invitation based — I believe — on the same incident.) In most all support environments, one incident may be comprised of several interactions, possibly through multiple communication channels. This survey would seem to presume that all incidents need only one interaction. Look at the penultimate question: Was your issue completely resolved with the response you received? I will bet the management views “no” responses as negative. Yet anyone with a support service background knows that several interactions may be necessary to understand the nature of the issue and to address all the issues. Note they did not ask if my issues had been resolved.

Second, they have limited control over the survey submissions. It appears I could submit countless survey responses. Perhaps my last one overwrites my first one.  If I can submit multiple responses, that of course, would corrupt the data set. Beyond that, non-customers of HomeAway could submit surveys. Without proper survey administration controls, data validity is definitely called into question. I certainly wouldn’t feel confident taking action based on these data.

That leads to my third point. This transactional survey does not generate structured, actionable data. As the manager of HomeAway’s customer support operation, what would you do with the results of this survey each day, week, or month? The survey has three very high level summary questions and one open-ended question to solicit diagnostic information. The survey has no closed-ended questions to diagnose shortcomings in the service delivery. Comments can be very useful, but to rely on that sole question for actionable data seems dubious.

Fourth, perhaps the most critical action that one can take from a transactional survey is to engage in service recovery, that is, to address and resolve issues of a specific customer to save them as a customer. How do you do that with this survey? They don’t know who the submitter is, and they don’t ask if the respondent wants a follow-up. In the opening paragraph, I said that I had wasted my time giving specific feedback. This is why. I actually thought that someone with some authority would review my comments and take action on them or would follow up with me. How silly of me.

This survey can only be used for summary evaluations of the support organization. They can’t even evaluate specific support representatives. Maybe the senior customer service manager needed a metric, just a metric, to argue for his bonus.

Here’s some free consulting to HomeAway. If you really want to tick off customers, ask them for specific feedback and then ignore them. Here, HomeAway has no ability to engage in recovery — unless they’re tracking the IP address of my computer.

I actually told the customer support representative that unless HomeAway changes a critical policy that I would not be renewing my subscriptions to both HomeAway and VRBO. They have no customer guarantee, so canceling the subscriptions gets me no refund, but I will be looking for other advertising vehicles for my Harpswell, Maine rental.

The survey practices of this transactional survey reinforce my view that HomeAway has a very poor customer ethic. I guess I understand why HomeAway purchased the website, HomeAwaySucks.com — but you’ll find many forums for rental owners that sing the shortcomings of HomeAway.