When deciding on an online web survey tool what are the key features to consider and how should you go about the selection process? This article reviews important features of online survey tools and the decision process for selecting survey software, such as SurveyMonkey or QuestionPro.
Internet survey software tools make it easy for anyone to do a survey. SurveyMonkey, the best known of these web survey tools, states its goal is “to enable anyone to create professional online surveys quickly and easily.”
That’s true. Survey Monkey, Zoomerang ™, QuestionPro, and all the rest of the online survey tools DO make it simple to create a survey and launch it. They can be a boon to capturing customer feedback as well as from employees, and other stakeholder groups, quickly and simply.
However, using these survey software tools can be like driving without a rear view mirror and without a map — and perhaps with a distorted windshield. You may think you’ve reached your intended destination using the best route. But in fact, you may not have taken the best route. Worse, you may not even be at your intended destination, and you won’t know it. And another destination may have been a better one to meet your needs.
In this article, I’ll define survey software tools, and then discuss their benefits, the two types of web survey tools, their key features, and — to conclude the thought above — the dangers of “driving” with these online survey tools. (Hint: they can’t tell you how to create a good survey.)
Simply put, survey software tools provide the capability to create a survey where the painstaking survey administration tasks are automated. Typically, we think of these automation tools as online survey tools that generate an HTML form to post to the internet, allowing your group of interest to submit their responses via the web. That’s the focus of the remainder of this article. However, there are other survey automation tools, such as
Computer Aided Telephone Interviewing (CATI) software that manages surveys delivered by telephone interviews.
Interactive Voice Response (IVR) surveys, which are like telephone surveys, except that the scripts are recorded and the respondent enters answers using the keypad of their telephone. These are frequently used in call centers at the conclusion of customer service calls.
Optical Mark Recogniztion (OMR) or Optical Character Recognition (OCR) that allow you to scan a paper-based survey to capture the responses. (Think back to your high school SAT exams…)
The key benefits that can be derived from internet survey software tools are:
Simplicity. You can create a survey quite quickly — once you learn how to use the tool. Survey invitations are typically sent out electronically through an email. That email will contain a link to an HTML form that is the survey questionnaire. Responses are then captured electronically, either through incoming emails or by direct loading into a data base, depending on the type of survey tool used. This data collection function is by far the key benefit of online survey tools. If you’ve ever processed a hardcopy survey by postal mail and keyed in the data, you’ll see the benefit from survey automation tools very quickly!
Low Cost. The out-of-pocket cost of these tools is quite low. You can buy a PC-based tool for $500 or less or you can buy it as a service in a hosted application, typically paying a monthly or annual subscription fee. The fees can be as low as $15 to $20 per month for a basic tool. If you need more sophisticated features, the monthly cost could be much higher. As noted above, the major cost is not the acquisition cost of the survey capabilities, but rather, the time to learn how to use the survey software tool.
Scalability. Unlike telephone or postal mail surveys, you can survey 10,000 people as easily as 10 people. This scalability is part of what makes this a low cost method.
Quick Data Capture Turnaround. Since you may be soliciting feedback information about organizational performance, such as a customer satisfaction survey, how quickly you get the data back is important, especially if this is a transactional or event survey. Quick data capture allows addressing a customer issue promptly, which is a key element in successful service recovery practices.
With email most of us practice a “do or delete” approach. People will either take the survey or delete the invitation. So, you’ll find with a web form survey that most of your responses will come back in one to two days. Responses will quickly tail off to nothing. Only IVR surveys can be faster since they are administered at the close of the transaction.
Response Rate. When web-form surveys were a novelty, response rates were quite high, perhaps above 50%. Based upon my own survey project work and feedback from those in my Survey Design Workshop, response rates typically are in the 20% to 40% range. However, the relationship you have with your group of interest is the single most important factor in driving the response rate, not the administration method.
Questionnaire Complexity. Internet web form surveys are fairly flexible. (We can argue forever about which is more flexible: a printed page or a web form.) You can create a fairly complex survey questionnaire with these tools, using extensive branching and other available survey features. Do note that the survey tool you chose will set constraints on the questionnaire you design! It’s better to determine what functionality you need and then select the tool, rather than the other way around.
Yes, Dangers. I opened this article with some seemingly twisted logic about survey software taking you blindly to a wrong destination without you realizing you were blinded. Let me explain…
Survey software tools make it easy to write a good survey. True. They also make it easy to write a lousy survey! And you’re probably better off doing no survey than doing a lousy survey. Why? Because a poorly designed survey can generate misleading data that provides you with delusions of knowledge. Researchers refer to this as “instrument validity“, which simply means that the survey instrument measures what you intend to measure. That may sound like a no-brainer. It’s not! Examples of invalid surveys abound, many done by professional market research organizations.
These online survey tools cannot tell you how to create a good survey. They also can’t tell you that you have a bad survey! Only knowledge of good survey design practices will help prevent you from designing a bad survey. (Obviously, this is why I recommend you buy my Customer Survey Guidebook or attend one of my Survey Design Workshops!)
The Internet web form survey approach exacerbates this blind spot. With a paper-based survey, respondents will give you feedback on the quality of your survey questionnaire. You’ll see big question marks and comments in the margin. (I’ve been known to do this on occasion…) With telephone surveys, you can listen to respondents formulate questions or talk with the interviewers to find out where the survey instrument has shortcomings.
How does the respondent provide you this feedback about an online, web survey? Perhaps you’ll see something in a comment field. But I doubt it. This is an incredible danger zone for this surveying method — and one that the survey software tool vendors never mention.
Technical knowledge of how to use the survey software too does not equate to knowledge of how to design a valid survey questionnaire!!
A twelve year old can push a car’s gas pedal and turn the steering wheel. Would you want them to drive a car? Yet, today many well-intentioned people are using SurveyMonkey and the like, creating survey questions that lack validity and then make business decisions on that bogus data.