Summary: Survey projects are frequently treated without the due diligence required to ensure success. They seem deceptively simple. Why the need to plan? That, as any project manager knows, is a recipe for failure. Some level of resource commitment is needed from an organization to provide a firm foundation for a successful survey project. This article examines those requirements from both monetary and personnel resource perspectives.
In the decade or more that I’ve been doing survey work I have one observation that I can’t quite explain. Company budgets for customer feedback programs are widely bifurcated. Many companies will write good-size, six-figure (in US dollars) checks to run a comprehensive, customer feedback office. These may be totally outsourced, or done entirely with internal resources, or a combination.
Then there are the companies that will barely spend a dime. I wish that were as big an exaggeration as it sounds, but it’s not. I find there’s little middle ground. Companies either:
- “Get it” and are willing to invest in listening to the customer
- “Get it” but view it as an expense with a dubious return so they spend little, or
- “Don’t get it” but may go through the motions of a poorly funded program.
Far be it for me to say it’s a waste of money to hire one of the large, well-known professional firms to run a feedback program – though you may also be paying a lot for their “name.” They do a very good job, employ sound methodologies — maybe with a few exceptions — deliver a lot of analysis and information with considerable hand holding. (The exceptions I have seen are survey programs designed by market research firms, not the dedicated survey firms, that don’t have solid background in survey design and apparently think there’s nothing to it. I have written some articles about these types of surveys I have encountered.)
On the other extreme, many years ago I got a call from a gentleman in Florida who was put in charge of doing a customer satisfaction survey for his company. He asked me a bunch of questions, and it was readily apparent that he had no background in conducting surveys. In fact, he was very open about that. I gave him a lot of advice, and, yes, I did suggest to him to attend one of the workshop. Sorry, no budget for that. So, I suggested my book. Sorry, no budget for even a book! About a year later I was speaking at a conference in Florida, so I wrote to this person just on the chance he might be able to attend the conference. I was relieved to learn that the survey project had been axed. Given the level of commitment that management was showing, the project would have failed.
So, what budget do you need to run a decent survey project if you’re doing this internally? Let me break the budget into personnel and dollars.
From a personnel standpoint survey programs have two key players. First, a project leader is needed. This person will likely be the person who drafts the survey instruments and thus needs a solid background in survey questionnaire design as well as knowledge of the range of tasks in a survey project. Running the survey program may be a full-time job for the project leader, but that depends on the size of the company and the scope of the program. What is absolutely necessary is that the survey project management role be part of the person’s job plan. I’ve encountered service managers who are told to do a survey “in their spare time.” Yea, right. Who has spare time in their work life, and why would you spend time doing something on which you’re not measured or rewarded?
Perhaps the more important player is the sponsor or champion. In my tale above about the Florida gentleman, it was clear that no one was championing the project. The sponsor needs to argue for the monetary budget, fight the political battles, and make sure that any people assigned to the project team don’t get pulled off for some other project.
The aforementioned project team is the third category of personnel needed on any survey project, especially if the survey program is to be executed primarily or exclusively with internal resources. A survey project has a number of tasks to be executed, more than one person can do handily. Most important, the project team needs to be the review and refinement body for the survey instrument that the project leader is likely writing. The worst survey instruments are those written by one person with no reviewers.
For the monetary side of the budgeting process, the most important and perhaps largest piece is the headcount dollars for the project leader and project team. This could be one half of a full-time equivalent (FTE) to several FTEs for a highly involved, customer feedback program done internally. But beyond headcount there are going to be out-of-pocket expenses. Let me address those through the time line of a survey project.
During the questionnaire design phase of a survey project, most expenses will be minor. If you’re running focus groups as part of an exploratory research project to build a better instrument, you will likely need to rent meeting room space and provide refreshments and some token gift to participants. You may also want to provide some token gift to participants in the pilot testing phase, and you may pay someone to transcribe recorded notes. All of these are minor, especially compared to the personnel costs. (Of course, we’d recommend you attend our survey workshop series to get a solid grounding in survey methodology.)
The survey administration stage will incur most of the out-of-pocket cost of a survey project since that’s where the logistical tasks lie. If you’re using web-form administration, you will need to buy software or rent one of the hosted survey tools. This cost could run from hundreds of dollars to tens of thousands of dollars. A broad range of capabilities exists in these tools, depending upon your requirements. Many companies start out with one of the inexpensive hosted tools, such as SurveyMonkey, QuestionPro, or Zoomerang. (The list of survey tools is very extensive.) These tools will do the job, but as your requirements or sophistication increase, you may well outgrow them. But as with most any product, you don’t know what you don’t know until you use them. Using one of these tools will make you a more informed purchaser of the next generation survey tool. Again, most of the cost here will be personnel-oriented. As with most software applications, the cost of learning to use the survey tool will be far greater than the cost of buying or renting the tool.
Phone surveys done internally on a small scale probably encounter little to no out-of-pocket expenses, but when you get to a certain scale, you need to invest in a software application to manage the surveying process. Postal mail surveys entail considerable expense due to the mailing logistics. Even if you use internal resources to stuff envelopes and key in the data from the returned forms, you still have costs for envelopes, paper, copying, and postage both outbound and inbound – you must provide return postage.
The data analysis and reporting stage will again have very low out-of-pocket expenses. The survey tool you use will likely have an analytical package, and your company likely has a statistical package that you can use. Excel will work fine for most survey analysis purposes. Personnel time is again the major cost here. And don’t forget that included in this stage is any service recovery program where you respond to customers who have voiced serious issues with your products or services along with implementing the findings of the survey program.
If you are budget constrained, you might be asking why you would consider outsourcing a survey project. As with any project, the two main reasons to outsource are:
- Capacity. Do you really have the bandwidth in your staff to do the job? Survey programs look simple on their face, but they are more involved then most people realize — until they are knee-deep into the administration process.
- Competency. The devil’s in the details. Do you have the knowledge of surveying methodology to know all the details that need to be considered to correctly design and execute a survey project? If you don’t, then you might compromise the whole value sought from the program. Bad, invalid data is worse than no data at all.
Survey projects add a third reason for outsourcing:
- Credibility and Confidentiality. Your customers may not tell you things that you really need to know, but they may be willing to tell a third party “the bad news.” Because respondents feel more assured about the confidentiality of their responses, the findings of the program may be more credible. The most obvious example of these factors is personnel surveys, which almost all companies outsource.
It’s your call about whether to outsource your survey project or do it internally, but make sure the decision is made knowledgeably. A company can run its own survey program and do an admirable job, but don’t be penny-wise and pound foolish. You don’t need to spend hundreds of thousands of dollars, but you do need to establish a budget that properly recognizes the resources needed to do a credible job of generating valid feedback from your customers or your employees.