Section headers are an important component in survey questionnaire design. By presenting the general topic, they help keep the respondent on track regarding the questions being asked. But we also have to be sure the survey headers don’t confuse the respondent. Yes, that can happen – as we will show from (Im)Proper Use of Section Headers.
In this article we’ll address Section Headers: the advantages of including them in a survey structure, proper use of section headers, and finally, improper use of section headers. We’ll show how reckless use of headers can compromise a questionnaire’s validity. Examples are drawn from a corporate survey and a town resident survey.
Each design element in a survey questionnaire – invitations, introductions, closings, etc. – has one or more roles to play in the surveying process. The roles may be to:
Sections Headers are one of the survey questionnaire design elements. Their proper use delivers three main benefits.
The point is that section headers are organizational elements, and a survey designer will use them to organize his thinking about the survey research – and to catch holes in the survey questionnaire.
Frankly, I’m not aware of any – unless you misuse them. (see below.) Please tell me if you think there are any!
So, what’s the right way to use section headers?
Describe the Topic. The section headers should describe what that section covers. That seems obvious, but many times I see cryptic, perfunctory headers that may do more harm than good. (See below.) Section headers should be brief, but not so brief that they are meaningless. 1 to 5 words would seem appropriate, but be cautious about one-word headers.
You might also want to include some descriptive text, but beware of over describing. Respondents aren’t going to read a bunch of blah blah blah. Concisely say what’s needed, if at all. For example, a section on Demographics should give a brief explanation of why these questions are needed. They’re prying, after all. Explain the necessity for the personal intrusion.
Running your own survey programs? Learn how to do it right from us — the premier worldwide trainers for survey design & analysis.
Featured Classes:
Section Organization & Implementation. Ideally try to have one topical section per web page and try to get all the questions visible on one screen. With surveys rendered for mobile devices, that’s not possible, but it may be possible for surveys taken on laptops. Sometimes scrolling will be necessary within a section, but try to minimize it.
Section Flow. The topical sections should flow in a way that is logical to the respondent, which may not be what’s logical to the survey designer or sponsor. We also want the first questions in a survey to engage the respondents, make them see that the survey is interesting and will provide an opportunity to say what they have to say. However, in a logical flow of topical sections, the engaging questions might not be in the first section! If so, I would probably first ask some summary evaluation questions to engage the respondent. Respondent engagement is crucial!
Okay, let’s get to some examples. Look at the nearby screenshot. This company was surveying employees who worked out of field offices, but interact frequently with folks in headquarters – a common situation. The section header says “Human Resources” so the questions pertain to “HR”, right?
HR Section on “HR” in Corporate Survey
Read the questions. Questions 8 & 10 sure seem like classic HR questions: 8) proper managerial training and 10) resource availability. But what about question 9? Given the header – and the preceding question – aren’t they asking about HR folks in the Home Office? That’s what I thought, but that was not the goal of the survey sponsor. I was told the question related to any Home Office personnel that the field person might contact: logistics, sales, HR, etc.
Wow. That’s a game changer – or a section changer. “Human Resources” was not supposed to refer to the HR department (in the Home Office). It refers to any human resources (meaning “people”) in the Home Office. This section header creates ambiguity.
Respondents will likely have different interpretations of the questions, especially question 9, driven by the section header, the interpretations will differ from the survey designer’s intent. This is a good example of a question’s lack of validity; the data captured won’t represent the true situation; analysis will be faulty – all due to the header.
Let’s move to another example. My town’s planning committee recently conducted a town-wide survey. (Full disclosure: I had been asked by a town employee to help the committee with a related survey. I agreed. Then I was told they didn’t want my assistance. Yes, they turned down free assistance from someone with extensive survey design experience.)
The second section of the town survey has “Services” as its header. (See nearby screenshot.) So, the section is about town services, right? The first question in the section (Question 4 overall) asks for the town’s “top three strengths.”
Services Section of Town Resident Survey
As written, it’s a head scratcher. Is this about the three best services delivered by the town? That’s what the header implies. But if you read the question skipping over the header, as many people are likely to do, the question may be interpreted as “the top three reasons for living in Bolton.”
But wait! Just prior, Question 2 asked, “Why do you live in Bolton?” (See nearby screenshot.) Look at the response option checklist. Wanna bet that those are parroted back in the “top three” list for Question 4? Wouldn’t you live in a town (Question 2) due to its strengths (Question4)? They’re essentially the same question.
Of course, only the “Good schools” option is a service. So, whatever the intentions of the survey designer, a serious sequencing issue will color the entries in question 4.
Key Town Attractions – Town Resident Survey
In addition to the question wording, cryptic header, and sequencing issues, soliciting ordered responses as free-form text will be painful to analyze.
Now let’s show how a header can set the stage and lend (some needed) clarity. The next section in the Bolton Town survey was titled “Housing”, and the first question was “Do you have difficulty living in Bolton?” (See nearby screenshot.)
Housing Section of Town Resident Survey
Without the header the question is another head scratcher. What does “difficulty” mean? Even with the header, you have to think about the meaning of the question. I inferred, perhaps incorrectly, that the question is asking about the cost of housing, which may create a “financial difficulty” living in the town.
In other words, is housing prohibitively expensive and are the property taxes a personal budget killer, making it “difficult” to live in Bolton? But nowhere are housing finance or expenses stated.
The header leads the respondent to infer the question is about cost of housing. But using the header for this purpose is a leap of faith.
The question intent was obvious to the question writer, but was it obvious to the question reader? I guarantee this: respondents to this question will have different interpretations of the question. Ambiguity is the killer of survey validity.
Tip: a well-written question leads the respondent to think about their answer, not about what the question means.
Further – and getting away from the role of the header – the question is posed as a binary Yes/No question. Isn’t financial difficulty a scalar concept?
A later section has “Transportation” as its header, but the first question asks about “employment status.” That’s not transportation. The next 3 questions ask about commuting, and they want to analyze those questions in combination with the employment.
But the questions don’t have to be contiguous in order to perform that analysis. The employment question would fit better in the demographics section. Here, it serves to muddy the concept of Transportation.
Opening question in Transportation section of town resident survey
(Aside: the response options aren’t mutually exclusive, and this is a multiple-choice, single-response question type on the web version of the survey. You could be a “Stay at home parent” and be “Employed”. On the hardcopy version of the survey you could check more than one.)
But let’s return to the “Transportation” header. What does that mean? Remember my comment about headers being too cryptic? Here’s another example.
Transportation means more than just commuting. For example, transportation also includes getting to local shopping, especially for seniors who might take a shuttle van. Yet, commuting is the only topic the section covers. So, the header is both cryptic and non-descriptive of the questions posed.
As you can see, even a simple survey can have multiple design flaws – and my town’s survey had more than those mentioned. Here I have focused on (im)proper use of section headers.
Section Headers have a valuable role to play in a survey design, but we have to use them properly! They set the stage for respondent as they encounter the questions that follow. But the headers have to properly set the stage.
Novice surveyors would (and did) miss these and other problems in a survey. That’s why a project team that includes some people with reasonable surveying experience is vital for a valid, reliable survey design that meets the information needs of decision makers.
~ ~ ~
Note: Proper use of questionnaire design elements is one of the many topics covered in our Survey Design Workshop.