Ever go to a conference where the speaker – in order to generate engagement – asks for a show of hands from the audience regarding some question? Watch how people act. Some will be hesitant to put up their hands unless others have. Only then will they raise their hands.
That’s a conformity bias.
The photo at the top of this article is from 12 Angry Men, a classic 1957 movie that should be on everyone’s movie bucket list. (The cast is incredible.) The plot: a young man is tried for murdering his father. It seems like an open-and-shut case. So, the foreman takes an initial open ballot of the 12 jurors hoping to wrap this up fast.
“All those voting guilty, please raise their hands.” Several hands go right up, but a few hesitate, going up one by one only when they see everyone is voting guilty.
That’s conformity bias.
Well, not everyone voted guilty or it would have been a very short movie. Juror #8, played by Henry Fonda, votes not guilty. The movie revolves around Fonda getting his fellow jurors to critically review the evidence, which the defendant’s lawyer failed to do. One by one he convinces his colleagues there’s reasonable doubt and a strong likelihood of outright innocence.
If you’re a student of decision making and how to influence others, watch the movie.
But I digress.
By bias, we mean that the respondents are not giving answers that reflect their true views – for whatever reason. And in survey research we have many opportunities to introduce bias into our data, be it from instrumentation bias, sample bias, or response bias.
In a nutshell, response bias is a bias that the respondent brings to the process that the surveyor can trigger either through the questionnaire design or the administration process. Or both. The result is that the respondent is not giving an answer that truly reflects his or her views. Frankly, I think respondent bias is a more descriptive term since the bias emanates from the respondent.
Many (most?) surveyors are either blissfully unaware of response biases or assume they are trivial. They are not. Researchers should always challenge themselves about whether they are introducing some response bias that can pollute the research data.
Conformity bias occurs when the respondent gives answers that fit social norms regardless of whether that’s how they truly feel. It’s also known as Social Desirability Bias. The respondent wants to be “part of the crowd” or “not rock the boat.” Some of the Less Angry Jurors conformed to the views of the overwhelming majority. That’s why initial jury votes are typically done closed or anonymous to eliminate the desire to conform.
Conformity bias is mostly a concern where the respondent is interacting with a live interviewer either in person or over the phone. The lack of anonymity heightens the desire to conform. Acquiescence is another type of response bias where the respondent wants to please the interviewer. So, both types of biases may be in play with a live interviewer.
Conformity Bias Example: Flying the Flag
But even on a webform or paper survey, conformity bias may be in play. Look at the nearby picture of a survey taken in the United States right before our Independence Day celebration in 2019. Flying the flag is patriotic – a social norm. Do you really want to say you’re not patriotic? I very much doubt that unbiased research – driving down streets in randomly selected towns and cities – would find that high a percentage of homes displaying the flag.
What if your research involves personal behaviors? For example, asking:
No matter how you ask those questions, you’re asking people if they lied, cheated, broke laws, or engaged in dangerous behavior. There’s a natural inclination to give the “right” answer, the socially desirable one.
Perplexingly, for questions like teenage drug consumption the social norm could go in either direction:
Running your own survey programs? Learn how to do it right from us — the premier worldwide trainers for survey design & analysis.
Featured Classes:
As with all biases in research, we never eliminate them, we only control the bias to some degree. (More control is better obviously.) We need to critically examine the various aspects of our surveying process:
Again, look at your surveying practices for things that could induce conformity (or other biases). Better yet, have an outsider probe and prod for the survey shortcomings. An outsider may see things to which you are blind.