Ever go to a conference where the speaker – in order to generate engagement – asks for a show of hands from the audience regarding some question? Watch how people act. Some will be hesitant to put up their hands unless others have. Only then will they raise their hands.
That’s a conformity bias.
The photo at the top of this article is from 12 Angry Men, a classic 1957 movie that should be on everyone’s movie bucket list. (The cast is incredible.) The plot: a young man is tried for murdering his father. It seems like an open-and-shut case. So, the foreman takes an initial open ballot of the 12 jurors hoping to wrap this up fast.
“All those voting guilty, please raise their hands.” Several hands go right up, but a few hesitate, going up one by one only when they see everyone is voting guilty.
That’s conformity bias.
Well, not everyone voted guilty or it would have been a very short movie. Juror #8, played by Henry Fonda, votes not guilty. The movie revolves around Fonda getting his fellow jurors to critically review the evidence, which the defendant’s lawyer failed to do. One by one he convinces his colleagues there’s reasonable doubt and a strong likelihood of outright innocence.
If you’re a student of decision making and how to influence others, watch the movie.
But I digress.
Conformity Bias: A Prominent Type of Response Bias
By bias, we mean that the respondents are not giving answers that reflect their true views – for whatever reason. And in survey research we have many opportunities to introduce bias into our data, be it from instrumentation bias, sample bias, or response bias.
In a nutshell, response bias is a bias that the respondent brings to the process that the surveyor can trigger either through the questionnaire design or the administration process. Or both. The result is that the respondent is not giving an answer that truly reflects his or her views. Frankly, I think respondent bias is a more descriptive term since the bias emanates from the respondent.
Many (most?) surveyors are either blissfully unaware of response biases or assume they are trivial. They are not. Researchers should always challenge themselves about whether they are introducing some response bias that can pollute the research data.
Survey Sample Size Calculator
Get our Excel-based calculator. It can also be used to gauge statistical accuracy after the survey has been completed.
Conformity bias occurs when the respondent gives answers that fit social norms regardless of whether that’s how they truly feel. It’s also known as Social Desirability Bias. The respondent wants to be “part of the crowd” or “not rock the boat.” Some of the Less Angry Jurors conformed to the views of the overwhelming majority. That’s why initial jury votes are typically done closed or anonymous to eliminate the desire to conform.
Conformity bias is mostly a concern where the respondent is interacting with a live interviewer either in person or over the phone. The lack of anonymity heightens the desire to conform. Acquiescence is another type of response bias where the respondent wants to please the interviewer. So, both types of biases may be in play with a live interviewer.
Conformity Bias Example: Flying the Flag
But even on a webform or paper survey, conformity bias may be in play. Look at the nearby picture of a survey taken in the United States right before our Independence Day celebration in 2019. Flying the flag is patriotic – a social norm. Do you really want to say you’re not patriotic? I very much doubt that unbiased research – driving down streets in randomly selected towns and cities – would find that high a percentage of homes displaying the flag.
What if your research involves personal behaviors? For example, asking:
People how much over the speed limit they typically drive
New mothers if they smoked or drank alcohol during pregnancy
Teenagers if they consume marijuana in some form
People if they are totally honest on their taxes
People if they wash their hands after using a public toilet
No matter how you ask those questions, you’re asking people if they lied, cheated, broke laws, or engaged in dangerous behavior. There’s a natural inclination to give the “right” answer, the socially desirable one.
Perplexingly, for questions like teenage drug consumption the social norm could go in either direction:
Not admitting to breaking the law, or
Claiming to doing what other teenagers are doing even if they’re not. (In my youth, we’d say that was trying to “look cool.”)
As with all biases in research, we never eliminate them, we only control the bias to some degree. (More control is better obviously.) We need to critically examine the various aspects of our surveying process:
Invitations and Introductions. Stress that we want people’s candid views and that there are no right answers. It’s very difficult to say “There are no legal repercussions from your answers” without making people think about any legal issues. Better in most cases to stay away from that.
Question Wording. Avoid loaded language that leads respondents to the right answer or away from giving honest answers. Questions should be worded in a neutral fashion. This could be a real challenge. Be sure to pilot test your survey and look for conformity behavior.
Requiring Answers. Requiring answers puts respondents on guard. I typically don’t like to require answers, or I include a “Prefer not to answer” response option. If the respondent feels threatened, they won’t answer the question, will abandon the survey, or – worse – lie. Oddly, I find when I personally take surveys that I am more likely to give a response if the researcher has given me an opt-out option like “Prefer not to answer.”
Assure Confidentiality. Respondents should be assured that their responses will be treated in a confidential manner. This should be part of the Invitation and/or Introduction.
Conduct Anonymous Surveys. Confidentiality may not be enough to sway people to be candid. Logically, anonymity is the best way to get people to be honest on questions where they might feel threatened if they are honest. But what surveys are anonymous? Paper surveys by postal mail provide the best guarantee to the respondent of anonymity. You can promise anonymity on a webform survey. Will people believe you?
Use a Third Party to Conduct the Survey. Strangely, people will say things to a third party that they wouldn’t say to a first party. So, if your survey may provoke conformity behavior, consider hiring a firm to conduct the survey on your behalf. Somehow, that buffer between the respondent and the survey sponsor can mitigate this bias.
Again, look at your surveying practices for things that could induce conformity (or other biases). Better yet, have an outsider probe and prod for the survey shortcomings. An outsider may see things to which you are blind.