CNN published a poll on June 10, 2020 (see nearby data table & graph) that asked whether people now felt “comfortable” returning to their “regular routine”. The data show an increase in comfort level, but the startling finding can be found in the graph.
Comfort Level Returning to Regular Routine
While both (self-described) democrats and republicans show increasing comfort in the past month, the disparity between the two groups is huge – more than 3 to 1.
Comfort Level w/Regular Routine by Party
In a companion article I examined possible reasons for this huge disparity between respondents from each political party:
But there are yet two other possibilities, :
As a survey trainer, I find people are skeptical about the impact of biases upon a survey’s results.
This poll provides indisputable proof of some bias distorting the data. The pollsters provides some detail about the polling methodology, and they report the sampling error – which is driven by the number of responses. But they provide no discussion about sample bias or conformity bias.
A moment’s reflection upon their findings shows something is in play.
Look at the nearby results taken from CNN’s detail report for Question 15, which asks if “you, personally, know someone who has been diagnosed with the novel coronavirus.”
CNN Poll: % Know Someone Covid Positive
The number of positive cases in the US increased by 263% from April 6 to May 10 and by 43% from May 10 to June 5. The percent saying they knew a Covid-19 positive person almost doubled from April to May.
Yet the percent of people knowing someone who tested positive stayed flat from May to June. How is that possible?
The percentage clearly should have tracked up! (Going down would be truly odd.) And as the number of positive cases increases, the probability of anyone knowing a positive person increases exponentially. (The process would parallel the logic of the virus spread.) It is highly unlikely that this can be explained only by sampling error.
Further, look at the nearby table.
Count Knowing Covid Positive Person
If the polling data are accurate, one helluva lot of people knew each person who was positive with Covid-19. Extrapolating to the US adult population for the June 5 polling, each Covid-19 positive person was known by 44 people on average. (For May it would be 63 people, and for April, 127 people.)
Even though the question emphasizes “personally” knowing someone with Covid-19, the numbers just don’t make sense.
This strikes me as a classic example of conformity bias, which is a type of response bias where respondents don’t provide an accurate response due to something inherent in them. Conformity bias is where people give an answer to be “one of the crowd.” (I won’t about it here.)
For some reason people may want to say they personally know someone with Covid-19 even if they don’t. (False empathy?) I noted a similar effect on another survey involving Boston’s Mayor Menino.
Here’s a commonly used analogy (from my home city): Tens of thousands of people claim to have been at Fenway Park for Ted Williams last game in 1960 in which he hit a home run in his last at bat. But attendance was only 10,454. People want to say they were part of it. Same thing here?
But how could the percent not increase from May to June? Are people just making this up – that is lying to conform? And they didn’t lie consistently from May to June? Yup, that’s possible.
Running your own survey programs? Learn how to do it right from us — the premier worldwide trainers for survey design & analysis.
Featured Classes:
Another possibility is that the sampling done to conduct the survey had errors, inadvertent or not. The polling methodology (see nearby call-out) makes no mention how the 1,259 people sampled were selected. We know there was oversampling of smaller demographic groups, presumably to decrease the sampling error for those groups. But there’s no mention of a sampling frame to ensure the sample is representative of the entire US.
CNN Poll Methodology
If no sampling frame was used, that is, if people were just randomly called from the entire US rather than from a series of districts that were representative of the US, then it is possible that the June poll overweighted urban, democratic areas. We do know that they oversampled minorities, and minorities are most likely to live in urban areas.
Did they adjust all the data to remove the oversampling process? That’s not clear and could lead to a true sample bias!
From just reading the summary data and statements of methodology, it’s impossible to know what lead to the bizarre “findings” from the question asking about knowing someone who has tested positive. However, logic tells us that something is wrong.
What astonished me most is that no one involved in the polling and reporting noticed this oddity. That’s a real head scratcher.
Note that this casts doubt upon the rest of the findings from this survey. The effects that lead to the result for this question may have corrupted other questions, but the result isn’t so obvious.
~ ~ ~
Survey data analytics and response biases
are topics covered in our
Survey Design & Data Analysis Workshops.