Did you ever wonder why polls consistently report finding that 90+% of Americans want universal background checks preemptively-prove-your-innocence prior restraint on rights, and other such nonsensical infringements…and it rarely passes when put to a referendum? Even when it does pass, actual voting results never come within 30 points of the claim.
Blame poor polling technique. Or good technique, if your goal is to invent support for things you know don’t actually have popular support. To find this you have to go beyond news stories about a poll’s results, and look at the poll itself.
Most often, questionable polling results are attributed to asking the wrong (or right) question. You certainly see that in virtually every poll on gun control proposals.
We saw an excellent example of this when the Texas Tribune declared that a recent poll showed overwhelming support for ex parte “red flag” confiscation laws. The problem is that the poll never actually asked anything about that.
Q30. Do you support or oppose allowing courts to require a person determined to be a risk to themselves or others to temporarily surrender guns in their possession?
The courts, of course, can do that now under current laws on the books in every state. The question fails to mention the defining characteristics of new “red flag” orders: ex parte and an absence of probable cause.
Ask one question, present the results as something else.
Another area subject to abuse is the respondent selection methodology or who you’re including in the poll results. Ideally, you want a random sample statistically likely to represent the overall population, and the sample should be large enough do the job.
Most pollsters now avoid one of the selection traps: calling listed landlines. That once-useful, pre-cell phone technique now tends to select an older sample of respondents. The preferred method is random-digit dialing; you’ll get both young and old, rich and poor…a group more likely to represent the population as a whole.
Once you have someone on the phone, there are multiple techniques used to pick one respondent out of a household.
GOOD: The Kish method; asks for the adult with the most recent birthday.
GOOD: The Troldahl-Carter method; birthdate-based, but alternates things like male/female.
BAD: Self-selection; this is why online polls need to be taken with a huge grain of salt. Respondents decide to participate based on having a specific interest in the subject. There is no randomization.
BAD: Quinnipiac, noted for blowing political predictions, polls a mere 15 “regions,” heavily biased toward blue/purple states. And they not only survey the entire state of New York as a “region,” they also poll New York City as its own separate region. Most of the country is left unrepresented. I view Quinnipiac polls only for their entertainment value, although they may be useful for Democrat-specific surveys like primary candidate preferences.
HORRIBLE: “For the landline sample, interviewers requested to speak with the youngest male member of the household who is at least 18 years of age; if there was no male in the household, interviewers requested the youngest female.” This technique is becoming very common; anecdotally, I’ve been told by people who all the telephone survey calls they get now ask for the youngest person; usually the youngest female, and male if no female is available.
That automatically biases the responses, as younger voters tend to be more liberal. And, sadly, in my experience that is also the demographic least likely to understand existing firearms law. Young, ignorant socialists; what could possibly go wrong?
Perhaps a poll doesn’t disclose its methodology, or it appears reasonable. It helps to spot check claims, like the 90% background check approval that is never reached in real voting.
A Pew survey claimed to have found that 30% of adults possessed firearms. Of that group, 19% supposedly claimed to be National Rifle Association members.
The problem is that worked out to over fourteen million “NRA members,” a number more than twice as high as even the NRA has ever claimed. (Pew’s survey was presented as proving that gun owners, and NRA members specifically, support more gun control laws.)
A recent McLaughlin & Associates survey purported to find that Donald Trump has 43 million Twitter followers. Between the @realDonaldTrump and @POTUS, addresses, Trump has 93.3 million followers. Perhaps he does only have 43 million US voter followers, but I’m dubious that more than half of his followers are non-voters or foreigners following another nation’s head of state.
When all else fails, another clue about a poll is accountability. The University of New Hampshire (commonly referred to as the University of North Massachusetts, for its liberal leaning fell over completely to the left) published a survey purporting to show that 94% of New Hampshirites wanted universal background checks. The poll question did specifically ask about that, but…
The result struck many people as highly unlikely for the state’s then-demographics. No one could find anyone who admitted to participating in the survey, pro- or con-background checks.
When asked for the raw polling data, UNH refused to release it.
Nonetheless, the state’s Democrats pushed for background checks. It failed. And in the next election, annoyed voters turned the Executive Council, State Senate, and State House over to the Republicans.
If a pollster won’t show you his work, don’t trust his polls. If he will, make sure it really is what he claimed.