Junk Science: Everytown’s ‘Research’ Ignores Established Data to Generate Biased Results

By Elizabeth McGuigan

There are many policy topics worthy of study in the area of criminal justice and the criminal misuse of firearms. That’s why it is mind-boggling to read a recent article in the journal Violence and Gender that takes a well-researched issue — how criminals obtain firearms — and creates a terribly-designed study on the topic. It’s like the researchers decided to reinvent the wheel and came up with a rectangular design.

The article, titled “Baltimore’s Underground Gun Market: Availability of and Access to Guns,” reviews a survey conducted by the anti-gun authors in 2016. Reading through the survey design and the results, it should come as no surprise that the effort was funded by the gun control activists at Everytown for Gun Safety.

The premise of the study makes sense. The authors want to understand how individuals who are prohibited by law from owning firearms end up acquiring guns illegally. The first issue here is that there is a solid body of research that shows time and time again how criminals illegally obtain their firearms.

In 2019, the Department of Justice published their periodic survey of prison inmates, showing that criminals rarely obtain their firearms from retailers. Over time, the share of firearms acquired outside of retail stores has risen to 90 percent. Other studies have confirmed the same fact: criminals frequently obtain firearms by buying or trading from a friend or acquaintance.

Not to be stymied by the fact that the answer is already known, the gun control researchers decided to hang out in “public spaces directly outside of seven parole/probation centers across Baltimore City” and offer men who were exiting the centers a $50 gift card to take a survey. This survey couldn’t possibly be less randomized and is riddled with sources of bias and error.

Biased in Every Way

They talked to 448 men about the survey. Only about half were interested and the final sample ended up at 195 men over 18. The sheer level of sampling bias here is enormous.

The sample was limited to those on parole or probation who are complying with their terms and checking in with the centers. So we know they were caught, convicted, released and are now complying with at least part of their sentence.

The experience of this particular subset of criminals excludes anyone female, under 18, those who did not receive parole/probation as part of their sentences and those who did but are not in compliance. We know that 80 percent were African American and 74 percent were unemployed, but not how that compares to a broader sample of criminals in Baltimore.

The sheer level of sampling bias here is enormous.

This design also excludes any criminal not interested in telling random strangers on the street outside of their parole/probation center how they acquired their firearms. One could assume this would be more likely to filter out those who stole firearms, for example, especially if they have not been caught or prosecuted for the theft. Or, given the 30 to 45 minute process, may have also screened out anyone with somewhere else to be such as work, school or other activities.

The compensation alone biases the willingness to participate and even the responses. What is the impact of offering compensation on the selection bias? It’s not known because there was no control group asked to participate without compensation. And naturally the “brief overview” that the men were greeted with was not published, so it’s not clear if the bias of these known-anti-gun researchers was evident right off the bat.

If there were clear “right” or “wrong” answers, there was likely an element of social-desirability bias and the men may have been answering according to what they believed would secure their $50.

We do know that the participants were only asked about their activities within the prior six months. For those on parole/probation, the six months prior to the survey may have looked dramatically different than before their convictions.

Clearly the non-randomized, compensated reports of 195 men is not even slightly generalizable to Baltimore’s illegal underground gun market. The researchers didn’t let that stop them from arguing that “the results of this study highlight the role of gun laws in creating barriers such as limiting the supply or sources of guns.”

If that were true when this survey was conducted back in 2016, it would follow that all of the gun control laws imposed in Maryland since that date are irrelevant and should be repealed. Seeing how 90 percent reported that they have access to handguns, not rifles, the so-called “assault weapon ban” should be the first to go.

Results

It’s barely worth noting the results of this junk survey. Of the 195 men who agreed to participate in the survey, only 58 had attempted to acquire a firearm in the prior six months. They were asked a series of questions about how they went about their attempts. These were direct questions such as “have you tried to buy a gun from someone who is known for selling guns on the street?” (Yes: 45 percent).

But when it comes to the accusations the survey makes about the ease of obtaining firearms from retailers, it doesn’t even ask whether the participants bought a firearm from a retailer. It asks whether the participant is aware of gun shops in the state that are known to be easier to get guns without background checks (Yes: 24 percent) or whether there are certain gun shop employees that will sell guns “off the books” (Yes: 31 percent).

Awareness of rumors is different from actual “success” in illegally obtaining a firearm from a heavily-regulated retailer. Respondents were also not asked to identify any of these supposedly corrupt licensed retailers.

What really comes out of this survey is a strong connection between drugs and criminal activity. Rather than irrationally targeting the legal purchases of law-abiding Americans, gun control groups and the “researchers” they fund should take a look at addressing how to actually help make communities safer.

The firearm and ammunition industry is in Baltimore and across the country, working with law enforcement and offering resources to help retailers prevent thefts and straw purchasers as well as how the firearm industry encourages the safe storage of all firearms. There are real problems in communities and the firearm industry supports Real Solutions. Let’s stick with the wheels we have, rather than artificially constructed rectangles.

Elizabeth McGuigan is Director of Legislative and Policy Research at the National Shooting Sports Foundation.