It’s the holy grail of gun control activists: proof that more guns equals more crime. Quantitative evidence that the mere existence of firearms increases crime, therefore justifying the disarmament of Americans. A study claiming to have reached just that conclusion came out last week, and naturally all of the usual suspects are jumping all over it. The study, produced by Stanford University, may make some interesting claims but even the press release touting it notes that it just might all be a wild-ass guess . . .
From the presser:
The murder rate increased in the states with existing right-to-carry laws for the period 1999-2010 when the “confounding influence” of the crack cocaine epidemic is controlled for. The study found that homicides increased in eight states that adopted right-to-carry laws during 1999-2010.
“Different statistical models can yield different estimated effects, and our ability to ascertain the best model is imperfect,” Donohue said, describing this as the most surprising aspect of the study.
He said that many scholars struggle with the issue of methodology in researching the effects of right-to-carry laws. But overall, his study benefits from the recent data.
Donohue suggested it is worth exploring other methodological approaches as well. “Sensitive results and anomalies – such as the occasional estimates that right-to-carry laws lead to higher rates of property crime – have plagued this inquiry for over a decade,” he said.
The problem is that the study appears to use a complex model to “control” for other factors, like the “crack cocaine epidemic.” But when you are looking at a complex system like crime in the United States, controlling for different variables is extremely hard and often impossible. That’s why they went ahead and hedged their own announcement, making sure to mention that their model might be imperfect.
In fact, in the study itself, the authors admit that they only have a 10% confidence in the validity of the outcome:
Across the basic seven Index I crime categories, the strongest evidence of a statistically significant effect would be for aggravated assault, with 11 of 28 estimates suggesting that RTC laws increase this crime at the .10 confidence level. An omitted variable bias test on our preferred Table 8a results suggests that our estimated 8 percent increase in aggravated assaults from RTC laws may understate the true harmful impact of RTC laws on aggravated assault, which may explain why this finding is only significant at the .10 level in many of our models.
Here’s a little tip: if the strongest indication from the study you’re performing indicates that the results you want to see (guns = crime) is only present in less than 50% of your results (11 of 28 estimates above), then you might not want to trumpet that result from the rafters.
The reason HuffPo is crying Eureka! is that studies “proving” that guns result in more crime are about as rare as studies “proving” that climate change doesn’t exist. Both are hot-button issues based on complex systems and long term trends, and neither are particularly convincing to scientists and statisticians in general. In this case, the researchers themselves seem to be heavily couching their findings due to the extremely tenuous level of confidence.
So I only needed one chart to debunk the results of the Stanford study:
I’m not sure how the authors (or the breathless journalists reporting it) can claim an increase in violent crime when violent crime in the United States has been dropping for two decades at the same time firearm sales are at historic highs. I take that back — I know exactly how they can do that. If you know the results you want and the study is sufficiently long, you can make it say whatever you want. Even if you need to drop the confidence interval to hilariously low levels and ignore half of your results.