The National Shooting Sports Foundation (NSSF) recently released a study report concluding that hunting’s safer than many other popular participation sports. Media Matters was all over it like a cheap suit. The left-leaning matter minders maintained that the NSSF’s methodology was “almost laughably shoddy.” Media Matters’ blog post pinged my radar, mainly because they called us out by name for picking up the story immediately. I’m the guy in charge of tearing apart the often fundamentally flawed firearms studies propagated by The Brady Campaign to Prevent Gun Violence, the Violence Policy Center and others. As TTAG’s resident stats geek, I was inspired to take a closer look at the hunting safety numbers, blogged here as Hunting Is Safer Than Sex. So I contacted the NSSF to get the methodology and data sources from the horse’s mouth . . .
Media Matters’ main argument: the report “mashes” together two incomplete data sources to form the dataset for hunting accidents and that the bad data negates the research. The NSSF did indeed use two different sources to figure out their numbers — they combined information from the Consumer Products Safety Commission and the International Hunter Education Association to get an idea of how many injuries were reported from hunting accidents.
The CPSC was only concerned about injuries from treestands and skeet shooting, but not any other hunting related accidents. The IHEA data was used to fill in the gaps based on a “worst case scenario” approach, taking the maximum number of hunter injuries reported per year by state for the period between 2000 and 2010 or (in cases where data was unavailable) using the average of the total injuries per state.
For example, if there were two hunting accidents per year in a state except for a single year where there were 30, the NSSF would use 30 as the value for that state. If none were reported they’d average the rest of the reporting states and use that number. CPSC and IHEA data was combined (despite probable overlap in the results which led to over reporting) to give a more accurate idea of the total number of hunting accidents per year.
While these two data sources represent the best information available (as the FBI and ATF do not track which firearms incidents are specifically relating to hunting accidents) it still is, in fact, incomplete. The CPSC doesn’t have much information about hunting accidents (two categories for all hunting accidents compared with six categories for accidents involving fireplaces, for example) and the information available to the IHEA is submitted on a voluntary basis by each state’s wildlife management agency (some states don’t report anything).
The use of voluntarily reported data in a report like this does indeed have the ability to under-report hunting accidents—which could lead to a lower level of injury for hunting compared to other sports. (CPSC’s data is culled from hospital records and doesn’t rely on voluntary reporting.)
Also unclear from the press release: how the NSSF treated fatalities in the comparison.
Media Matters points out that the IHEA claims 1,000 people per year are accidentally shot while hunting; a figure that puts the injury rate at about 6.25 for every 100,000 hunters. For comparison, Wikipedia claims (or did when I last checked) that about 1.4 out of every 10,000 people in the United States die in a car accident every year.
Still, despite the low numbers it’s hard to see how the NSSF integrated those fatalities into their report. Don’t get me wrong, I’m sure someone has died from playing chess at least once, but is that death worth five injuries? Or do you exclude the deaths completely? It wasn’t made clear in the press release.
So I asked, and this was the response from Jim Curcuruto, the head honcho for NSSF’s research team (Director of Research & Analysis if you want to be all formal):
The press release focused on total injuries per activity and did not delve into how many injuries resulted in a fatality as I was unable to find a reliable source for fatality data per activity.
In other words, in a report that appears to examine the relative safety of various sports, the NSSF completely glossed over the whole “death” thing.
The NSSF’s press release on the report used the headline “Hunting: One of the safest activities in America.” In my opinion the word “safety” means free from injury OR death, as both outcomes are generally bad for your health. Despite the use of the word “safety” in their press release the NSSF doesn’t look at fatalities at all when they performed their calculations, a decision consciously made due to the lack of data available about sports fatalities. According to Jim the word “safety” is still applicable despite the lack of fatality data, but I don’t agree and think it’s a deceptive use of the word.
Excluding fatal accidents from the dataset automatically skews the result and makes that report relatively meaningless. In my opinion the probability of death from a hunting related accident should be much higher than the probability of death from an injury sustained while golfing or tennis or playing any of the “typical” sports, as not only are the implements of the sport deadly by design but emergency medical help is pretty far away when you’re in the middle of the woods. This would mean that the consequences of an accident were greater, but if the probability of such an accident occurring were small then the result may still be valid. Unfortunately such data was not included in the analysis.
We now know that you’re less likely to wind up in an emergency room while hunting than many other sports, but unplanned refrigerated trips to the morgue are still a mystery. And that right there is the million dollar question the NSSF failed to answer in terms of the “safety” of different sports.
The truth is that the NSSF’s report is incomplete, and the press release based on that report was more deception and propaganda than statistics and facts. The report is headed in the right direction in terms of general methodology, but until we see some concrete numbers regarding deaths related to hunting accidents we can’t draw any conclusions about the relative risks of sporting activities in the way that the NSSF’s press release does.
[EDIT:] Following the publication of this article I was contacted by Mr. Curcuruto from the NSSF who provided additional information and some minor error correction. According to them the press release was based off an “analysis” piece and was NOT a formal “study” as originally reported. I have updated the language of the post to change “study” to “report,” the term used in the NSSF’s original press release, in an effort to be as accurate as possible in our analysis.