Good Data, Bad Statistical Inferences

Last-line-of-defense-statisticsA February 2015 report from NHTSA’s 2013-2014 survey[1] shows a continued decline in the percentage of drivers under the influence of alcohol. In 1996, 16.9% of those surveyed had a Breath Alcohol Concentration (BrAC) greater than .05 gm/210 L, compared with a mere 8.3% in 2013-2014. One might believe from these statistics that we’re making great progress in reducing the incidence and impact of drunk driving. Wow! We cut the percentage of alcohol-impaired drivers in half!

A closer look at other NHTSA numbers[2] from their Fatality Analysis Reporting System tells a very different story. Although traffic fatalities due to DUI dropped from 13,451 in 1996 to 10,076 in 2013-2014, the percentage of DUI-caused fatalities (>.08 BAC) didn’t budge (32.0% to 30.9%). The drop in DUI-caused fatalities was created almost entirely by the drop in overall fatalities, which went from 42,065 to 32,609.

It’s wonderful to see that safer roads, safer cars, and less driving caused by the severe economic recession resulted in fewer traffic fatalities. But we shouldn’t fool ourselves into believing that we’re making substantial progress in reducing the impact of DUI.  The number of crashes that involve intoxicated drivers is still just as high when compared to the decrease in traffic fatalities due to the economic factors.

The vastly different outcomes of the two NHTSA reports, one based primarily on thousands of coroner’s reports, the other on voluntary roadside surveys, should make us question the basis of quoted statistics.

Another February 2015 report from NHTSA[3] studied over 3,000 drivers involved in crashes compared with 6,000 comparable controls who were not involved in crashes. The report did not find a statistically significant increase in crash risk associated with marijuana use. But failure to find an association is not the same as finding there is no association. Especially with a study that was never designed to detect marijuana’s impact on crash risk in the first place. Perhaps this is why the authors cautioned, “While the findings of this case control study were equivocal with regard to the crash risk associated with drug use by drivers, these results do not indicate that drug use by drivers is risk-free.”

Here is why we claim that the latter study was never designed to detect marijuana’s impact on crash risk:

  • All drivers involved in crashes were not surveyed or tested. Only those drivers who volunteered to participate were surveyed or tested. Perhaps those drivers who were involved in crashes but knew they were stoned declined to participate? See the above example to understand the weakness of voluntary survey-based data.
  • The 3,000 drivers who volunteered to participate were from 2,600 crashes. Yes, that’s right. The drivers included not only those who caused the crash, but innocent victims as well. The mix of culpable drivers vs. victims was not reported. But we know that inclusion of any innocent victims should dilute the effect of any intoxicant in the study results.
  • The study relied on oral fluid testing for all control subjects, whereas drivers involved in crashes used a combination of oral fluid testing and blood testing. Marijuana’s THC is highly concentrated in oral fluid, compared with blood, so oral fluid tests can yield a positive THC result even when the concentration in blood is too low to yield a positive result. We know this is the case because 100% of subjects in this study who tested positive for marijuana’s inactive metabolite, carboxy-THC, also tested positive for the active intoxicant THC, something that never happens with blood analysis. So we are left with the conclusion that the controls had a reported positive THC level higher than what would be expected from blood testing. This would reduce the calculated effect of marijuana’s effect in causing crashes.
  • The study was conducted in Virginia Beach, a military town with a highly supportive police department that has worked diligently to reduce drunk and drugged driving. The Virginia Beach reported results, compared with national results, demonstrate the effectiveness of these efforts. But that also means that Virginia Beach may not be a representative community in which to conduct the study.

Unfortunately, deficiencies in reports such as those described above are sufficiently esoteric that they will be overlooked, thus permitting incorrect inferences by the media and the public.

[1] “Results of the 2013–2014 National Roadside Survey of Alcohol and Drug Use by Drivers,” Amy Berning, Richard Compton, and Kathryn Wochinger, National Highway Traffic Safety Administration, TRAFFIC SAFETY FACTS Research Note, DOT HS 812 118

[2] NHTSA FARS http://www-fars.nhtsa.dot.gov/Trends/TrendsAlcohol.aspx

[3] “Drug and Alcohol Crash Risk,” Richard P. Compton and Amy Berning, National Highway Traffic Safety Administration, TRAFFIC SAFETY FACTS Research Note, DOT HS 812 117

Categories

DUID Articles