<< Previous

Secret evidence
If you think Americans charged with crimes should be able to see the evidence against them, SFSTs disappoint. SFST accuracy evidence is secret. The government makes extravagant scientifically invalid claims about the accuracy of the SFST, but it has never published and now refuses to release the data on which those claims are supposedly based.

This sounds nutty. We have the scientific reports from NTHSA's own scientific SFST validation studies. They report the scientific accuracy of the scientific SFST. It's true they give summary statistics and not their raw data, but that's exactly what scientific papers generally do. We have as much scientific information about SFSTs as we do about any other scientific medical diagnostic test.

Unfortunately, it ain't so. SFST's are "validated" by the same bureaucracy that promotes them, sort of like abolishing the EPA and having coal companies validate strip mining. Over and over NHTSA's SFST "science" fails to meet the minimum standards of science.

Compare NHTSA standards with the standards of free and open science.

• Studies arranged by NHTSA
• Research contractors picked by NHTSA
• Research contractors paid by NHTSA
• No financial disclosure.
• Research results published by NHTSA in-house, without outside peer review.
• No review by an independent biostatistician. As far as I can tell, no review by any statistician.

In the USA governments convict people of DUI crimes using pseudo-medical "science" based on secret data that was not originally subject to outside peer review and cannot now be examined by the defendant—or anyone else.

Journal of the American Medical Association

"For industry-sponsored studies, an analysis of the data (based on the entire data set and evaluation of the study protocol, and prespecified plan for data analysis) must be conducted by an independent statistician at an academic institution, rather than by statisticians employed by the sponsor or by a commercial contract research organization. ....Details of this independent statistical analysis, the name and institutional affiliation of the independent statistician, and whether compensation or funding was received for conducting the analyses should be reported in the Acknowledgment section of the manuscript. The results of this independent statistical analysis should be the results reported in the manuscript."

Journal of the American Medical Association
Instructions For Authors, 2008, pg 2-3;
also in JAMA, July 2, 2008-Vol 300, No. 1

All data

A FOIA request to the NHTSA failed to get any FST study data. So I called Washington, DC and spoke with the NHTSA's Chief Counsel's Office—the agency's self described "FOIA experts." Agency lawyers claim all the FST data was lost "when we moved to this new office." The data was also lost, "when we moved to a paperless office." This happened, "a few years ago"—some time after a similar FOIA request was honored, resulting in Dr. Hlastala's 2005 scientific article sharply critical of NHTSA's statistical methods.

All data

Southern California Research Institute, as I read validation reports, contracted with the NHTSA to do the Florida, Colorado, 1981, and 1977 studies. SCRI data seems to be secret. SCRI's executive director, Dr. Dary Fiorentino, answered my emails, but refused to release any data. At least I think he did—I was never able to decode his responses to tell whether the data has been lost, or destroyed, or whether it still exists but is secret.

SCRI's executive director even refused to release the "case log" appendix originally included in the Florida study's official final report (a study he worked on), although again his apparent evasiveness left me unsure if the official report SCRI was paid hundreds of thousands of dollars to prepare has been lost, destroyed, or is secret.

All data

Dr. Marcy Burns, principal author of the San Diego, Colorado, Florida, 1981, 1977 studies, refused to release any data. The only response I've yet gotten from her is the green USPS certified mail receipt she signed when my request was delivered