In the 1998 San Diego "Validation" study at the 0.04% BAC level, 99% of the drivers given an SFST failed the SFST. The accuracy of the SFST on innocent drivers was 7%. That's right, seven percent.

Scientific Accuracy

This web site gives the opinions of Dr. Greg Kane. Everything you read here is expressed only as my personal opinion.

HERE'S AN OVERVIEW of what the San Diego FST validation study's raw data actually shows.

The FST study does not study FSTs
Study officers did FSTs, and gathered FST results data, but they did not rely on FSTs to make their BAC estimates. The statistic the study reports and "validates" is the officers' BAC estimate. This accuracy number is different from the accuracy of the FST. Here's how the study was done:

The San Diego study is not alone. All FST validation studies keep the accuracy of the SFST secret.

 

Officers ignore FST results.
In the San Diego study thirteen drivers failed the HGN test and passed both the OLS and WAT tests. These are their SFST results, and the officer's estimate of each drivers BAC.

Notice these thirteen drivers had identical SFST scores. According to the standardized FST interpretation criteria, each driver should have had a BAC estimate of ">=0.08". Instead, officers came up with nine different BAC estimates.

What's more, instead of the SFST's standardized BAC estimates— "<0.08" or ">=0.08"—officers were somehow able to estimate BAC levels to 1 part in 100. There were then and are now no standardized FST interpretation criteria for estimating BAC to 1 part in 100.

Officers did not—could not had they wanted to—rely on these identical SFST scores to come up with their nuanced, 1 part in 100, BAC scores. What's more, the officers somehow knew almost exactly which SFST results to throw out. All these drivers failed the SFST. Yet officers estimated that six them had BACs in the legal range—flatly contradicting the SFST. Five of those six in-the-legal-range BAC estimates were correct. How'd officers do that? How did officers know exactly which SFSTs to ignore?

FST highly inaccurate
We've seen these contingency tables (aka "decision matrixes") before. In their official reports NHTSA contractors give matrices for officers' BAC estimates and arrest decisions and, rarely, for isolated components of the SFST (HGN, WAT, OLS). No FST "validation" study has ever released the decision matrix of the SFST itself. Here it is.

296 drivers took the SFST
292 failed—99%.
4 passed— 1 %
On innocent people the accuracy is 7%!

If juries rely on the SFST to decide the guilt of drivers charged with DWAI at the current 0.05% level, they will wrongly convict 93% of the innocent drivers who go to trial.


Here's the decision matrix for the SFST at 0.08% BAC:

At 0.08% BAC, on innocent people the SFST is only 29% accurate! That's worse than a coin toss.

If juries rely on the standardized field sobriety test to decide the guilt of drivers charged with DUI at the 0.08% level, they will falsely convict 71% of the innocent drivers who go to trial.

The Predictive Value of the Standardized Field Sobriety Test — there is none
The in depth articles show you [Field Sobriety Tests: How Basic Science Proves They Have Little Power to Tell Impaired From Sober, page 58 ff] how science calculates the implications of a failed test using a statistic called the Positive Predictive Value. Turns out scientific tests don't really give absolute probabilities, what they do is move the probability from one number to another—a subtle point that makes a big difference. A failed FST makes a impairment more likely. A passed FST makes impairment less likely. The question is, how far does the FST move the probability?

The answer is given by the scientific formula for the PPV [see Field Sobriety Tests: How Basic Science Proves They Have Little Power to Tell Impaired From Sober, page 58]. The PPV formula uses the FSTs pretest probability of impairment, and calculates a new, updated post-test probability of impairment. The PPV reflects 1) the pretest probability of impairment, plus 2) the increment of certainty added by the FST. Here are the results:

For the FST using the standardized criteria for identifying impairment at 0.04% BAC

When a driver's pretest probability of impairment is 20%, a failed FST indicates a post-test probability of impairment of 21%. The FST moved the probability 1%. 50% becomes 52%. 70% becomes 71%. None of these changes are greater than the margin of error in the pretest probabilities themselves. The FST makes no difference. None. Within the margin of error, the FST does not move the probability at all.

In English
This is all science's precise way of saying the obvious. On innocent people, the FST gives the wrong answer 93% of the time. A test can't be wrong that often and give meaningful results. When an FST says "impaired," you can't tell whether the test is positive because the person is really impaired, or whether this is just one of those 93% of innocent people for whom the FST gives the wrong answer.

So, how do NHTSA contractors prove the SFST is "extremely accurate"?

1

They report officers' decisions instead of SFST results

2

Validation study officers violate SFST interpretation criteria and, using undisclosed non-standardized criteria, release 60% of the innocent drivers who fail the SFST

3

They study a skewed sample—they load up their study groups with drunks. That inflates the accuracy they "discover" compared with the accuracy in the general population.

 

Here are PPV results using SFST criteria for identifying impairment at 0.08% BAC

Again, no meaningful change in the probability of impairment.

 

The science has been done. The science proves FSTs do not work.

top