DUI expert witness: Accuracy of the NHTSA's Standardized Field Sobriety Test SFST FST

This web site gives the opinions of Dr. Greg Kane. Everything you read here is expressed only as my personal opinion.

In NHTSA's San Diego SFST validation study for six of seven DUI officers, every single driver who was able to take the SFST failed the SFST [BAC 0.04]. NHTSA reported those results this way, "Officers’ estimates ...were accurate in 94 percent of the decisions to arrest."

Home

Data

Accuracy

They Changed
The Answers

Secret Evidence

Threats?

On this page:

All you need to know
Comparing SFST theory with SFST fact
Links to Greg's published articles on SFST accuracy

Attorney, call now to schedule a FREE confidential expert review of your client's records.

SFSTs are not perfect. They make mistakes. NHTSA validation projects prove SFSTs make mistakes when they test innocent people. That's a bad thing. Police officers are trained to rely on SFSTs. Every time an SFST makes a mistake on an innocent person, and the officer follows his training, an innocent person is arrested. And when the jury believes what it is told, that the SFST is a valid reliable scientific test, the person is convicted. Wrongly convicted. Falsely convicted. By a test that made a mistake.

Courts have decided to rely on SFSTs, on tests that cause false convictions. Courts have decided to convict innocent people. Cost of doing business.

How often are SFSTs wrong?

AROUND the country DUI defense attorneys share strategies to overcome field sobriety test evidence. Prosecutors do the same, from the other direction. The sides bicker about mechanics. Did the officer follow procedure e x a c t l y? Did the officer rule out medical conditions that cause incoordination?

What no one does is doubt NHTSA's theory about what a meticulously administered coordination test actually implies about alcohol impairment. The driver failed the SFST. No one asks, "Exactly what does that mean?"

FieldSobrietyTest.info answers the question: What is the probability that a driver who failed an SFST—a meticulously administered SFST—was in fact impaired. The answer will surprise you.

The claimed "extreme accuracy" of the SFST is a statistical trick. NHTSA hides the trick by hiding the raw data. The raw data prove SFSTs are extremely inaccurate—so inaccurate that officers in NHTSA's own validation studies simply ignore the test's results.

Easy accuracy
This web site is all about SFSTs, but not the way you maybe think. There's nothing here about how SFSTs are done. I'm not going to show you how to prove D's SFST doesn't count because the night was too dark, or too cold, or the AO waved the HGN pen too fast.This site isn't about how SFST measurements are made.

This site is about how SFST measurements are interpreted. What they mean. What they tell us about the person who took the test. This site is about the accuracy of the SFST.

Understanding accuracy, scientific accuracy, is harder than you probably think. So let's start easy, with the accuracy the security metal detector.

Some bright guy has noticed that everyone who has a gun makes the metal detector beep. Hey, this thing isn't a metal detector, it's a GUN detector! Let's arrest anyone who makes the GUN detector beep.

Metal detector = Gun detector theory	Coordination test = SFST theory
The fundamental theorem of Gun Detectors Everyone with a gun sets off the metal detector	The fundamental theorem of SFSTs Everyone who is drunk is uncoordinated
That man set off the metal detector, therefore he has a gun. Arrest him.	That man is uncoordinated, therefore he is drunk. Arrest him.
Measures: metal Theory: metal = gun	Measures: coordination Theory: uncoordinated = high alcohol

Everyone with a gun sets off the metal detector, but most people who set off the metal detector do not have a gun.

When travelers set off a metal detector, we don't arrest and convict them of gun smuggling and terrorism. Everyone with a gun sets off the metal detector, but most people who set off the metal detector do not have a gun.

Metal detectors are highly accurate at spotting people who do have a gun, but they are not especially good at telling the difference between gun and no-gun. On innocent people metal detectors often give the wrong answer. They ring for keys and phones and belt buckles, for lots of stuff.

Everybody with a high BAC is uncoordinated, but most people who are uncoordinated do not have a high BAC.

SFSTs work the same way. It is true that everyone who is drunk is uncoordinated. But it is not true that everyone who is uncoordinated is drunk. If you arrest people for being uncoordinated, you'll end up arresting, and convicting, lots of people who are innocent.

That's it. That's pretty much all you need to know to know why SFSTs don't work. The rest of this web site is for people who are gung ho. You know who you are.

Let's compare SFST theory with scientific fact. When you are called on to interpret a "failed" result from a test that gives the correct answer, "failed", on guilty people but the same answer—also "failed"—on many innocent people, the question you have to answer is, "Did this person "fail" because they're really guilty, or because they're just one of the innocent people the test misclassifies?" *When a test frequently gives the wrong answer on innocent people, the likelihood is very high that any one specific failed* result came from a person who is innocent.** Common sense and a few pictures show how this works.
NHTSA's SFST theory SFSTs do not measure alcohol level and (NHTSA validation studies are clear) they do not measure impairment. SFSTs measure coordination. The officer measures the driver's coordination score. NHTSA's SFST theory holds that each driver's coordination score points back to that driver's BAC. Knowing coordination score is the same as knowing BAC. For everyone. In theory.
More SFST theory Innocent people are blue circles. High BAC people are red circles. The officer measures the driver's coordination score. People with a coordination score above an arbitrary threshold fail the SFST and in this diagram get a black spot. People with a low coordination score pass the SFST and don't get a spot. According to SFST theory, innocent people will pass (blue- no spot), but high BAC people will fail (red- spot). According to NHTSA's SFST theory knowing whether the person has a spot tells you what color their circle is. Theoretically.
Scientific fact Official NHTSA SFST validation study data proves that SFST theory doesn't line up with scientific fact. NHTSA's own data proves that at BAC 0.05% (the per se limit in Colorado) 63% of innocent people fail the SFST! NHTSA's own science proves that knowing whether the person has a spot does NOT tell you what color their circle is.
Amazing fact On the road, most drivers are innocent (blue circles). 2/3 of these innocent drivers will have a high coordination score (a spot) and be detained. A few drivers do have high BACs (red circle). Most of these will fail an SFST. Officers will release drivers with no spots, and detain those with spots. But look! Most circles with spots are blue, because most circles were blue to start with. If juries depend on the SFST to decide guilt, about 86% of convictions will be false conviction of innocent people.
This is not a trick. If the SFST identified innocence accurately, the false conviction rate would be 7%

Criticize NHTSA science, get sued

Read this web site while you can. Best I can tell we live in a country where, if you criticize government science, the agency will look the other way while it's contractor tries to intimidate you into shutting up.
Best I can tell, I am under ongoing threat from the NHTSA's contract scientist Dr. Jack Stuster for exposing the scientific errors you are reading about here.

I'm about to say unflattering things about NHTSA science. This web site is about science—the science of the NHTSA's SFST validation theory. I do not know, I do not care, I do not have an opinion about Dr. Jack Stuster's knowledge or intentions at any time ever in his life. I'm not even saying he had knowledge or intentions. But if he did, this web site isn't about them. Or him. This web site is not about Jack Stuster, PhD, CPE. This web site is about the flawed science in the NHTSA's SFST validation theory and supporting studies—and in every courtroom where SFSTs are admitted as evidence.

If you're an expert critical of the SFST and you've been threatened or intimidated by NHTSA or it's associates, let me know. I'm putting together a list.

In depth

These papers first appeared in Trial Talk, the journal of the Colorado Trial Lawyer's Association.

Crying Wolf
What never before published data prove about field sobriety tests.

Never before NHTSA SFST validation study data prove that to a first approximation the SFST works this way: the test says everyone is guilty; the officer ignores the test and arrests or releases people according to his unstandardized gut instincts.

NHTSA science proves that for six of seven highly experienced DUI patrol officers, every single driver who is able to take the SFST fails the SFST.

Trial Talk, August/September 2008, pg 35 ff

Percentages of Percentages, Why validation Studies Fail to Validate

From the article:

Simply by adjusting the balance of impaired and sober drivers in the study group, the NHTSA can dial in the accuracy its research "discovers."

For example, in the Colorado Validation Study NHTSA contractors chose study drivers in a way that led to the discovery of an officer arrest accuracy of 93%. But, keeping the fundamental accuracies exactly the same and simply reversing the percentage of impaired and innocent drivers chosen to be studied would have changed the officer arrest accuracy to 52% - a coin toss.

Trial Talk, August/September 2006, pg 33 ff
347 K pdf opens in a new window.

Field Sobriety Tests: How Basic Science Proves They Have Little Power to Tell Impaired From Sober

From the article:

Look at the results for the Colorado Validation Study. Drivers with a 1% chance of impairment, when they failed the Colorado Validation Study roadside sobriety test, still had only a 4% chance of being impaired. A driver with a 5% likelihood of impairment, after an “arrest” answer, had only a 16% chance of being impaired. All across the table, the FST added only a few percentage points to the probability of guilt - never enough to turn unlikely into likely.

Trial Talk, October/November 2006, pg 57 ff
255 K pdf opens in a new window

Two Statistical Tricks Let NHTSA Contractors Validate Any FST as "Extremely Accurate"

From the article:

Two tricks fix validation studies so any FST can come off "extremely accurate."
Trick One: Move the FST's inaccuracy over onto the innocent drivers.
Trick Two: Hide the innocent drivers.

....Please realize, all Trick Two does is hide the Innocent Driver Inaccuracy. It hides the inaccuracy by hiding innocent drivers from the "overall accuracy" calculation. Trick Two does not fix the Innocent Driver Inaccuracy. According to the NHTSA, officers using standardized FSTs still arrest 29% of the innocent drivers they assess.

San Diego validation study author Dr. Stuster responds:
Only three drivers were arrested during the study who had BACs below 0.08: one was under the influence of drugs (BAC=0.0), one was too impaired to drive at 0.07, and one was 18 years old with a BAC of 0.07 (in a zero-tolerance state). Not a single “innocent driver” was arrested. How can you claim otherwise?

Read Greg's reply.

....The results of reversing Trick Two are in line 9 [of table 1]. As soon as you reverse Trick Two, in study after study, the "overall accuracy" of the roadside test plummets to a number close to the accuracy of a coin toss. The two statistical tricks are the direct cause of the high accuracies "discovered" in NHTSA validation studies. But for the two tricks, FST validation studies fail to validate.

Trial Talk, April/May 2007 pg 33 ff
288 K pdf opens in a new window.