This web site gives the opinions of Dr. Greg Kane. Everything you read here is expressed only as my personal opinion.

Accuracy science                                                                              back

Scientifically, "accuracy" is not a property of a test, accuracy is a property of the test AND the group the test is done on.

In everyday conversation we talk about accuracy as if it were a property of a test. You do a test, you get an answer, the "accuracy" tells you the percent chance that answer is correct.

Everyday accuracy looks like this. You do the test, the accuracy, a simple, single number, tells you how often the test's answers are correct.

Probability of the condition tested for →

Turns out, looking at tests this way leads to flagrant logical contradictions. A simple, single, one-number "accuracy" doesn't work.

So in science, "accuracy" is a property of the test AND the group of people the test is done on, and tests don't have accuracies, they move probabilities. Like this...

In this group, a positive test moved the probability from 50% to 90%.

This test was done on a group of people.
50% of the people in the group have the condition tested for.
OK, now everyone has taken the test and we're considering a smaller group, the sub-group of people for whom the test said "Yes." 90% of these people have the condition tested for.
In this group, the accuracy of a positive test is 90%

Here EXACTLY THE SAME TEST is done on a different group of people. 20% of this group had the condition.

In this group, a positive test moved the probability from 20% to 60%.

Same test, different group.
20% of this group have the condition tested for.
In this group, the accuracy of the same test is 60%

"Accuracy" is a property of the test AND the group of people the test is done on.

Notice the big difference
Our everyday idea about test accuracy leads us to expect that any group the test was given to would to end up with the same percentage of correct answers. In real life, it doesn't work that way. Doing exactly the same test on different groups leads to different percentages of correct answers. Formally: the probability that a person with a positive test has the condition tested for depends on the probability they had the condition before the test was done.

Strong Tests
 The good news is many tests are accurate in the everyday sense that their answers are usually right. They're so strong, so good at identifying what they're testing for, that wherever the probability started, a Yes test moves the probability all the way to 100%.

DNA tests, blood chemistries like glucose, sodium, potassium, liver enzyme levels, etc. are like that. It doesn't matter what group the test is done on, if the lab says your blood sugar is 236, then it is highly likely your blood sugar is 236.

In fact the accuracy of even these tests still depends on the test and the group, but the tests themselves are so powerful that, in the mathematics, their strength overwhelms the any changes caused by the group factor.

For strong tests, the probability always ends up close to 100%.



 

In the jargon of science, strong tests are said to have high sensitivity and high specificity.

The good thing about strong tests is, they act as if they have everyday accuracy. It doesn't matter what the pretest probability was, Yes mean Yes and No means No.

Weak Tests
 When a test is only strong enough to move the probability part way to 100%, where the probability ends up is highly dependent on where it started.

For weak tests, the general rule still applies. The accuracy, the percent chance a test answer is correct, depends on the test AND on the pretest probability of the condition. It is scientifically impossible to know what the test implies about the condition unless you know that pretest probability.

For weak tests, where the probability ends up is highly dependent on where it started.


 

In the jargon of science, weak tests are said to have low sensitivity and/or low specificity.
 

Back at the main pages of FieldSobrietyTest.info, you'll learn that the SFST is an extremely weak test. At 0.04% BAC it had no power to move the probability at all! At 0.08% BAC the SFST moves the probability of impairment no more than 8%.