Polygraphs: The evidence

[CFLarsen]

In this thread about how polygraphs purportedly revealed that over 80% of atheists really believe in God, skeptigirl had this to say about the validity of polygraphs:

Just to dispense with the polygraph discussion, (though I doubt it will, perhaps if not the mods will see fit to split the thread), it doesn't matter if the polygraph is not good enough for court, because the majority of the answers would be correctly determined. Therefore the claim ALL 100 atheists were lying or all the atheists in the above story were lying and really believe in gods would not be entirely discredited on the basis of the polygraph reliability alone.


Here's a very thorough evaluation of polygraph testing. THE POLYGRAPH AND LIE DETECTION; Committee to Review the Scientific Evidence on the Polygraph; Board on Behavioral, Cognitive, and Sensory Sciences; and Committee on National Statistics; Division of Behavioral and Social Sciences and Education; NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES and on this page is a graph of results showing

FIGURE 5-1 Sensitivity and false positive rates in 52 laboratory datasets on polygraph validity.

NOTES: Points connected by lines come from the same dataset. The two curves are symmetrical receiver operating characteristic (ROC) curves with accuracy index (A) values of 0.81 and 0.91.

involving naïve examinees untrained in countermeasures: for such examinees and test contexts, the polygraph has an accuracy greater than chance. Random variation and biases in study design are highly implausible explanations for these results, and no formal integrative hypothesis test seems necessary to demonstrate this point.

Second, with few exceptions, the points fall well below the upper left-hand corner of the figure indicative of perfect accuracy. No formal hypothesis test is needed or appropriate to demonstrate that errors are not infrequent in polygraph testing.

The entire report is available at the link.

If a polygraph is wrong 2% or 5% or 10% of the time one can conclude out of 100 answers the truthfulness of 98 or 95 or 90 respectively will be correctly determined to be truthful or not. Since the OP is talking about a polygraph determining that 80 atheists were lying, then were it true, which there is no evidence it is, then the majority of those atheists would indeed be lying.

How the question was phrased would also need to be taken into consideration before drawing the lie-for-Jesus conclusion implied in the urban myth as well.

The report's conclusion says:

We have reviewed the scientific evidence on the polygraph with the goal of assessing its validity for security uses, especially those involving the screening of substantial numbers of government employees. Overall, the evidence is scanty and scientifically weak. Our conclusions are necessarily based on the far from satisfactory body of evidence on polygraph accuracy, as well as basic knowledge about the physiological responses the polygraph measures. We separately present our conclusions about scientific knowledge on the validity of polygraph and other techniques of detecting deception, about policy for employee security screening in the context of the U.S. Department of Energy (DOE) laboratories, and about the future of detection and deterrence of deception, including a recommendation for research.
Source

skeptigirl,

Please present your evidence that polygraphs are able to tell when you lie or not.

 

[Skeptic Ginger]

What is that link if not evidence? I don't understand your problem. It is an in depth detailed analysis of the polygraph.

Holy cow just looking at the title should tell you that:

THE POLYGRAPH AND LIE DETECTION; Committee to Review the Scientific Evidence on the Polygraph; Board on Behavioral, Cognitive, and Sensory Sciences; and Committee on National Statistics; Division of Behavioral and Social Sciences and Education; NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES

What more do you want?

 

[CFLarsen]

Well, you can start with explained what you should have read instead: Namely, the conclusion.

How is that evidence that polygraphs are able to tell when you lie or not?

 

[Skeptic Ginger]

The discussion was about the results of 100 polygraphs. You cannot tell which of the hundred results are wrong but you can make a general statement that the majority of the results are not wrong.

It's called sensitivity and specificity. I use it every day interpreting lab results. Some results are erroneous but I can determine with whatever degree of confidence the specificity and sensitivity is that the results from my patient's lab tests are reliable.

 

[Lonewulf]

This should be fun...

 

[Skeptic Ginger]

Hypothetically, say you asked 100 theists if they really believed in god and they all answered yes and the results showed they were all lying. If the polygraph gives valid results 80% of the time (without trying to fool it the report said the results were valid close to 100% of the time but say it was only 80%), then 80 theists would be lying. You could not tell which 80 were lying and which 20 were not lying but you could conclude 80 were lying based on the average validity of the polygraph results.

I really don't understand why the two of you don't understand the scientific concept of sensitivity and specificity.

 

[Lonewulf]

Hypothetically, say you asked 100 theists if they really believed in god and they all answered yes and the results showed they were all lying. If the polygraph gives valid results 80% of the time (without trying to fool it the report said the results were valid close to 100% of the time but say it was only 80%), then 80 theists would be lying. You could not tell which 80 were lying and which 20 were not lying but you could conclude 80 were lying based on the average validity of the polygraph results.

I really don't understand why the two of you don't understand the concept of sensitivity and specificity.

I don't get why you don't understand the simple concept of interpretation that people brought up, but whatever.

 

[Skeptic Ginger]

What concept is that, Lonew? I really don't have a clue what you are pissed about or what you are talking about here. Did you see my post about the off topic matter in the dungeon banished thread?

 

[CFLarsen]

The discussion was about the results of 100 polygraphs. You cannot tell which of the hundred results are wrong but you can make a general statement that the majority of the results are not wrong.

It's called sensitivity and specificity. I use it every day interpreting lab results. Some results are erroneous but I can determine with whatever degree of confidence the specificity and sensitivity is that the results from my patient's lab tests are reliable.

Hypothetically, say you asked 100 theists if they really believed in god and they all answered yes and the results showed they were all lying. If the polygraph gives valid results 80% of the time (without trying to fool it the report said the results were valid close to 100% of the time

Hello??

The whole idea of the polygraph is to detect if people try to fool it.

but say it was only 80%), then 80 theists would be lying. You could not tell which 80 were lying and which 20 were not lying but you could conclude 80 were lying based on the average validity of the polygraph results.

I really don't understand why the two of you don't understand the scientific concept of sensitivity and specificity.

But we are not talking about this concept. We are talking about whether or not polygraphs are able to tell when you lie or not.

You claim that they are. The conclusion of the study you pointed to clearly says the opposite of what you claim it says.

So, where is the evidence?

 

[Lonewulf]

What concept is that, Lonew?

You weren't reading in the thread it was discussed in? Thought not.

How does a lie detector work? Do you know? It monitors your body to see if it reacts to a question.

It can easily misread certain signs that could indicate something other than lying. Someone reading the polygraph would have to "translate" the body signs.

In short, it's subject to more than just simple machine error. It's also subject to human error.

Furthermore, there are ways for trained individuals to bypass a polygraph.

Could this or could this not affect your hypothesis that anytime the polygraph is used, it will always allow 80 out of 100 people, 100% of the time, be caught in lying? The polygraph is not a mind-reader. It's a body-reader. And some can control their bodies better than others, and some can read bodies worse than others. It's as simple as that.

Did you see my post about the off topic matter in the dungeon banished thread?

Now who's getting off topic?

 

[Skeptic Ginger]

I cannot reply to either of you because I don't for the life of me get your issues.

Are you talking about testing one person or the average results of 100 tests?

Hello.

 

[Lonewulf]

Lol.

Okay, I guess there isn't anything to discuss then.

Good day.

 

[Skeptic Ginger]

From the cited source:

Estimate of Accuracy Notwithstanding the limitations of the quality of the empirical research and the limited ability to generalize to real-world settings, we conclude that in populations of examinees such as those represented in the polygraph research literature, untrained in countermeasures, specific-incident polygraph tests for event-specific investigations can discriminate lying from truth telling at rates well above chance, though well below perfection.

 

[CFLarsen]

I cannot reply to either of you because I don't for the life of me get your issues.

Are you talking about testing one person or the average results of 100 tests?

Hello.

What difference does it make?

If you can't tell if one person lies or not, how can you say anything about how the polygraph does on average?

You can't!

That's why your explanation isn't valid. You can't point to an average performance, unless you can tell if it works at all.

 

[CFLarsen]

From the cited source:

Okie doke.

Are you willing to present a paper at TAM, where you present the scientific evidence that polygraphs work?

 

[Skeptic Ginger]

Claus, why would I bother? However, if you have an argument with the logic, then spell it out. If you test 100 people and the results are expected to be correct 80% of the time then you can draw a conclusion about the group that you cannot draw about an individual.

The idea a test must be 100% accurate to be useful will be news to the medical community. Care to present your arguments why we should not use a test if it isn't 100% accurate?

 

[CFLarsen]

Claus, why would I bother?

Are you serious??

You believe you have scientific evidence that polygraphs work. We cannot let that opportunity pass us by, as skeptics. That's precisely what the meetings are for: To examine such claims and evaluate the evidence.

You think you can attend the Amazing Meeting - the biggest conference for skepticism and critical thinking in the world - and not be challenged because of your claim that polygraphs work?

What do you think being a skeptic is?

However, if you have an argument with the logic, then spell it out. If you test 100 people and the results are expected to be correct 80% of the time then you can draw a conclusion about the group that you cannot draw about an individual.

How do you know it is correct any of the time?

The idea a test must be 100% accurate to be useful will be news to the medical community. Care to present your arguments why we should not use a test if it isn't 100% accurate?

Whoa, whoa. Who said anything about 100%?

 

[Skeptic Ginger]

For pete's sake, this is a really annoying discussion.

What percent accuracy do you want? What is your estimate? What's your evidence?

Take that then apply it to the group of 100 tests. What do you get?

 

[Lonewulf]

From Wikipedia, here: http://en.wikipedia.org/wiki/Polygraph_test#2003_National_Academy_of_Sciences_Report

The accuracy of the polygraph has been contested almost since the introduction of the device. In 2003, the National Academy of Sciences (NAS) issued a report entitled “The Polygraph and Lie Detection”. The NAS found that the majority of polygraph research was of low quality. After culling through the numerous studies of the accuracy of polygraph detection the NAS identified 57 that had “sufficient scientific rigor”. These studies concluded that a polygraph test regarding a specific incident can discern the truth at “a level greater than chance, yet short of perfection”. The report also concluded that this level of accuracy was probably overstated and the levels of accuracy shown in these studies "are almost certainly higher than actual polygraph accuracy of specific-incident testing in the field.” [1]

The report: http://www.nap.edu/openbook.php?isbn=0309084369

When polygraphs are used as a screening tool (in national security matters and for law enforcement agencies for example) the level of accuracy drops to such a level that “Its accuracy in distinguishing actual or potential security violators from innocent test takers is insufficient to justify reliance on its use in employee security screening in federal agencies.” In fact, the NAS extrapolated that if the test were sensitive enough to detect 80% of spies (a level of accuracy which it did not assume), in a hypothetical polygraph screening of 10,000 employees including 10 spies, 8 spies and 1,598 non-spies would fail the test. Thus, roughly 99.6 percent of positives (those failing the test) would be false positives. The NAS concluded that the polygraph “…may have some utility” [2] but that there is "little basis for the expectation that a polygraph test could have extremely high accuracy."[3]

The NAS conclusions paralleled those of the earlier United States Congress Office of Technology Assessment report "Scientific Validity of Polygraph Testing: A Research Review and Evaluation”.[4]

Certainly nothing that I would want to use in any scientific study.

 

[Skeptic Ginger]

But this discussion was never about using polygraphs to weed out spies. It was about drawing a conclusion about 100 tests grouped together.