Polygraphs have all the characteristics of a woo claim: The theory behind it is very weak, the evidence is very weak, there is no standardization, there is no scientific progress, there is no accumulated knowledge or accumulated evidence in favor of it.
Please provide the scientific evidence that the error rate in polygraphs is close to zero.
This is what this thread is entirely about.
Calling it a "laboratory trick" is very spot on.
It's an example of pseudoscience, which definitely is a subject skeptics are interested in.
Polygraphs are pseudoscience. They cannot be regarded as reliable tools to tell whether people lie or not.
So:
Cervical smear tests are pseudoscience because they don't
reliably detect all cancers and ignore the healthy condition.
Academic examinations are pseudoscience because they don't
reliably grade candidates into discrete categories of knowledge and ability.
Post mortems are pseudoscience because they don't
reliably distinguish between death by natural and unnatural causes.
Do you begin to see the logical fallacy?
Do you seriously believe that any test (forensic, medical, whatever) that doesn't have false positive and false negative rates close to zero is 'pseudoscience'?
If the evidence really is strong enough (as skeptigirl claims), then we cannot refrain from using it.
And if the scientific evidence showed that polygraphs could tell us who lied and who told the truth, it would be a tremendous tool to fight any kind of deceit, be it when screening government employees, employees in private companies, possible spies or people suspected of crimes.
That's why it is so important for proponents of the polygraph to state their case in a forum of skeptics. If skeptics are not able to distinguish between pseudoscience and real science, who is?
Oddly enough, it seems to be very hard to get proponents of the polygraph to step up and defend their - purportedly scientifically supported - claim that the polygraph can tell if people lie or not.
Throughout this thread you have consistently pointed to the lack of proven
utility of the polygraph in real-world settings as a supposed rebuttal to the
scientific validity of polygraphy as a concept. This is quite illegitimate, and in order to further your understanding of the subject (and the area of screening and testing in general), it is essential to separate the purely scientific questions from considerations of utility and public policy.
It is entirely possible for the scientific basis of the polygraph to be valid but unfortunately such that no possible refinement of the method can improve accuracy to a useful level – that is a common occurrence in many scientific areas.
p122 to 124 of the
National Academies report summarise the results from 52 datasets from laboratory studies (all the studies of sufficient quality from a literature search).
First, the data (and their errors of estimate; see Appendix H,
Figure H-3) clearly fall above the diagonal line, which represents chance accuracy. Thus, we conclude that features of polygraph charts and the judgments made from them are correlated with deception in a variety of controlled situations involving naïve examinees untrained in countermeasures: for such examinees and test contexts, the polygraph has an accuracy greater than chance. Random variation and biases in study design are highly implausible explanations for these results, and no formal integrative hypothesis test seems necessary to demonstrate this point.
Second, with few exceptions, the points fall well below the upper left-hand corner of the figure indicative of perfect accuracy. No formal hypothesis test is needed or appropriate to demonstrate that errors are not infrequent in polygraph testing.
Third, variability of accuracy across studies is high. This variation is likely due to a combination of several factors: “sampling variation,” that is, random fluctuation due to small sample sizes; differences in polygraph performance across testing conditions and populations of subjects; and the varying methodological strengths and weaknesses of these diverse studies. The degree of variation in results is striking.
...
It is important to emphasize that these data and their descriptive statistics represent the accuracy of polygraph tests under controlled laboratory conditions with naïve examinees untrained in countermeasures, when the consequences of being judged deceptive are not serious. We discuss below what accuracy might be under more realistic conditions.
Do you understand what the ROC figure is telling us? (If not, do a search on 'receiver operating'.) Considered solely as a test of whether the claimed effect is real, these results are
extremely convincing. They are also well within the range of accuracy required in many real-world screening (though not diagnostic) tests. (If, for example, a pregnant woman has a routine serum screening test for fetal chromosomal anomaly, then the accuracy of that test is reasonably well represented by the better of the two ROC curves above.)
If Randi etc. are claiming that polygraphy 'does not work' in the
scientific sense - that it cannot be shown to be a real, repeatable effect, independent of subjective, unblinded, post hoc rationalisation - then I am confident that they cannot back up that claim. On what
evidence do they base their belief?
The polygraph is a tool that "works" by intimidation: The subject is put under strain from the technobabble and pseudoscience: "We can see that you are lying - so confess!"
Of course, if the polygraph really could tell if people lie or not, there would not be any need for any confession....
The reason why polygraphs reportedly "work" is not because you can tell from the readings that people lie. You can use polygraphs to intimidate those who don't know that it is bull.
But that doesn't mean they should be used. We don't use pseudoscience, we shouldn't encourage it.
If your suspect was superstitious, would you threaten to cast a curse, if he didn't confess?
Polygraphs are solely methods of intimidation, to be used on the uninitiated.
They are unreliable because they work by intimidation. Similarly, torture doesn't work to extract truthful information either.
The technology isn't gauging whether you lie or not. It intimidates you to become confused so you trip up.
...
Polygraphs "work" because people believe they do.
No, the above tests were performed under laboratory conditions, where the questions were of no real-world significance, subjects had no interest in the results, and testers did not know the answers. The results can have
nothing whatever to do with the factors you suggest. This point is obviously of extreme importance in the context of claims from you (and, taking your word for it, Randi and Shermer) that the effect is non-existent and the
subject matter is pseudoscience.
The evidence isn't getting stronger, no standardized method is emerging, and the more we know of the basic premise, the less reason we have to believe it works.
It's quite similar to homeopathy.
Note that
all the evidence turned up by the report's literature search (though not of the highest quality) indicated a real effect.
Almost the entire body of published data suggests that, in laboratory conditions, polygraph tests give results that are better than chance. How on earth can you (or anyone) suggest that this is in any way comparable to studies of homeopathy? (I would say it's more akin to intelligence testing, but that's another discussion.)
Now, the point is made throughout the report that successful laboratory tests may not translate to successful applications, and in the case of polygraphy there has been a striking lack of progress. The inevitable methodological problems (in particular, inability of the technique to cope with a trained suspect) quite likely cannot be solved. And it is certainly true and relevant that throughout the polygraph's history it has been the subject of wildly exaggerated claims of its efficacy (politically and financially motivated).
However, exaggerated claims, limited discriminatory power of the technique, inadequate theory, flawed studies and lack of proven
utility do
not equal
pseudoscience. Polygraphy may not be particularly useful science, and (from the quality of some of the studies included in the NA report) it may not so far be very well conducted science, but it is most decidedly not pseudoscience. (Even if larger and better studies eventually found the effect to be small and unreliable, and based on a mistaken theory, it would be faulty science, not pseudoscience.) I hope you will take a sceptical look at the evidence and stop propagating this myth.
)
