Polygraphs: The evidence

[drkitten]

Does a polygraph work if the examiner cannot see the subject?

Studies have shown so, yes. From the FAS report:

The experimenters also used objective measurements of all physiological response measures with the aid of computers and persons who had no knowledge of the field evaluations or treatments administered. The experimenters used the decisions made by the independent blind evaluator to assess the validity of the polygraph test. This was, however, equivalent to using the polygraph examiner’s decision, because the independent rater and the examiner agreed on 100 percent of their decisions.

 

[Aoidoi]

Well, one standard way is to have as much of the data processed by computer (objectively) as possible. If the machine shows that on standardized data set alpha-niner-four, it was possible to draw a separating hyperplane between the truthful and non-truthful candidates' measurements, and that my measurement is on the lying side of that hyperplane, it's hard to attribute that to bias. We may argue about whether alpha-niner-four is a representative control group, but there's not much to argue about involving the categorization itself.

Another way is simply to look at the evidence upon which the operator makes his/her decision and see if it appears reasonable. If the operator says "you're lying," that doesn't mean much. But if the operator says "your skin temperature says you're lying," then there should be a visible difference in the graphs, specifically in the one for skin temperature. If the operator gives an opinion, but doesn't have a basis for the opinion, then one could rightly disregard that particular opinion -- but that's hardly unique to polygraphs. This problem has been around in "expert" testimony for centuries. When the doctor says that such-and-such a person died of heart disease and not pneumonia, you expect him to have citable medical reasons for her opinion.

I'm clearly not an expert... but is it possible for other trained polygraphers (er... whatever the word is for that) to go back over the data from a session and come up with the same answers as the operator involved? Expert testimony in court is regularly impeached by other experts based on the data (or in Law & Order based on wild theories), but I'm not sure how practical that is for polygraphs. Seems like one guy can say "look, his heartbeat increased" and the other say "skin temperature remained the same" and a the operator can say "that SOB farted and damn near killed me."

Well, you get the idea, anyway.

How consistent is the interpretation of results? Seems like there's always going to be that bias problem if there's enough data that the operator can just pick one that agrees with him as justification.

I guess my issue is that I don't really see how interpreting polygraph data is any different from, say, a psychologist saying that based on his professional opinion the subject is being truthful or lying. They can point to all sorts of physical cues (not meeting eyes, fidgeting, posture, etc) and claim that this conveys more information, but it is largely dependent on the individual's perceptions.

I suppose basically I'm worried about potential lack of reproducibility, lack of effective documentation, and concern over how one determines expertise.

You don't need to, unless you're suggesting that so many people were molested by priests that they constitute a significant minority of the target population, on the same order of magnitude as the detection threshhold. In some cases, this may be the case (e.g. if you're doing this experiment at a "survivors of priestly molestation" convention). But otherwise, any reasonable estimate of the probability of this would put it down in the "lost in the noise" section of the spectrum.

But during a large scale test you'd have to have some way to determine all the factors involved in the noise. It's unlikely that in a random sample you'll get a whole host of the same reason, but I'm sure there's more reasons to be worried/erratic/freaked out/whatever about something as dramatic as religion than I can come up with off the top of my head. I've seen people do things for religious reasons I can't even explain having seen them, I have no way of determining percentages of populations that will have strange reactions to things.

The first and only time? Possibly not. The thirtieth time over the course of two hours? Quite probably. As I said earlier, the emotional response from being asked essentially the same nosy question over and over by the same person bleaches quite quickly.

I'm rather intrigued by this theory. Seems to me that I know lots of people who would get progressively more upset by being asked the same (or similar) questions over and over. I know I generally get pissed having to repeatedly explain myself on subjects on which I am quite certain I am telling the truth. Heck, just having to repeat myself at family gatherings because half the people aren't paying attention annoys me.

 

[drkitten]

I'm clearly not an expert... but is it possible for other trained polygraphers (er... whatever the word is for that) to go back over the data from a session and come up with the same answers as the operator involved?

See my immediately previous post. It doesn't even need to be "other trained polygraphers."

 

[LTC8K6]

I don't think that meets the requirement, drkitten.

 

[drkitten]

Just a little question, since I am not particularly familiar with the skeptic-specific litterature, have the leading skeptics (or anyone else) performed an experiment that demonstrated that the polygraphs are not better than chance in determining the proportion of liars in a sample population? Is this published somewhere, I would be curious to see what kind of test led them to this conclusion?

I have seen no such far-reaching paper published, although I'm sure that CFL will be happy to provide buckets of them upon demand. Unless, of course, he woudl prefer to admit that he holds his opinion that they don't work based on no supporting evidence whatsoever.

I have seen a number of papers published that indicate that a type X polygraph, under circumstances Y, as operated by Z, is not accurate. But that, of course, isn't an indictment of the entire field, any more than an observation that vitamin C cannot be used to treat cancer means that it's not a good for anything (and for scurvy in particular). In fact, it's very difficult to prove that "polygraphs" as a whole are ineffective for partly the same reason that it's very difficult to prove that astrology is ineffective --- there are still variations of both that have not been subject to testing.

The primary difference is that there are no variants of astrology that reproducibly test positively, and there are many variants of polygraphy that do.

 

[drkitten]

I don't think that meets the requirement, drkitten.

How about this quotation, from the same study?

An objective quantitative analysis for each physiological measure was employed to determine if each was effective in discriminating between guilty and innocent. Most of the measures yielded significant discriminations, with the exception of a few of the cardiovascular measures.

or from another study

Some of the latest work of the Utah laboratory explores the use of computers in the analysis of polygraph recordings. Kircher (91a) compared the accuracy of a computer decisionmaking process to the accuracy of assessments of a field examiner. The computerized analysis cannot be included in the statistical analysis of this technical memorandum, because it is not presently a field scoring method, but the decisions of an independent evaluator who was used can be. This mock crime study followed the basic procedures of Podlesny and Raskin (127) with 100 subjects from the community. The accuracy of the original examiner was not reported though the results of an independent evaluator were. The independent evaluator, who numerically scored the charts blindly, correctly diagnosed 87 percent of the subjects; misdiagnosed 6 percent; and made a judgment of inconclusive on 7 percent. The six errors were evenly divided between three false negatives and three false positives. In comparison, different computer decision models, on the average, correctly identified 84.9 percent of subjects, misidentified 7.85 percent, and placed 7.2 percent in an inconclusive category.

 

[LTC8K6]

It's interesting, but I meant on the fly.

Does the fact that the subject can see the examiner have an effect, for example.

Are you more likely to have a reaction when you can see the examiner looking right at you?

 

[drkitten]

It's interesting, but I meant on the fly.

Does the fact that the subject can see the examiner have an effect, for example.

Are you more likely to have a reaction when you can see the examiner looking right at you?

Ah. That I don't know. I do know that subjects hooked up to "dummy" polygraph machines (machines that they "know" aren't working or are turned off) display less response than subjects hooked up to machines they know are working. So I would suspect the answer is yes, but have no data to support it.

 

[CFLarsen]

Just a little question, since I am not particularly familiar with the skeptic-specific litterature, have the leading skeptics (or anyone else) performed an experiment that demonstrated that the polygraphs are not better than chance in determining the proportion of liars in a sample population? Is this published somewhere, I would be curious to see what kind of test led them to this conclusion?

I'm not sure where you are going with this. Why is that important?

Your statement is vague. Please quote the claim made that you object to.

See post #51.

Does a polygraph work if the examiner cannot see the subject?

The polygraph is a tool that "works" by intimidation: The subject is put under strain from the technobabble and pseudoscience: "We can see that you are lying - so confess!"

Of course, if the polygraph really could tell if people lie or not, there would not be any need for any confession....

Shrug. I've lectured on them, professionally. I've reviewed papers on them. I don't need to have that much knowledge to spot misstatements, errors, and misrepresentations.

In which case your article/presentation would be regarded with much more interest.

What about Skeptic Magazine? TAM?

I'm sorry. I've followed both these threads and finally decided to come in and ask what the hubbub was about. I've stated why I would even bother reading a thread on this subject. I have a certain understanding, and am willing to admit to errors. However, I have no idea what the central contention is.

Is it: polygraphs are not 100% accurate? Polygraphs should never be used? Some have suggested that it's about the efficacy of individual vs. group testing.

Polygraphs are pseudoscience. They cannot be regarded as reliable tools to tell whether people lie or not.

drkitten and skeptigirl in particular claim otherwise.

 

[Cardelitre]

I'm not sure where you are going with this. Why is that important?

Why is that important? What a strange question...

If the leading skeptics as you call them concluded that polygraphs don't perform better than chance to detect the proportion of liars in a sample population, I guess that this is not just wishfull thinking. They must have based their opinion on some experiments demonstrating that point in a way or another, that's what skeptics do right?

At least that's what they do when addressing something like radiesthesia or telepathy or whatever, as far as I know.

So I would be curious to see one of these experiment reports, if there are?

 

[Jimbo07]

Polygraphs are pseudoscience. They cannot be regarded as reliable tools to tell whether people lie or not.

drkitten and skeptigirl in particular claim otherwise.

Thank you. You seem to have summarized it in two points:

i) Polygraphs are pseudoscience: My personal preference would be to say that polygraph interviews are an art. Art can generate reproducable results, and those can be studied... scientifically, I suppose.

ii) They cannot be regarded as reliable tools to tell whether people lie or not: is very difficult, IMO. No piece of evidence is a 100% guarantee that someone is, or is not, lying. However, I think you are being a little bit obstinate in rejecting their claims that there is some effect.

Would you call for a ban on the polygraph interview? Do you think it's a law-enforcement strong-arm tactic? Has the idea of the polygraph as some sort of magic lie detector become a strawman for any objections?

I would agree if everyone was saying, "the polygraph interview is an effective law-enforcement tool." Everything else, I'm regarding as educational!

 

[CFLarsen]

Why is that important? What a strange question...

If the leading skeptics as you call them concluded that polygraphs don't perform better than chance to detect the proportion of liars in a sample population, I guess that this is not just wishfull thinking. They must have based their opinion on some experiments demonstrating that point in a way or another, that's what skeptics do right?

At least that's what they do when addressing something like radiesthesia or telepathy or whatever, as far as I know.

So I would be curious to see one of these experiment reports, if there are?

Of course they base their opinions on experiments.

But why do they have to have performed the experiments themselves?

 

[verisimilidude]

Why is it after reading this thread I am not able to come to any conclusion about the reliability of this device?

Judging from this thread, however, it's obvious that no consensus exists regarding the reliability of these machines. That's probably why their findings are not admissible as evidence in most states unless otherwise agreed to by both parties. And believe me, it doesn't take much validation for any "scientific" technique, method or device to be considered reliable and therefore admissible in court.

Should something which is at best partly woo be considered totally woo?

 

[CFLarsen]

Thank you. You seem to have summarized it in two points:

i) Polygraphs are pseudoscience: My personal preference would be to say that polygraph interviews are an art. Art can generate reproducable results, and those can be studied... scientifically, I suppose.

ii) They cannot be regarded as reliable tools to tell whether people lie or not: is very difficult, IMO. No piece of evidence is a 100% guarantee that someone is, or is not, lying. However, I think you are being a little bit obstinate in rejecting their claims that there is some effect.

Would you call for a ban on the polygraph interview? Do you think it's a law-enforcement strong-arm tactic? Has the idea of the polygraph as some sort of magic lie detector become a strawman for any objections?

I would agree if everyone was saying, "the polygraph interview is an effective law-enforcement tool." Everything else, I'm regarding as educational!

The reason why polygraphs reportedly "work" is not because you can tell from the readings that people lie. You can use polygraphs to intimidate those who don't know that it is bull.

But that doesn't mean they should be used. We don't use pseudoscience, we shouldn't encourage it.

If your suspect was superstitious, would you threaten to cast a curse, if he didn't confess?

 

[Cardelitre]

Of course they base their opinions on experiments.

But why do they have to have performed the experiments themselves?

I didn't particularly mean that they had to do it themselves, maybe I could have been more explicit sorry. They can of course base their opinion on other experiments that already exist.

So what are the experiments that led them to conclude that there is no basis in the principle of polygraphs exactly? Do you know the reference of one that conclusively demonstrates this?

Until now, all I have seen is the meta-analysis of the NAS, which concludes that the results, although very variable depending on the conditions, are still significantly more efficient than chance alone.

 

[CFLarsen]

I didn't particularly mean that they had to do it themselves, maybe I could have been more explicit sorry. They can of course base their opinion on other experiments that already exist.

Gotcha.

So what are the experiments that led them to conclude that there is no basis in the principle of polygraphs exactly? Do you know the reference of one that conclusively demonstrates this?

No single experiment conclusively demonstrates anything. You can start with Bob's article on SkepDic.

Until now, all I have seen is the meta-analysis of the NAS, which concludes that the results, although very variable depending on the conditions, are still significantly more efficient than chance alone.

Not quite. The problem lies precisely in the many different conditions. Some seem to work some of the time under some conditions, at other times not.

You would think that after this many years, we would see some strenghtening of the evidence - some standardization of method that was universally accepted by the scientific community.

We don't. The evidence isn't getting stronger, no standardized method is emerging, and the more we know of the basic premise, the less reason we have to believe it works.

It's quite similar to homeopathy.

 

[Cardelitre]

No single experiment conclusively demonstrates anything.

That's right. So it means that there must be many experiments showing the same negative results I guess, skeptics wouldn’t conclude on only one experiment. Is there one that you think is more representative? I'm afraid I don't have the time to review all the literature on this subject...

You can start with Bob's article on SkepDic.

I found this article on SkepDic, but no reference to any Bob. Is that the one you are talking about?
Anyway it gives an interesting point of view on the polygraph and I agree on many points. But I couldn't find any reference to the type of experiments I was talking about, maybe I missed it. Would you know if this has been done, it must have, right?

Not quite. The problem lies precisely in the many different conditions. Some seem to work some of the time under some conditions, at other times not.

Yes, as I understand it, it works at different degrees depending on the conditions. But it still works significantly better than chance, even if it is not much better.

You would think that after this many years, we would see some strenghtening of the evidence - some standardization of method that was universally accepted by the scientific community.

We don't. The evidence isn't getting stronger, no standardized method is emerging, and the more we know of the basic premise, the less reason we have to believe it works.

The fact that the method is not improved or standardized is not a direct argument against the possible efficiency of the polygraph. It only proves that no one had the will, opportunity or fund to do that, or that the way it is currently used (more as a dissuasive threat) is considered sufficient, or whatever. It is not an evidence against the efficiency if the polygraph.

It's quite similar to homeopathy.

Yes, and those experiments have been conducted with homeopathy countless times and demonstrated that homeopathy is clearly unable to do anything better than placebo.
There should be somewhere a study of the same type for polygraphs right? This would surely help to shed some light on this...

 

[TV's Frank]

Polygraphs are pseudoscience. They cannot be regarded as reliable tools to tell whether people lie or not.

I would just like to ask a clarifying question, that hopefully will improve the discussion. What exactly do you mean by "reliable"? Better than chance? 100% always catching the liar and never generating false positives? Somewhere in between? I think something specific here would be nice.

Also, under what conditions? For example, in this thread, I have seen a discussion of two separate real-world operating conditions: finding a crook in a small sample of suspected bad guys, and sifting through many people in an attempt to ferret out spies. The cited articles suggest that these two scenarios have different rates of "reliability".

 

[CFLarsen]

That's right. So it means that there must be many experiments showing the same negative results I guess, skeptics wouldn’t conclude on only one experiment. Is there one that you think is more representative? I'm afraid I don't have the time to review all the literature on this subject...

No, not particularly.

I found this article on SkepDic, but no reference to any Bob. Is that the one you are talking about?

Bob Carroll. The owner of the site.

Anyway it gives an interesting point of view on the polygraph and I agree on many points. But I couldn't find any reference to the type of experiments I was talking about, maybe I missed it. Would you know if this has been done, it must have, right?

As you can see in the references, Bob refers to e.g. the NAS report.

Bob does his homework.

Yes, as I understand it, it works at different degrees depending on the conditions. But it still works significantly better than chance, even if it is not much better.

The fact that the method is not improved or standardized is not a direct argument against the possible efficiency of the polygraph. It only proves that no one had the will, opportunity or fund to do that, or that the way it is currently used (more as a dissuasive threat) is considered sufficient, or whatever. It is not an evidence against the efficiency if the polygraph.

It isn't as if the polygraph is a fringe technology: It is widely used in many situations in the US, both government and the private sector.

Yes, and those experiments have been conducted with homeopathy countless times and demonstrated that homeopathy is clearly unable to do anything better than placebo.
There should be somewhere a study of the same type for polygraphs right? This would surely help to shed some light on this...

No, not clearly: Some studies show that homeopathy works - only they are small, and/or poorly designed and conducted.

I would just like to ask a clarifying question, that hopefully will improve the discussion. What exactly do you mean by "reliable"? Better than chance? 100% always catching the liar and never generating false positives? Somewhere in between? I think something specific here would be nice.

Also, under what conditions? For example, in this thread, I have seen a discussion of two separate real-world operating conditions: finding a crook in a small sample of suspected bad guys, and sifting through many people in an attempt to ferret out spies. The cited articles suggest that these two scenarios have different rates of "reliability".

That's precisely it: When you find a crook, you find a spy or you test a job applicant, you are talking about the exact same goal: To find out if they lie or not. Yet, we find very different results.

That underlines the unreliability of polygraphs - which is a better way to say it: Polygraphs are unreliable.

 

[TV's Frank]

That's precisely it: When you find a crook, you find a spy or you test a job applicant, you are talking about the exact same goal: To find out if they lie or not. Yet, we find very different results.

That underlines the unreliability of polygraphs - which is a better way to say it: Polygraphs are unreliable.

OK, given all that, what do you mean by "reliable"? Anything better than chance? 100%? What?