Polygraphs: The evidence

[Aoidoi]

You know, it wouldn't suprise me if the Polygraph does work better than chance at determining a 50/50 proposition under some circumstances. My main problem with it is that it's practically the opposite of double blind. It all comes down to an operator making the call. How does one eliminate operator bias in this situation?

In your (apparently apocryphal, wish I'd realized that earlier cause I was thinking people were actually supporting that factoid as real) 100 athiest test, would your operator be an athiest or a theist? How could you tell, ask them? They could be lying.

Run multiple runs with multiple operators of different backgrounds and compare results? What if the operator is racist? Or sexist? Or just likes to say every prime number candidate is lying because he can? While I can see a polygraph being a potentially effective tool for interrogation (better than electrodes on the naughty bits, at least), it seems like the entire premise is baised on perfectly rational and unbiased operators. There ain't no such animal.

Operator issues aside, you can certainly draw statistical conclusions from imperfect tests. The problem is I don't believe that polygraphs have a known error rate for beliefs. They don't detect "lies" they detect when someone is stressed about something. As they pointed out in mythbusters, there has to be a reason for there to be stress for it to work, having actually stolen something or having a punishment/reward for fooling the test. How can you seperate out "I'm lying about my belief" from "I was beaten as a child for not going to church" or "I was molested by a priest?"

If you asked 100 people who say they're heterosexual if they're lying about being straight, would you trust the answers? Sexuality, religion, politics... these are things that people get emotional about.

Or I could totally be blowing smoke, guess you'll have to hook me up to a machine to find out. I'd like my polygraph operator to be Natalie Portman, please. You won't get good results, but I'll enjoy it.

 

[skepHick]

I've seen a variant of it that could be called the "gambler's proof": where you ask someone to bet a very large amount of money about the truth/validity of a given issue. Generally, the attacker sets the amount so high such that he can be confident the opposing party either doesn't have that much money available, or won't be willing to risk it on something so (comparably) trivial. It also requires that the validity of the issue in question cannot be determined easily, such that neither party can be certain of the truth prior to accepting the bet.

(Example: "If you're so sure the Patriots will win the Super Bowl, you should be willing to bet your house on it. Let's bet your house against mine, OK?")

In some instances, it can be used effectively to demonstrate the depth of someone's conviction in their stated ideas, but has no bearing whatsoever on determining the validity of the ideas themselves. It is generally a tactic used to attempt to stop discussion of a topic and force a concession to the attacker's viewpoint.

The requests to write a paper and/or present at TAM in this thread are similar.

I interpret it this way, as well. An extreme variant of the old "put up or shut up." It certainly doesn't promote discussion. Someone's willingness, or lack thereof, to write an article or make a presentation at TAM doesn't speak to the merits of their position. This is a discussion forum, for Pete's sake. There are acceptable levels of evidence that one could provide here that are well below doing either of these things. Especially the TAM presentation. Even if someone is willing and able to go to that extreme, and even if their position is deemed valid, that doesn't mean it will be accepted as a topic. So what would that prove, exactly?

Oh, and the Pats are SO going to win the Super Bowl!

 

[drkitten]

You know, it wouldn't suprise me if the Polygraph does work better than chance at determining a 50/50 proposition under some circumstances. My main problem with it is that it's practically the opposite of double blind. It all comes down to an operator making the call. How does one eliminate operator bias in this situation?

Well, one standard way is to have as much of the data processed by computer (objectively) as possible. If the machine shows that on standardized data set alpha-niner-four, it was possible to draw a separating hyperplane between the truthful and non-truthful candidates' measurements, and that my measurement is on the lying side of that hyperplane, it's hard to attribute that to bias. We may argue about whether alpha-niner-four is a representative control group, but there's not much to argue about involving the categorization itself.

Another way is simply to look at the evidence upon which the operator makes his/her decision and see if it appears reasonable. If the operator says "you're lying," that doesn't mean much. But if the operator says "your skin temperature says you're lying," then there should be a visible difference in the graphs, specifically in the one for skin temperature. If the operator gives an opinion, but doesn't have a basis for the opinion, then one could rightly disregard that particular opinion -- but that's hardly unique to polygraphs. This problem has been around in "expert" testimony for centuries. When the doctor says that such-and-such a person died of heart disease and not pneumonia, you expect him to have citable medical reasons for her opinion.

Operator issues aside, you can certainly draw statistical conclusions from imperfect tests. The problem is I don't believe that polygraphs have a known error rate for beliefs. They don't detect "lies" they detect when someone is stressed about something. As they pointed out in mythbusters, there has to be a reason for there to be stress for it to work, having actually stolen something or having a punishment/reward for fooling the test. How can you seperate out "I'm lying about my belief" from "I was beaten as a child for not going to church" or "I was molested by a priest?"

You don't need to, unless you're suggesting that so many people were molested by priests that they constitute a significant minority of the target population, on the same order of magnitude as the detection threshhold. In some cases, this may be the case (e.g. if you're doing this experiment at a "survivors of priestly molestation" convention). But otherwise, any reasonable estimate of the probability of this would put it down in the "lost in the noise" section of the spectrum.

If you asked 100 people who say they're heterosexual if they're lying about being straight, would you trust the answers?

The first and only time? Possibly not. The thirtieth time over the course of two hours? Quite probably. As I said earlier, the emotional response from being asked essentially the same nosy question over and over by the same person bleaches quite quickly.

 

[NeilC]

skeptigirl and drkitten (and you other supporters of statistical accuracy)

This is all very interesting, and a nice laboratory trick, but Polygraph testing is not done on groups--it is done on individuals.
A single false positive is one too many.

Our system on laws is based on the theory (but not always the practice) that is is better to allow 100 guilty to go free than to imprison 1 innocent.
Polygraph testing has just the opposite effect.

Statistical analysis is wonderful stuff, used properly. Statistically, the probability of you getting injured by a drunk driver after 2:00 AM is .0001 (a number I made up), or 1/10000. Good odds, yes? Unless you happen to be that 1.

But we're not talking about the legal use of it. In fact everyone seems to agree it shouldn't be used to convict people.

And the supposed experiment that spawned this debate was (well wasn't in fact) done on a group. So if you could demonstrate a reasonable but far from perfect accuracy to the polygraph then the results would have some statistical significance.

 

[CFLarsen]

I've seen a variant of it that could be called the "gambler's proof": where you ask someone to bet a very large amount of money about the truth/validity of a given issue. Generally, the attacker sets the amount so high such that he can be confident the opposing party either doesn't have that much money available, or won't be willing to risk it on something so (comparably) trivial. It also requires that the validity of the issue in question cannot be determined easily, such that neither party can be certain of the truth prior to accepting the bet.

(Example: "If you're so sure the Patriots will win the Super Bowl, you should be willing to bet your house on it. Let's bet your house against mine, OK?")

In some instances, it can be used effectively to demonstrate the depth of someone's conviction in their stated ideas, but has no bearing whatsoever on determining the validity of the ideas themselves. It is generally a tactic used to attempt to stop discussion of a topic and force a concession to the attacker's viewpoint.

The requests to write a paper and/or present at TAM in this thread are similar.

That is a preposterous idea.

First, there is no money involved. Nobody will lose anything. Second, this is not an attempt to stop a discussion, quite contrary: The thread is opened specifically to get the evidence out in the open, to a much bigger audience.

Exactly. And that's how the various studies cited in the NAS report determined the accuracy and ROC curves for their polygraphs; they compared the results of the polygraph examination to the ground truths that they knew from other sources. (And, yes, they know about things like "blinding""; the polygraph examiner can't be the one who knows the ground truths.)

This is hardly rocket science; it's standard scientific practice and has been for literally decades. It's also how Randi knows that dowsers don't work -- he knows where the water is and he compares it to where the dowsers tell him that it is.

The difference is, in Randi's studies, the dowsers' performance is generally within the chance expectations --- it has never been beyond the rather stringent alpha cutoff of 0.001. The polygraph tests appear to be well beyond the threshhold of accuracy established by chance.

Save it for your article/presentation.

What have you got to lose by submitting an article to SkepticReport?

What about Skeptic Magazine? TAM?

You keep evading this.

skeptigirl and drkitten (and you other supporters of statistical accuracy)

This is all very interesting, and a nice laboratory trick, but Polygraph testing is not done on groups--it is done on individuals.
A single false positive is one too many.

Our system on laws is based on the theory (but not always the practice) that is is better to allow 100 guilty to go free than to imprison 1 innocent.
Polygraph testing has just the opposite effect.

Statistical analysis is wonderful stuff, used properly. Statistically, the probability of you getting injured by a drunk driver after 2:00 AM is .0001 (a number I made up), or 1/10000. Good odds, yes? Unless you happen to be that 1.

Exactly. Calling it a "laboratory trick" is very spot on.

Change their minds about what?

Unfortunately, I can't view videos at this time. I'm not sure what position they are arguing, per se. Are they claiming that polygraphs are not useful in law-enforcement? Are they claiming that they are not magic lie detectors?

I guess I'm just coming too late to the issue. I'm still hoping that someone with a law enforcement background will show up and either confirm or deny my (what must seem like a random) claim that: polygraphs are a law-enforcement interview tool (or even technique), not magic lie detectors.

Did you read the links?

I interpret it this way, as well. A extreme variant of the old "put up or shut up." It certainly doesn't promote discussion. Someone's willingness, or lack thereof, to write and article or make a presentation at TAM doesn't speak to the merits of their position. This is a discussion forum, for Pete's sake. There are acceptable levels of evidence that one could provide here that are well below doing either of these things. Especially the TAM presentation. Even if someone is willing and able to go to that extreme, and even if their position is deemed valid, that doesn't mean it will be accepted as a topic. So what would that prove, exactly?

Oh, and the Pats are SO going to win the Super Bowl!

It is precisely why I want to take it outside this forum: It has gone way beyond a discussion. We are talking about skeptics claiming evidence that lie detectors works.

That's a whole different ballgame.

 

[drkitten]

This is all very interesting, and a nice laboratory trick, but Polygraph testing is not done on groups--it is done on individuals.
A single false positive is one too many.

Huh? How.... "interesting."

Surely that depends on the actions you take w.r.t. the false positive, doesn't it?

As a simple example, if you're using a polygraph to investigate a specific criminal incident with small group of suspects, the results can be used to eliiminate (tentatively) a number of people who are likely to be innocent, so that you can concentrate the police resources more effectively. One robbery, ten suspects, eight of whom "pass" the polygraph test and are thus cleared. The cops still don't have enough evidence to convict anyone -- but they now only have two suspects to investigate in detail instead of ten, so they can use only 20% of the manpower and time. The alternative to the polygraph investigation would be to investigate all ten in detail -- in this case, the "false positive" will be subjected to exactly the same investigation that he would have been in any case. However, eight people have been spared that nuisance --- and eight cops are free to work on other crimes.

You might as well complain that blood typing should not be used, because of the possibility of spurious matches. If we know the robber's blood is type B negative, we still don't know who did it. Maybe there are three people with B negative blood in the suspect pool. But we do know it wasn't me (I'm O positive), and that's a step forward, yes?

Our system on laws is based on the theory (but not always the practice) that is is better to allow 100 guilty to go free than to imprison 1 innocent.
Polygraph testing has just the opposite effect.

Which is why people aren't imprisoned on polygraph evidence alone. Indeed, since polygraphs aren't usually admissible in (US) courts (by fiat), polygraph testing cannot be used even as part of a case to imprison someone.

 

[NeilC]

CFLarson: this whole TAM thing is irrelevant and just makes the discussion irritating and increasingly boring to follow. Why not drop it? No-one is evading anything. No-one is bound by you or anyone else to present anything at TAM.

If people have evidence or interpretations of evidence then they can "present" it here and we can discuss it. Hopefully like adults.

 

[rwguinn]

But we're not talking about the legal use of it. In fact everyone seems to agree it shouldn't be used to convict people.

And the supposed experiment that spawned this debate was (well wasn't in fact) done on a group. So if you could demonstrate a reasonable but far from perfect accuracy to the polygraph then the results would have some statistical significance.

I guess that is the point:
It's a nice lab trick, but from a practical standpoint, not particularly useful. The problem is that once people get it in their minds that it is 80%, or 90% (or whatever you want to go with) accurate, those numbers will be used to convince juries/employers/investigators that the value is as valid for individuals as for groups.
Therein lies the danger...

 

[drkitten]

What have you got to lose by submitting an article to SkepticReport?

My time and trouble, combined with the high probability that you will misrepresent what I write in such a way as to attribute to me opinions that I do not hold, to the general detriment of "the truth." Basically, the same reason that Dawkins doesn't write guest editorials to the Journal of Creation Science. He's got more important calls on his time.

I do write on this subject (well, I have not written on polygraphs specifically, but on the general problem of the forensics of moderately accurate information), and at length; indeed, I hope to have a book on this subject out in the next several months. I see no reason to give valuable "previously unpublished" that I can use in a professional setting to a cat-box liner run by an editor of questionable integrity and with an overinflated ego. (And if you think SkepticReport is anywhere near as useful or as valuable a publication medium to me as the Journal of Forensic Sciences, that's proof enough of ego.)

Make it worth my while. Explain why SR should trump JFS.

 

[Jimbo07]

Did you read the links?

This?

On the other hand, it is possible that one of the main reasons so many government, law enforcement and private sector employers want to use polygraphs is that they think the test will frighten away liars and cheats who are seeking jobs, or it will frighten confessions out of those accused of wrongdoing. In other words, the users of the machine don't really believe it can detect lies, but they know that the people they administer it to think the machine can catch them in a lie. So, the result is the same as if the test really worked: they don't hire the liar/cheat and they catch the dishonest employee.

Ding! I feel justified that at least my assertions aren't coming totally out of my rear end (I know they aren't, but now you have some evidence, too). The polygraph is not the only technique that law-enforcement officials use to trip up interviewees. It is one tool. There have been interviewees charged using techniques much less extreme (read: expensive, in operator time, etc.).

A really neat psychological experiment would be to see which interview techniques work best. As I suggested earlier, interviews are highly subjective. This would add science, but it's also an art.

The rest of that article, btw, looks like a rant about American politics. The FBI failed to learn this, pharma companies can do that, etc. I'm back to asking, what is being opposed, here?

 

[Jekyll]

The first and only time? Possibly not. The thirtieth time over the course of two hours? Quite probably. As I said earlier, the emotional response from being asked essentially the same nosy question over and over by the same person bleaches quite quickly.

Shouldn't the same be true of lying in response to a nosy question?
I can see hew generating and maintaining a consistent lie on the fly could be more stressful than telling the truth, but we're just talking about repeating a single answer in response to a question.

 

[drkitten]

I guess that is the point:
It's a nice lab trick, but from a practical standpoint, not particularly useful. The problem is that once people get it in their minds that it is 80%, or 90% (or whatever you want to go with) accurate, those numbers will be used to convince juries/employers/investigators that the value is as valid for individuals as for groups.
Therein lies the danger...

Why single polygraph testing out, though?

Every forensic technology has this problem. Even ballistics and toolmarks. Telling me that you can distinguish size 8 from size 9 shoes with 100% accuracy is very important, and telling me that there is a 100% chance that the burglar wears size 9 is very important. And that's much better than polygraph testing will ever be able to offer in terms of confidence....

So what am I supposed to do when the DA steps from "we know that the burglar wears size 9 shoes, and we know that rwguinn wears size 9 shoes" to "rwguinn must be the burglar"?

Are you going to blame the toolmarks guy, or the incompetent DA?

 

[Cardelitre]

I guess that is the point:
It's a nice lab trick, but from a practical standpoint, not particularly useful. The problem is that once people get it in their minds that it is 80%, or 90% (or whatever you want to go with) accurate, those numbers will be used to convince juries/employers/investigators that the value is as valid for individuals as for groups.
Therein lies the danger...

This looks a lot like an argumentum ad consequentiam...

 

[drkitten]

Shouldn't the same be true of lying in response to a nosy question?

Perhaps it should be, but it isn't. That's an empirical question, and, indeed, one of the key questions underlying "do polygraphs work."

Since they demonstrably work, the answer appears to be "no, the same isn't true for lying in response to a nosy question." For evidence, see the bibliography of the NAS report we've been discussing.

 

[CFLarsen]

CFLarson: this whole TAM thing is irrelevant and just makes the discussion irritating and increasingly boring to follow. Why not drop it? No-one is evading anything. No-one is bound by you or anyone else to present anything at TAM.

If people have evidence or interpretations of evidence then they can "present" it here and we can discuss it. Hopefully like adults.

No, it's very relevant. Remember that we are skeptics - and skeptics must follow the evidence, regardless if it means that we would have to change our minds.

That's what skepticism is all about.

I guess that is the point:
It's a nice lab trick, but from a practical standpoint, not particularly useful. The problem is that once people get it in their minds that it is 80%, or 90% (or whatever you want to go with) accurate, those numbers will be used to convince juries/employers/investigators that the value is as valid for individuals as for groups.
Therein lies the danger...

There is also the opposite to consider: If the evidence really is strong enough (as skeptigirl claims), then we cannot refrain from using it.

A tool which will give us a very high reliability - or in drkitten's case, substantially above chance - on detecting lies?

Sorry, that's too valuable not to use.

My time and trouble, combined with the high probability that you will misrepresent what I write in such a way as to attribute to me opinions that I do not hold, to the general detriment of "the truth." Basically, the same reason that Dawkins doesn't write guest editorials to the Journal of Creation Science. He's got more important calls on his time.

I do write on this subject (well, I have not written on polygraphs specifically, but on the general problem of the forensics of moderately accurate information), and at length; indeed, I hope to have a book on this subject out in the next several months. I see no reason to give valuable "previously unpublished" that I can use in a professional setting to a cat-box liner run by an editor of questionable integrity and with an overinflated ego. (And if you think SkepticReport is anywhere near as useful or as valuable a publication medium to me as the Journal of Forensic Sciences, that's proof enough of ego.)

Make it worth my while. Explain why SR should trump JFS.

If you don't want to write for SkepticReport, what about Skeptic Magazine? Or present it at TAM?

This?

Ding! I feel justified that at least my assertions aren't coming totally out of my rear end (I know they aren't, but now you have some evidence, too). The polygraph is not the only technique that law-enforcement officials use to trip up interviewees. It is one tool. There have been interviewees charged using techniques much less extreme (read: expensive, in operator time, etc.).

A really neat psychological experiment would be to see which interview techniques work best. As I suggested earlier, interviews are highly subjective. This would add science, but it's also an art.

The rest of that article, btw, looks like a rant about American politics. The FBI failed to learn this, pharma companies can do that, etc. I'm back to asking, what is being opposed, here?

What about the rest of the links?

 

[Belz...]

Hypothetically, say you asked 100 theists if they really believed in god and they all answered yes and the results showed they were all lying. If the polygraph gives valid results 80% of the time (without trying to fool it the report said the results were valid close to 100% of the time but say it was only 80%), then 80 theists would be lying. You could not tell which 80 were lying and which 20 were not lying but you could conclude 80 were lying based on the average validity of the polygraph results.

I really don't understand why the two of you don't understand the scientific concept of sensitivity and specificity.

I was under the impression that the polygraph didn't detect "lies" but, specifically, body functions. If someone's very nervous or very calm, doesn't that kinda defeat the whole thing ?

 

[Belz...]

I cannot reply to either of you because I don't for the life of me get your issues.

Are you talking about testing one person or the average results of 100 tests?

Hello.

100 flawed tests average wrong.

ETA: Never mind, I get the statistics thingy...

 

[rwguinn]

Why single polygraph testing out, though?

Every forensic technology has this problem. Even ballistics and toolmarks. Telling me that you can distinguish size 8 from size 9 shoes with 100% accuracy is very important, and telling me that there is a 100% chance that the burglar wears size 9 is very important. And that's much better than polygraph testing will ever be able to offer in terms of confidence....

So what am I supposed to do when the DA steps from "we know that the burglar wears size 9 shoes, and we know that rwguinn wears size 9 shoes" to "rwguinn must be the burglar"?

Are you going to blame the toolmarks guy, or the incompetent DA?

Well--
In your example, you cannot even rule out people with smaller feet (or to a certain extent, larger feet) because a small foot can go inside a big shoe, and a bigger foot can be somehat squeezed into a smaller shoe. So you neefd more data.
In acourt of law, there will be other evidence--not just shoe size but sole pattern, my fingerprints on the window, i was seen in the area at about that time, nobody could support my contention as to my whereabouts, mud matching the location, pawnbroker saying I sold him the jewells,etc--all of which make the shoe size very insignificant. A polygraph would require the same level of proof--supporting evidence, and probable cause to direct an investigation.


and as far as argumentum ad consequentiam goes--in this case I don't believeit is applicable. My argument is not a logical falacy, it is just an evaluation of human nature and greed. From my reading, and from what I know, people have lost jobs and had careers derailed --all from a single polygraph test.

 

[Rob Lister]

As a simple example, if you're using a polygraph to investigate a specific criminal incident with small group of suspects, the results can be used to eliiminate (tentatively) a number of people who are likely to be innocent, so that you can concentrate the police resources more effectively. One robbery, ten suspects, eight of whom "pass" the polygraph test and are thus cleared. The cops still don't have enough evidence to convict anyone -- but they now only have two suspects to investigate in detail instead of ten, so they can use only 20% of the manpower and time. The alternative to the polygraph investigation would be to investigate all ten in detail -- in this case, the "false positive" will be subjected to exactly the same investigation that he would have been in any case. However, eight people have been spared that nuisance --- and eight cops are free to work on other crimes.

More to the point, the one guilty person would likely confess. As a "lie detector", the lie detector has no value in terms of admissible evidence, some value in terms of culling (as you alluded), and a great deal of value in terms of getting a confession. That value greatly overshadows all others.

This is off-topic but the primary use of lie-detectors by police agencies (as opposed to other 'agencies') is one of interrogation. It is a tremendously effective intimidation tactic, given a skilled operator [interrogator] and a naive subject. Often, that's all you need. Additionally, it is a form of interrogation that is completely free of attorney representation and, even though the results are inadmissible in court, the things said during the test are completely admissible. Often that's all you need. It reminds me of the case of the run-away bride (do you remember it) wherein they asked the boy friend to take a lie detector test. He agreed under the condition that it would be video-taped and he could have his lawyer present. They, of course, declined...bitching all the way.

 

[Jimbo07]

What about the rest of the links?

The first was a video, which I can't watch right now. I've read the second. Like DrK, I'm coming up on a barrier of time I want to invest. Please just answer me.

What is the issure here? What is being contested? Polygraphs only? Polygraph interviews? Abuse of authority?

ETA: Thank you, Rob Lister.