Changes to protocols to avoid cheating?

[lister]

Whilst the protocols for the MDC are general extremely exhaustive in covering cheating, I've often wondered what would happen if an applicant was able to cheat in a way not covered.

I'm not talking about them "fooling" the JREF, but rather cheating in a way that is not covered by the protocol. By the MDC's own rules, the protocol is not allowed to be changed once agreed, so how could the JREF refuse the award the money even if it was apparent that the applicant was cheating?

As an example, the applicant could claim to use telepathy to transmit Zener cards to a reciever, but instead used the timing of their reply instead to indicate the card. Now obviously the JREF is wise to such tricks, but say in this instance they hadn't thought of it beforehand and it only became obvious once the test was underway. The JREF could state that they were not using telepathy but timing, but the applicant could state, "No, I'm using telepathy, and have demonstrated this in accordance with the protocol that you signed".

Given that the final test allegedly uses the same protocol as the initial test, albeit with higher odds, surely the JREF is legally obliged to pay the money despite no true paranormal ability was displayed.

Now I know the JREF aren't stupid, and that protocols are also closely scrutinised by members of the forum, but in theory something could get through. After all, you can't always work out how a magician does a trick even after seeing it, let alone before you've even seen the performance.

In all of Randi's tales of testing I've read, he seems to fix cheating as the test goes on (eg sticking tape over the sides of blindfolds etc). The MDC seems to work in a different way to this and seems potentially more open to abuse to me.

 

[JoeEllison]

That's probably why there's a preliminary test, and then one for all the marbles.

 

[lister]

That's probably why there's a preliminary test, and then one for all the marbles.

But I thought the preliminary and final test had identical protocols other than lengthened odds (or is this just common JREF folk law?)


ETA: This is from the FAQ. Not sure how official this is.

5.2 What happens between the preliminary test and the official test?

The protocol itself will not be changed, and neither will any of the documents you and the JREF have agreed upon. The final test may be longer, or require more conclusive results through more sets of the test to ensure that the preliminary test was not a fluke.

 

[JoeEllison]

But I thought the preliminary and final test had identical protocols other than lengthened odds (or is this just common JREF folk law?)


ETA: This is from the FAQ. Not sure how official this is.

Well, Randi can insist on changes to the protocol, and the tests can come to a complete halt... not the best outcome, and I would hope that there would be videotape evidence of why the protocol needed to be changed. Plus, I think that part of this is covered by the need for a written statement of exactly what the claimant is supposed to be able to do... to cut out some of the wiggle room.

 

[lister]

Well, Randi can insist on changes to the protocol, and the tests can come to a complete halt... not the best outcome, and I would hope that there would be videotape evidence of why the protocol needed to be changed.

I think I agree this is what would probably happen if it can. I'm not sure what the legally binding stuff says about changes between preliminary and final tests. As you say, not a good outcome as no doubt the believers would call foul, claiming that Randi had changed the rules to make it "impossible to win".

Plus, I think that part of this is covered by the need for a written statement of exactly what the claimant is supposed to be able to do... to cut out some of the wiggle room.

But the written statement statement spells out what they need to accomplish in order to prove their statement. You couldn't fail them after they'd completed the agreed protocol simply because you believe they are cheating. That's what the protocol is for!

 

[alfaniner]

Randi is not at the tests. He has stated himself that, if you are able to do what is claimed, under the specifications agreed to by both sides, you win. Even if it is revealed that the testers have been fooled.

One reason why the protocols must be hammered out with such detail.

 

[Ririon]

(...) One reason why the protocols must be hammered out with such detail.

...by an expert at "fooling people" and "cheating" (read: magician.)

 

[RemieV]

From the Chris Cordero protocol:

If any of the above listed are violated, or if it is determined the subject has attempted to tilt the results in his favor due to trickery of any kind, the test will be halted immediately and considered a failure.

From the Karen Sellers protocol:

If any of the above listed are violated, or if it is determined the subject has attempted to tilt the results in her favor through trickery of any kind, the test will be halted immediately and considered a failure.

alfaniner, the bolded part of your statement refers to retrospective knowledge. If, after the person passes both Challenge tests and collects the money and they later reveal they passed through trickery, JREF representatives do not descend on their home and steal the money back. If cheating is detected at any point during the proceedings, however, the test is immediately halted. Period.

 

[JoeEllison]

I'm pretty sure that you couldn't sue if they refused to give you the money because they sound a hidden radio transmitter on your person during the test, even if the test protocol didn't specifically prohibit the use of one.

 

[lister]

From the Chris Cordero protocol:


From the Karen Sellers protocol:

Aha! Thanks for that. Answers my question pretty succinctly.

 

[Tetris]

What exactly constitutes "trickery of any kind"? Is there a particular reason that this can't be specifically written out in the protocol, with an additional clause of what happens if the applicant tries to not follow the protocol?

 

[saizai]

There are ways to prevent cheating of ANY kind, known or unknown.

Generally speaking, they're called randomized double-blinded controlled trials.

 

[Cuddles]

What exactly constitutes "trickery of any kind"? Is there a particular reason that this can't be specifically written out in the protocol, with an additional clause of what happens if the applicant tries to not follow the protocol?

But what if the applicant thinks of a trick that the JREF hadn't thought of? Far better to have a general clause that if the applicant does anything dodgy, the test is cancelled. As soon as you try to be specific, you're just back to hoping that the JREF is better at trickery than the applicant.

There are ways to prevent cheating of ANY kind, known or unknown.

Generally speaking, they're called randomized double-blinded controlled trials.

Nope. The whole point of cheating would be to break the blinding.

 

[JMBadger]

JREF Protocols keep pace with cheaters

It has been said (and I know I am mis-quoting this) "Any sufficiently advanced science is indistinguishable from magic." Instead of magic we could say paranormal.

Certainly today we have sound speakers that look nothing like conventional speakers that could be used to produce the "voice from beyond". A tiny speaker in a suit of armor could make it talk. 2-way radios allow communication across miles that 300 years ago would have gotten you burned at the stake.

If sometime in the next 300 years someone succeeds in demonstrating paranormal abilities, they may be accused of cheating because they have a sub-cranial chip with optic nerve parallel circuitry and cochleal reception programming. But if they had any of that stuff right now, they'd win the prize.

I'm sure as technology advances, JREF's protocols strive to advance with it. Radio-jamming while the test is in progress, faraday-cage constructed rooms, possibly even undergoing x-rays and MRIs prior to testing.

 

[saizai]

cuddles - Proper blinding can't be broken without being detected.

Like say if the pair match data is stored in a bank safebox under dual required keys.


However, I think Randi's on record (don't remember ref though sorry) as saying that if someone manages to win it nonparanormally, then that's okay by him because it'll be worth it just in terms of advancement of knowledge. And besides, it'd go against his ego to think that someone could think of a way to trick the test that he couldn't think of first, given that it's his (ex?) profession and all that.

I don't think that a "if you cheat it's off" clause would be good, because the JREF would then be defining "cheat" and thus has an allpurpose escape clause. Just make the protocol correctly and it won't be needed.

 

[Cuddles]

cuddles - Proper blinding can't be broken without being detected.

Yes it can.

I don't think that a "if you cheat it's off" clause would be good, because the JREF would then be defining "cheat" and thus has an allpurpose escape clause. Just make the protocol correctly and it won't be needed.

As has already been said, unless the JREF can guarantee that they've thought of every single possible way someone can cheat, a no cheating clause is needed. It doesn't give the JREF any kind of escape clause for two reasons. Firstly, what the applicant is supposed to be doing is in the protocol. If they do anything else, it's cheating by definition. Secondly, it's impossible for the JREF, or anyone, to know the results until after the test is finished, so they wouldn't know they needed an escape clause until it was too late.

 

[remirol]

cuddles - Proper blinding can't be broken without being detected... I don't think that a "if you cheat it's off" clause would be good ... Just make the protocol correctly and it won't be needed.

You may have read elsewhere, on this site even, that scientists and the like are actually _easier_ to fool when it comes to magician-style trickery and deception. These statements well illustrate one of the reasons why.

By assuming that you will be able to think of every means of subverting the protocol, you are making the presumption that you are more clever than your adversary (the applicant). You have no way to ensure the truth of this, though, and this would be very bad if you are ever wrong and fail to think of one way that the adversary happens to use.

Also consider this: while you are certainly an astute, intelligent person with a logical mind, I doubt you have spent your life deceiving people and studying the ways in which people unconsciously fool themselves[1] -- as such, this is _not_ your field of expertise, and you must be aware of this at all times. Assuming that your skills in logical thinking carry over into the realm of deception in trickery is a recipe for certain failure.

While Randi is _also_ extremely confident that he will be able to think of all means of subverting the protocol, this _is_ his field of expertise; remember, he's a magician, not a scientist. That said, he _still_ leaves the clause for "if cheating is detected" in his protocols, because you can't really afford to have not made every effort to stay unfooled.

 

[saizai]

cuddles - Do you have a suggestion for how this could be done WITHOUT giving the JREF a universal out, if (let's be hypothetical here, since you know it'll be raised elsewhere) they wanted to lie and cheat their way out of awarding the prize by accusing others of doing so? It would have to be phrased in a purely objectively determinable manner, since it can't be purely on JREF's say-so...

remirol - While I've done some magic in the past, and my academic field is cognitive science (which has plenty to say about how people have vulnerabilities to deception), I confess I am not a professional deceiver, and do not have nearly the amount of experience as Randi in prestidigitation.

I think that in the cases where cheating has been done on scientists, protocol was improper. E.g. they failed to do blinding, were conned into removing controls, etc.

In cases where it's not possible to do blinding, e.g. magical spoon bending, then I would heartily agree that since we don't have an easy catchall method, a magician should be consulted to systematically prevent cheating methods. It wouldn't hurt otherwise as well.


P.S. To clarify a distinction I'm making that I thought was assumed but perhaps not: There are things that can be blinded & RCTed, and things that can't. The latter is almost always physical manipulation of objects.

Medical trials use doubleblind RCT because it does control for unknown confounding factors increasing the result. The confounds can only make the result less (i.e. less certain that there is one) - so there are other procedures to specifically control for as many factors as feasible so as to increase the ability of the trial to detect things - but when done right, the whole point of it is that you can't make a positive result where there isn't one (at probability > p).

I think that the way scientists (and those of a similar mindset like myself) have failings is to be in unfamiliar territory, e.g. where you're not doing doubleblind RCT or similar standard framework. In this case, you have to explicitly control for everything because the framework doesn't, therefore you want someone more familiar with what you need to control for.

I think that this is a more useful distinction - not between the TOPIC but between the METHOD.

 

[XBoxWarrior]

I most sincerely suggest, the Breathalyzer.....

allow no drunk idiots....no how, no way!

only true woo playas........