Who runs the tests?

[TheBoyPaj]

I emailed the JREF with the following question:

On your web site you state:
"The JREF does not involve itself in the testing procedure, other than helping to design the protocol and approving the conditions under which a test will take place"

Yet in the actual agreement which applicants must sign it states (point 10)

"the implementation and management of the challenge will be carried out by James Randi via the James Randi Educational Foundation"

These seem to be contradictory, and may put people off the challenge. A believer friend of mine states that since the actual agreement does not state your non-involvement in the tests, they would have no grounds to complain if they later discovered that you were directly responsible for the test's administration. Could this be clarified?

--------------------------------------------------------

I received an email from Randi this morning simply stating "there is no contradiction".

Of course I have replied asking for more clarification, but in anticipation that I might recieve an equally terse reply does anyone here know how these can be reconciled? Who would actually run the formal test?

 

[The Mad Linguist]

"The JREF does not involve itself in the testing procedure"

vs.

"the implementation and management of the challenge "

One is talking about the testing proceudre. One is talking about the entire setup of the challenge. No contradiction.

 

[TheBoyPaj]

Is the testing procedure not part of the implementation of the challenge?

Anyway, I have a reply from Randi confirming their non-involvement in the testing.

But that does not remove the possibility that a fraudster could excuse themselves from the challenge by claiming that there is nothing in the agreement which states that the test will be impartial.

I think for this challenge to be an effective display of the weakness of the believer's argument, it should be clear in the signed agreement that the test will be administered by independent scientists with both parties approval.

 

[Beleth]

"other than helping to design the protocol and approving the conditions under which a test will take place" is the phrase that clears up the contradiction.

The JREF designs the test.
Someone else executes the test.


And you should consider yourself lucky. From what I've heard, getting an answer as lengthy and as polite as "there is no contradiction" from Mr. Randi is an unusual occurrence.

Anyway.

A key part of the protocol is to come up with test conditions that both parties agree to. There may be nothing in the agreement procedure which states that the test will be impartial, but there has to be an agreement in the first place for the test to take place, and if the person being tested agrees to an impartial test, well, that's his own fault.

 

[TheBoyPaj]

I agree that if someone decides to take an impartial test then they cannot be surprised by the outcome, but my concern is that the value of the challenge is diminished by the fact that the agreement is not clear on this issue. The bit of blurb on the intro page states that the JREF is not involved, but the actual agreement, the bit the people sign, does not. The JREF could feasibly change the intro page at any time as it does not constitute part of the agreement.

I got more from Mr Randi when I asked the following:

------------------------------------

I am sure you are correct and I would like to point out that I tend to agree with you on this sort of thing, but would it be possible for you to clarify how the two statements sit together? Does the JREF run the tests, and if not what is the extent of your implementation and management of the tests?

------------------------------------

I got the following reply:

------------------------------------

The actual conduct of any test is performed by other persons, though we are often asked to advise. We assist in the design, and we must -- of course! -- approve the final protocol and the personnel who conduct the test. On occasion, with the agreement and/or the request of the applicants, we have conducted the tests.

James Randi

------------------------------------

I also asked the following:

------------------------------------

Thanks for your reply. I would like to recommend, however, that you include this addition in the signed agreement to prevent a fraudster from saying that they do not want to take the challenge because they fear you will be the person actually running the test. They don't trust you, as I am sure you are aware!

------------------------------------

I got the following reply from Randi:

------------------------------------------

I don't care whether they trust me or not, and I don't care if they want to get out of taking the challenge...........!

------------------------------------------

My reply:

------------------------------------------

But don't you agree that one of the purposes of your challenge, and the fact that no one has ever passed it, is that it demonstrates to anybody who remains undecided on the issue that all paranormal claims melt away in the light of proper research? At the moment it seems you have a get-out clause in your agreement, which diminishes that demonstration in the eyes of the genuinely interested observer. I can understand that you feel you have nothing to prove, but why have the challenge at all if it is so easy to dismiss on the grounds that it would not be fair?

------------------------------------------

His reply:

------------------------------------------

Let them argue with me....

James Randi

------------------------------------------


That's the extent of my correspondence in the issue. Mr Randi doesn't seem to worry that the agreement leaves this element of doubt. I suppose it is his challenge and he can run it any way he likes, but all a fraudster has to say is "I'm not taking the challenge because he might renege on the bit about not being involved" and there would be nothing in writing to say that wouldn't happen.

 

[The Mad Linguist]

I think you're barking up a dead end here, if I may mix my metaphors. I suspect that the reason the JREF's non-involvement in the tests is not part of the agreement is exactly that some applicants do want to be tested by Randi himself (a lot of the tests he reports in his commentary are of this type).

Given that the test procedure must be agreed on by both parties, in advance, as a fair test of the claimed powers, I don't see that any other specific fairness measures than this need be enumerated.

I don't think there's any element of doubt.

 

[TheBoyPaj]

Then the agreement should state:

All formal experiments will be conducted by independent researchers with the approval of both parties or, in certain circumstances, by a member of the JREF, again with both parties' approval.

Without this clause the following scenario could occur:

A claimant who is genuinely under the impression that they have a paranormal ability may wish to have a crack at the challenge. But the wording of the agreement does not preclude the possibilty that is will be an unfair test, administered by people who already have ruled out the possibility of your success. This person fears that they will apply, find out that they cannot agree on a protocol that they believe will be fair, and subsequently have to pull out. This now leaves Randi with a nice story to put in his weekly commentary about the fool from Detroit (or whatever) who believes he can control fish with his thoughts and the bloke becomes a laughing stock. Pulling out of the test is seen as an admission of failure almost as much as failing the test itself, even when all the person wanted is a fair test of their abilities and a chance to win a million.

Now, imagine we have a fraudster who is conning millions. In this case the challenge is a weapon for us, the sceptics, to say "look, this person is too scared to even take a simple test of his abilities. He's clearly a con-man."
After all, that's the real point of the challenge, isn't it? It's such a simple thing, but yet they cannot pass it.
But all this fraudster has to do is state the example given above, and say that they are not going to subject themselves to ridicule if they are forced to withdraw when they find out it will be Randi conducting the test. Bingo! They have a ready-made excuse.

Randi doesn't care about what excuses they come up with. He is certain of his beliefs and knows that they're all fake anyway. But he also goes on TV to tell others about this challenge, and to hold it up as a great shining example for the masses. But this flaw in the agreement makes it shine a little less brightly IMHO.

 

[The Mad Linguist]

There's no response to this.

If the phrase "the test procedure must be agreed on by both parties, in advance, as a fair test of the claimed powers" doesn't explain it, then nothing will.

Fair means fair - in any way at all. Fair in terms of the protocol, fair in terms of the target results, or fair in terms of who's conducting it. If the test is unfair, the applicant doesn't take it - they work out one that is fair.

 

[Peter Morris]

The trouble with that is that if the applicant doesn't think it fair, Randi doesn't 'work out one that's fair', he tells them that's the test I'm offering, take it or leave it.

If they agree to take the test, then that is taken as an agreement that they think its fair, ignoring their statements to the contrary.

If they don't agree to the test, Randi claims this as a victory for himself.

For an example of such, see Randi's own description of the test for Chinese medicine here: http://www.randi.org/jr/122702.html

Note TRandi's failure to work out a test that is acceptable, his 'take it or leave it' attitude, and his declaration of victory when they leave it.

Now, if the test agreement gave them the opportunity to say << I don't think the test is fair, I don't think my power works like that, but its the only test Randi offered me, and I'm ready to try it anyway >> then maybe he'd get more people willing to try it.

 

[The Mad Linguist]

Now, if the test agreement gave them the opportunity to say << I don't think the test is fair, I don't think my power works like that, but its the only test Randi offered me, and I'm ready to try it anyway >> then maybe he'd get more people willing to try it.

But why one earth would anyone want to take an unfair test?

If the text is unfair, and they pass it, it means nothing.
If the test is unfair, and they fail it, it means nothing.

That's the whole point of making tests fair in the first place!

I read Randi's report of the Qigong applicant somewhat differently to you. I don't think it was a case of him offering "just that test".

The applicant initially attempted to diagnose Randi from a photograph, and failed. However, this is not a scientifically designed test, so even if he'd got Randi's ailments right, it wouldn't mean anything. The applicant does not appear to have outlined any notion of testing of their own. So Randi proposed a test - quick, easy, and valid.

The applicant turned this test down. Note that the applicant's daughter, in her response, did not suggest any other way of testing the claim. In fact the only really clear statement is "my father could give the diagnosis for him or her", which is a restatement of the claim, not a method for testing it.

The ball is therefore in the Qigong fellows court, to make clear what would be an adequate test of his powers, if Randi's suggestion isn't. Now we don't know if the correspondance went any further. Maybe it did, maybe it didn't.

But in general, Randi's offer of the death-test does make one thing very clear to the applicant straight away: that they cannot pass by chicanery or by manipulating the human beings that judge them, that they will have to actually pass by producing objectively verifyable results. This is why it saves Randi time because of course, none of them can produce objectively verifiable results.

 

[Peter Morris]

The ball is therefore in the Qigong fellows court, to make clear what would be an adequate test of his powers, if Randi's suggestion isn't.

It seems that there is no question of that, Randi has set the test, he has refused to alter it, the subjects may either take the test, declring it to be 'fair', or they can drop out. Altering the test isn't on the cards.

 

[kookbreaker]

The trouble with that is that if the applicant doesn't think it fair, Randi doesn't 'work out one that's fair', he tells them that's the test I'm offering, take it or leave it.

Oh yeah, like with the Penta water people, where he forced them to [/QUOTE]use the testing method they suggested.

(Of course, the Penta water folks never took the challenge)

If they agree to take the test, then that is taken as an agreement that they think its fair, ignoring their statements to the contrary.

If they don't agree to the test, Randi claims this as a victory for himself.

For an example of such, see Randi's own description of the test for Chinese medicine here: http://www.randi.org/jr/122702.html

Hardly a clear example. Theses folks were capable of suggesting a protocol. Randi suggested one he thinks was best. They refused the protocol, and did not follow up.

 

[kookbreaker]

It seems that there is no question of that, Randi has set the test, he has refused to alter it,

Where did you get that? Randi seems to suggest that he feels his test is the best design, but they are free to suggest a test design with equal control. ("dead" is a remarkable control).

 

[Peter Morris]

"Where did you get that"

From Randi's description of the correspondance.

"Dead is a remarkable control"

I'm not certain, but my understanding is that they claim they can diagnose from a photo, but its not divination. I think they mean they can diagnose current conditions from a current photo, or given an old photo they can diagnose conditions when the photo was taken.

Giving them an old photo of someone, whose condition has changed significantly since the photo was taken, is not a fair test IMHO.

DISCLAIMER: I'm not supporting their claims at all, just pointing out certain flaws in Randis offer of a test. Please bear that in mind.

 

[kookbreaker]

"Where did you get that"

From Randi's description of the correspondance.

"Dead is a remarkable control"

I'm not certain, but my understanding is that they claim they can diagnose from a photo, but its not divination. I think they mean they can diagnose current conditions from a current photo, or given an old photo they can diagnose conditions when the photo was taken.

Giving them an old photo of someone, whose condition has changed significantly since the photo was taken, is not a fair test IMHO.

OK, then they must suggest a protocol where there is a definate diagnosis without any potential weasel factor, a la the assorted Larry King psychics. I.e. there's no "I sense something in the liver area" without specifics and diagnosis. In other words, remove the cold reading.

There's nothing preventing them from making their own suggestions. The way I read it, Randi is not declaring victory but rather noting that they have refused one version of a controlled test.

 

[Peter Morris]

OK, then they must suggest a protocol where there is a definate diagnosis without any potential weasel factor

Ah, but would Randi accept it if they did so? Randi's own account strongly suggests otherwise.

You may interpret it differently if you wish to, but that's my understanding.

Any further discussion of the subject would be a waste of time.

 

[The Mad Linguist]

It seems that there is no question of that, Randi has set the test, he has refused to alter it, the subjects may either take the test, declring it to be 'fair', or they can drop out. Altering the test isn't on the cards.

Erm, where exactly does Randi say this?

I agree with you that the suggestion of the death test is meant to discourage. It's meant to discourage by making proponents of silly claims realise exactly what level of scrutiny they're going to face if they persist with the challenge.

At no point does the applicant actually suggest a test that could be performed. So you're right that the death test is the only one on the table at the time Randi wrote, but that's because the applicant hasn't put forward any alternative.

I'm not certain, but my understanding is that they claim they can diagnose from a photo, but its not divination. I think they mean they can diagnose current conditions from a current photo, or given an old photo they can diagnose conditions when the photo was taken.

Giving them an old photo of someone, whose condition has changed significantly since the photo was taken, is not a fair test IMHO.

You'd be absolutely right - IF the claim actually is that the Qigong fellow can diagnose conditions that the subject had at the time the photo was taken. At no point does the applicant specify this, however. If he/his daughter HAD, then of course Randi wouldn't have suggested the death test.

--------------------------------------------------------------------------------
OK, then they must suggest a protocol where there is a definate diagnosis without any potential weasel factor
--------------------------------------------------------------------------------

Ah, but would Randi accept it if they did so? Randi's own account strongly suggests otherwise.

Please quote anything that Randi says in the commentary that suggests that he would reject the applicant from the Challenge if the applicant suggested a protocol without any room for the "weasel factor".

I think it more likely he'd be overjoyed if they did so, because it would mean he could actually investigate their claim, rather than exchanging emails with psychos, which seems to take up an awfully large proportion of his working hours.

 

[Peter Morris]

Please quote anything that Randi says in the commentary that suggests that he would reject the applicant from the Challenge if the applicant suggested a protocol without any room for the "weasel factor".

The fact that they object to the test offered, and Randi's reply is essentially 'take it or leave it.'

That is my impression, you are entitled to a different impression if you wish.

 

[The Mad Linguist]

Well, we'll have to agree to differ. To me it seems that Randi's response was (understandably) to pour scorn on the applicant's objection, which is NOT the same thing as saying "take it or leave it".

Note that he was open to hearing again from these people (though we don't know whether he ever did). If they did get back to him with a suggestion realistic protocol, then the Challenge would have to go ahead.

 

[Peter Morris]

Pouring scorn on their claims is right, he does this a lot. His rudeness certainly adds to the impression that he is not going to conduct fair tests. I don't blame anyone who gives up on an attempt to negotiate a test with him.