Merged Odds Standard for Preliminary Test

I’m pretty sure Randi has addressed this in the past. If I recall correctly, he said there would not be any standard odds set in the Challenges rules because it is up to the applicant to describe their ability and how they will demonstrate the ability.

As Thabiguy said, this is a challenge. You claim you can do something paranormal. Randi challenges you to do what you say you can do. Then you have to prove it to win the million. If the result could occur by chance alone, the rate of success is determined along with the protocol. The number of trials and number of “hits” required will depend on the claim and the protocol. JREF may have to look at probability, significance, standard deviations, and confidence to determine what would be considered sufficiently unlikely so as to be considered “paranormal” and not just lucky.

If I recall correctly, Randi claimed he never said there is a fixed 1 in 1000 odds, and at one point someone on these forums traced that number back to a really old interview or TV or radio show where Randi tossed that number out as an example of what might be considered sufficient to pass a test and people just picked up on that number, then someone concluded that because the applicant would have to pass both the preliminary and final test the odds would be 1000 * 1000 = 1000000 for the final. Randi has repeated said that the success rate must be stated by the applicant as part of the claim and the final measure of success must be determined with the protocol.

If you look at the challenge applications, I think you will find that the final odds that are accepted are usually well under 1:1000, and usually hit around two standard deviations from the norm. I haven’t seen any instance in the challenge negotiations where JREF required outlandish odds. In my opinion, the odds accepted are usually too low to really be “paranormal”.

As far as long term tests like a Ganzfield test, in theory I don’t think JREF would have any problems, but in practice there are a number of difficulties. JREF relies on various organizations (like local skeptic groups) to conduct the tests and it may be difficult to find an organization that would be willing to put in the time. Of course the applicant would have to cover any costs. It may be possible if the applicant could come up with a good protocol.
 
The Challenge is about showing us something amazing. So the protocol starts with a description of those circumstances where we are likely to see something amazing. Isn't that the problem with the ganzfeld? Nobody knows what circumstances will lead to an amazing result because even though they repeat it over and over again, it only shows up as amazing once in a while?

Linda
 
I think virtually everyone accepts that fact that there were 1,008 hits in 3,145 trials. The dispute center on whether psi was responsible.

But there weren't only 3145 trials. And it's also unclear what number of hits should be expected in the absence of anomalous cognition for the given experiments.

Again, though, why don't you want to see a Ganzfeld challenge for the million dollars when even Ray Hyman is open to the possibility that Ganzfeld experiments are producing significant (due to psi) results?

Is he?

Linda
 
The Challenge is about showing us something amazing. So the protocol starts with a description of those circumstances where we are likely to see something amazing. Isn't that the problem with the ganzfeld? Nobody knows what circumstances will lead to an amazing result because even though they repeat it over and over again, it only shows up as amazing once in a while?

Linda
It's perfectly understandable that, if Ganzfeld experiments average hit rates of 30-32% over the long-term (where hit rates of 25% would be expected), there will be experiments of only 20-100 trials where the hit rates will be 25% or less. This is no difference than a much better than average baseball hitter, whose lifetime batting average is in the .300-.320 range, but who experiences many runs of 20-100 at-bats where his batting average is .250 or less. That's why a large number of trials would be necessary for a Ganzfeld MDC protocol.
 
But there weren't only 3145 trials.
Evidence?

And it's also unclear what number of hits should be expected in the absence of anomalous cognition for the given experiments.
Are you saying that a standard Ganzfeld protocol (expected hit rate of 25%) is different in this respect than another protocol with that same expected hit rate? Or simply that there have to be a large number of trials to establish anomalous cognition in Ganzfeld experiments?

He seemed to be in 1995. Has his position changed?
 
It's perfectly understandable that, if Ganzfeld experiments average hit rates of 30-32% over the long-term (where hit rates of 25% would be expected), there will be experiments of only 20-100 trials where the hit rates will be 25% or less. This is no difference than a much better than average baseball hitter, whose lifetime batting average is in the .300-.320 range, but who experiences many runs of 20-100 at-bats where his batting average is .250 or less. That's why a large number of trials would be necessary for a Ganzfeld MDC protocol.

I think that's the problem. The Challenge is set up to look at obviously amazing events, because psychics and other purveyors of the paranormal base their claims on obviously amazing events. The ganzfeld studies aren't obviously amazing - highly unexpected results are few and far between, and the overall average wouldn't be noticeably amazing until you've observed almost a thousand trials (who has the patience to stand on the street corner that long :)). If you confine yourself to considering situations that actually represent the types of claims that people make for the paranormal (rather than for parapsychology research), then most of your concerns over the odds of the Challenge would evaporate.

Linda
 
Evidence?

The existence of unpublished ganzfeld studies has been mentioned numerous times - including by those people you trust (i.e. believers).

Are you saying that a standard Ganzfeld protocol (expected hit rate of 25%) is different in this respect than another protocol with that same expected hit rate? Or simply that there have to be a large number of trials to establish anomalous cognition in Ganzfeld experiments?

I'm saying that the expected hit rate of 25% is theoretical rather than empirical, and in this situation it makes sense to actually measure the expected hit rate, rather than guess at what it would be. It's like pretending that the expected result from coin tosses is 50% heads and drawing conclusions based on deviations from this expectation.

Plus some biases in experimental design that alter even the theoretical expectations (such as drawing without replacement).

He seemed to be in 1995. Has his position changed?

What are you basing this idea on?

Linda
 
I think that's the problem. The Challenge is set up to look at obviously amazing events, because psychics and other purveyors of the paranormal base their claims on obviously amazing events. The ganzfeld studies aren't obviously amazing - highly unexpected results are few and far between, and the overall average wouldn't be noticeably amazing until you've observed almost a thousand trials (who has the patience to stand on the street corner that long :)).
Do you agree with me that the JREF should clarify whether a Ganzfeld protocol is eligible for the million dollar prize?

If you confine yourself to considering situations that actually represent the types of claims that people make for the paranormal (rather than for parapsychology research), then most of your concerns over the odds of the Challenge would evaporate.
Yes, but then the MDC devolves into little more than delusional challengers making unsupportable claims -- in most cases, if the challengers had simply bothered to take a couple of hours to test their claims with friends or relatives, they wouldn't have bothered to apply.
 
Do you agree with me that the JREF should clarify whether a Ganzfeld protocol is eligible for the million dollar prize?

What does it matter? Since when does Randi care what we think?

Yes, but then the MDC devolves into little more than delusional challengers making unsupportable claims -- in most cases, if the challengers had simply bothered to take a couple of hours to test their claims with friends or relatives, they wouldn't have bothered to apply.

Well, duh. That's what it already is, isn't it?

Linda
 
The existence of unpublished ganzfeld studies has been mentioned numerous times - including by those people you trust (i.e. believers).
According to Dean Radin: "If we insisted that there had to be a selective reporting problem, even though there's no evidence of one, then a conservative estimate of the number of studies needed to nullify the observed results is 2002." See Entangled Minds, Paraview Pocket Books (2006), at 121.

I'm saying that the expected hit rate of 25% is theoretical rather than empirical, and in this situation it makes sense to actually measure the expected hit rate, rather than guess at what it would be. It's like pretending that the expected result from coin tosses is 50% heads and drawing conclusions based on deviations from this expectation.

Plus some biases in experimental design that alter even the theoretical expectations (such as drawing without replacement).
If the experiment is designed properly, as even Hyman seems to concede has been true of the more recent Ganzfeld experiments, these are non-issues.

What are you basing this idea on?
As I noted in post #74: "In a 1995 paper discussing some of the challenges, deficiencies and achievements of modern laboratory parapsychology Ray Hyman said . . . 'I want to state that I believe that the SAIC experiments as well as the contemporary ganzfeld experiments display methodological and statistical sophistication well above previous parapsychological research. Despite better controls and careful use of statistical inference, the investigators seem to be getting significant results that do not appear to derive from the more obvious flaws of previous research.'"
 
According to Dean Radin: "If we insisted that there had to be a selective reporting problem, even though there's no evidence of one, then a conservative estimate of the number of studies needed to nullify the observed results is 2002." See Entangled Minds, Paraview Pocket Books (2006), at 121.

That's a comment about whether unpublished studies are more likely to be negative, not about unpublished studies, per se. As an aside, his estimate is based on studies with chance results, rather than selected results.

If the experiment is designed properly, as even Hyman seems to concede has been true of the more recent Ganzfeld experiments, these are non-issues.

Or at least the issue becomes moot.

As I noted in post #74: "In a 1995 paper discussing some of the challenges, deficiencies and achievements of modern laboratory parapsychology Ray Hyman said . . . 'I want to state that I believe that the SAIC experiments as well as the contemporary ganzfeld experiments display methodological and statistical sophistication well above previous parapsychological research. Despite better controls and careful use of statistical inference, the investigators seem to be getting significant results that do not appear to derive from the more obvious flaws of previous research.'"

I think you are over-interpreting his remarks.

Linda
 
Do you agree with me that the JREF should clarify whether a Ganzfeld protocol is eligible for the million dollar prize?
...

When a specific Ganzfeld protocol is submitted to the JREF we will have the answer to that, won't we?
 
When a specific Ganzfeld protocol is submitted to the JREF we will have the answer to that, won't we?
As I've already explained, that isn't likely to happen unless the rules are clarified. By the way, it's been 15 days since I sent an e-mail to challenge@randi.org, requesting such a clarification. Still no response.
 
Do you really think you can trick the JREF into changing it's rules so that your buddies can cheat?
 
The question still stands. Why do you think the JREF should change the rules to allow cheating and shoddy protocols?
 
As I've already explained, that isn't likely to happen unless the rules are clarified.

Funny.

If I thought I had a paranormal ability that would give me a slight edge over random chance, I'd simply sit down and work out a fair and secure protocol myself.

I would then submit it to the JREF.

See, it wouldn't be a lot of work, and either they'd say "no" or I'd get a fair chance to make an easy million bucks.

I would spend some time to work out how my ability would change my ability to play poker and I might alternatively come to the conclusion that it might be better to keep my mouth shut and move to Vegas. In neither case, however, would you find me arguing here about why the rules don't specify any odds...

Again: If I can do something paranormal it doesn't matter how good I am so long as I know how good I am! I can predict a fair coin with 51% accuracy? That could be easily tested. I'll apply and find out if the JREF considers this worthwhile and how many coin flips it would take to satisfy them. So what if it's a million coin flips? At 1$ per flip, that's a pretty decent deal!
 

Back
Top Bottom