Merged Odds Standard for Preliminary Test

This is from the Carina Landin thread
...

I edited your post because there are too many wrong and unsubstantiated assertions to rebut for me tonight.

Not to my surprise though, you seem to have missed the very next post in the thread you quoted from. It is the one below:

Hi Speed of Light.

The rule is generally that both the preliminary and final tests have odds of 1 to 1000, giving a total chance of 1 to 1,000,000.

We don't know why they gave Landin more favorable odds, but I suspect that it was as much the hassle of getting all those diaries as anything else. Even if Landin can do what she claims she wouldn't be expected to get 100%. For example, she might get an image of a husband who cherished his dead wife's diary for years and label the diary male.

If she could in fact get 80% right then it would take more than 20 diaries to get a good test. If the final test ever happens they'll have to come up with more than 20 diaries. This is all allowed for in the rules.

And then there is this one:

I cannot do these calculations myself, but others have performed them in this thread, notably here and here, and they disagree with you.
 
All of which is utterly irrelevant..
Seems vey opinionated.
If what others say is All irrelevant, then we are not going to get any meaningful dialogue.
A good illustration of 'lack of undestandiing or interest in subtleties'


It makes no difference at all whether someone claims to be 100% accurate or 51% accurate. If they can do what they claim, it doesn't matter in the slightest what the possibility of them passing by chance is.
Another dogmatic statement, with no hint at understanding what Rodney is trying to say.
This statement also seems to deny the very existence of statistics.

I am aware that many people do not understand, or are put off by statistics, but I don't think there is any need for meaningless dogmatic responses, completely ignoring the statistical arguments which have been made


The only way chance can matter is if they are actually relying on chance and not their claimed ability.

What the hell does that mean??

I agree that this explains why there are no non-delusional challengers, but not in the way that you seem to think.
Is this a riddle?
 
Last edited:
Seems vey opinionated.
If what others say is All irrelevant, then we are not going to get any meaningful dialogue.

That's right. The reason for that is that neither you nor Rodney have anythign meaningful to say. A meaningful dialogue is simply not possible under those circumstances.


I am aware that many people do not understand, or are put off by statistics, but I don't think there is any need for meaningless dogmatic responses, completely ignoring the statistical arguments which have been made.

No statistical arguments have been made, and there is therefore nothing there not to ignore. And, no, I'm not one to be put off by statistics; in fact I've taught the subject professionally from time to time. The simple fact is that the p-value is not relevant in circumstances where random chance is not a factor.

As a simple example : suppose you offer to predict the final score of a basketball game. The "chance" of you doing it correctly through random guessing is astronomically small, but if you've already seen the game (and we're watching it together on tape delay), then "chance" is irrelevant. You have special knowledge that in this case allows you to do something that would be arbitrarily difficult under "normal" circumstances.

Similarly, the chance of being dealt a hand of all spades in a bridge game is almost unthinkable -- unless a professional magician is doing the dealing, in which case it's routine.

The "p-value" is thus a test of which is which. If you have the special skill or knowledge you claim to have, then you can do a task that is arbitrarily unlikely. If you do not have, then the chances of your succeeding are arbitrarily small.


In the case of the Randi challenge --- Randi listens to your description of what you can do (and what accuracy you can expect). He will then propose a task that if you can do what you say, will be well within your skill level. The task will also be almost impossible to do "by chance" and he will take appropriate precautions against ordinary cheating-type manipulations.

For example -- I claim to be able to look at a tin of soup (with the label visible) and tell you what the contents of the can are. I claim I can do that with virtually 100% accuracy. (And I can, too; unless the manufacturer made a mistake, which rarely happens.) If Randi accepted the challenge, he might say something like : all right, we will give you a set of twenty soup tinss -- some tomato, and some cream of mushroom -- and you have to successfully pick out the ones that are tomato. I claim that I can do twenty out of twenty, so he will allow me two mistakes and see if I can get 18 out of twenty. Simple claim, simple protocol, simple task.

And do you see how it doesn't matter in this case whether he gives me twenty cans, or fifty, or a hundred? From my point of view, either I can read the labels or I can't. But reading a hundred labels is no more difficult than reading fifty, and if I can't manage a hundred, I can't manage twenty --- or two. The only reason I would care about the p-value is if I plan to guess instead of reading the labels.

Now, for a more "realistic" challenge. I have X-ray vision and I can do the same thing, even when the tins are sealed in an opaque container. The task is almost exactly the same, and the same argument shows that it doesn't matter whether he gives me a hundred or twenty tins. If I really have X-ray vision, I just read the labels through the shoe-box. If I don't have X-ray vision, I can't read the first one, let alone the twenty-first.

The p-value is simply not relevant.

So he would prefer to go with 100 cans, but I might object; it will be a long and boring afternoon, and soup tins aren't free. So I say "how about five", but he doesn't consider that a sufficiently stringent test. Here's where the negotiation comes in. It's my time and money vs. his time, money, and confidence.
 
Last edited:
Shadron, it's a question not so much of what the claimant claims, as what the JREF will require in order to agree to an MDC application. It's not just what someone claims, it's also a matter of what the JREF will require. As Rodney said,

If, for example, an applicant could consistently determine with 54% accuracy whether a face-down playing card was red or black, even 100 trials would be woefully insufficient to prove that, because 54 hits in 100 trials could easily be attributable to chance (P=.24). However, if the applicant could continue to perform at that level over about 3500 trials, (s)he would meet a P=.000001 criterion.​

So the p-value needs to be sufficient. I don't mean this rhetorically: why can't the sufficient p-value be established before hand? What circumstance would require different p-values for applicants who have an odds-based claim?
 
This is from the Carina Landin thread
I edited your post because there are too many wrong and unsubstantiated assertions to rebut for me tonight.

Crikey! You definitely know how to make someone feel good!

If everyone simply rebutted everyone else, then we wouldn't get very far, would we?

Not to my surprise though, you seem to have missed the very next post in the thread you quoted from. It is the one below:

Hi Speed of Light.
The rule is generally that both the preliminary and final tests have odds of 1 to 1000, giving a total chance of 1 to 1,000,000.
We don't know why they gave Landin more favorable odds, but I suspect that it was as much the hassle of getting all those diaries as anything else. Even if Landin can do what she claims she wouldn't be expected to get 100%. For example, she might get an image of a husband who cherished his dead wife's diary for years and label the diary male.
If she could in fact get 80% right then it would take more than 20 diaries to get a good test. If the final test ever happens they'll have to come up with more than 20 diaries. This is all allowed for in the rules.

And then there is this one:

I cannot do these calculations myself, but others have performed them in this thread, notably here and here, and they disagree with you.

You seem to be in a rush to prove people wrong -
What about this post, later on?

I also get 1 in 169.234, assuming "by chance" means coin-flipping.
2^20 ways to label the diaries "right" or "wrong" = 1048576
20!/(20! * 0!) ways to get 20 right = 1
20!/(19! * 1!) ways to get 19 right = 20
20!/(18! * 2!) ways to get 18 right = 190
20!/(17! * 3!) ways to get 17 right = 1140
20!/(16! * 4!) ways to get 20 right = 4845
1048576 / (1+20+190+1140+4845) = 169.2
If Carina takes into account the fact that there are at least 5 male and at least 5 female diaries her odds are even better.

ChristineR gets exactly the same value for the probability as I do, using more or less the same method

There is also this post later on, responding to ChristineR

You've got it upside down. It should be (1+20+190+1140+4845) / 1048576 = .0059
Or you can do it like I did in Excel: =1-BINOMDIST(15, 20, 0.5, 1) will give the probability of getting 16 or more correct which also gives .0059
ETA: I did make a computational error in my earlier post. I had the .0059 as the probability of getting 15 or more correct. It's not, it's 16 or more correct and 0.0013 is the probability of getting 17 or more correct [Excel formula 1-BINOMDIST(16, 20, 0.5, 1)]. Since JREF tests typically ask for p-values of .001, I suspect the person who arrived at 16 as the necessary number to succeed may have made the same computation error I did in my earlier post.
True.

Beth gets a probability of 0.0059, using Excel, which is almost exactly the same as we got (1 in 169 approximately)

GzuzKryzt, Why didn't you check the statistical calculations before trying to say they were wrong?
 
Last edited:
That's right. The reason for that is that neither you nor Rodney have anythign meaningful to say. A meaningful dialogue is simply not possible under those circumstances.


No statistical arguments have been made, and there is therefore nothing there not to ignore. And, no, I'm not one to be put off by statistics; in fact I've taught the subject professionally from time to time. The simple fact is that the p-value is not relevant in circumstances where random chance is not a factor.

As a simple example : suppose you offer to predict the final score of a basketball game. The "chance" of you doing it correctly through random guessing is astronomically small, but if you've already seen the game (and we're watching it together on tape delay), then "chance" is irrelevant. You have special knowledge that in this case allows you to do something that would be arbitrarily difficult under "normal" circumstances.

Similarly, the chance of being dealt a hand of all spades in a bridge game is almost unthinkable -- unless a professional magician is doing the dealing, in which case it's routine.

The "p-value" is thus a test of which is which. If you have the special skill or knowledge you claim to have, then you can do a task that is arbitrarily unlikely. If you do not have, then the chances of your succeeding are arbitrarily small.


In the case of the Randi challenge --- Randi listens to your description of what you can do (and what accuracy you can expect). He will then propose a task that if you can do what you say, will be well within your skill level. The task will also be almost impossible to do "by chance" and he will take appropriate precautions against ordinary cheating-type manipulations.

For example -- I claim to be able to look at a tin of soup (with the label visible) and tell you what the contents of the can are. I claim I can do that with virtually 100% accuracy. (And I can, too; unless the manufacturer made a mistake, which rarely happens.) If Randi accepted the challenge, he might say something like : all right, we will give you a set of twenty soup tinss -- some tomato, and some cream of mushroom -- and you have to successfully pick out the ones that are tomato. I claim that I can do twenty out of twenty, so he will allow me two mistakes and see if I can get 18 out of twenty. Simple claim, simple protocol, simple task.

And do you see how it doesn't matter in this case whether he gives me twenty cans, or fifty, or a hundred? From my point of view, either I can read the labels or I can't. But reading a hundred labels is no more difficult than reading fifty, and if I can't manage a hundred, I can't manage twenty --- or two. The only reason I would care about the p-value is if I plan to guess instead of reading the labels.

Now, for a more "realistic" challenge. I have X-ray vision and I can do the same thing, even when the tins are sealed in an opaque container. The task is almost exactly the same, and the same argument shows that it doesn't matter whether he gives me a hundred or twenty tins. If I really have X-ray vision, I just read the labels through the shoe-box. If I don't have X-ray vision, I can't read the first one, let alone the twenty-first.

The p-value is simply not relevant.

So he would prefer to go with 100 cans, but I might object; it will be a long and boring afternoon, and soup tins aren't free. So I say "how about five", but he doesn't consider that a sufficiently stringent test. Here's where the negotiation comes in. It's my time and money vs. his time, money, and confidence.

The original purpose of this thread, was to discuss a particular type of claim -That in which only slightly above chance effects are claimed, so probability in this case becomes an overiding factor. This is different from most past claims, because the results are not obvious, and must be analysed statistically.
The more a claim is only slightly above chance, the more mathematics must come into it
The calculated probability of succeeding is, in effect, a perfectly accurate measure of the 'Hardness' of the task, whatever the claim - So the application of a set probability standard, would mean that evey applicant who conforms to that standard, will have exactly the same chance of passing by chance alone - Very like imposing weights, in horse racing - to equalise the probabilities. Equalising probabilities is like a handicap system, so that no applicant has an advantage over any other, and the hardness of their tasks are exactly the same for everyone
 
The p-value is simply not relevant.
Only to those who, like you, have the simplistic view that the paranormal cannot be real unless it operates in an always on manner. I would suggest that the proper model for the paranormal is human performance. For example, on some days, even the best athlete in the world is way off his game. But, applying your logic, Tiger Woods should never hit a bad shot.
 
...
GzuzKryzt, Why didn't you check the statistical calculations before trying to say they were wrong?

I have to admit that I am in over my head with these statistics. I relied on the calculations of Beth.

I apologize if I was wrong, Speed of Light.



Perhaps someone else will come up with a clarification.
 
Only to those who, like you, have the simplistic view that the paranormal cannot be real unless it operates in an always on manner.

As opposed to your equally simplistic view which suggests that credit should be given for things which cannot be done either reliably or in a manner significantly more common than chance would indicate? I think not.

I would suggest that the proper model for the paranormal is human performance.
Wrong. The proper model for the paranormal is what the applicant claims he or she can do.

For example, on some days, even the best athlete in the world is way off his game. But, applying your logic, Tiger Woods should never hit a bad shot.
Strawman. Tiger Woods makes no claim that he will hit only good shots.

There is a reason that the claim is hand-tailored to each applicant, and that reason is that the Challenge tests what the claimant says he/she is able to do. If the claimant states that he can perform with 100% accuracy, that is the claimant's business; a protocol will then be negotiated that is well within the claimant's ability ACCORDING TO THE CLAIMANT, and is also sufficiently outside the range where success could be obtained by simple random chance.

While a claim that someone could perform "X" task with 54% accuracy might be worth entertaining, the number of tests required to rule out random chance alone would be large; and at some point the relevance of the task itself might be called into question, as well as the sincerity of the applicant. Bear in mind that nobody has successfully passed even the preliminary test; this indicates that any given claim is likely to be false or delusional, and thus endless patience is _not_ due for extremely time-consuming claims.

Also consider that where appropriate (see the dowsing protocols), each applicant is requested to confirm prior to the preliminary test that their ability is, indeed, "working" that day. If it is not working for some reason, the test could be rescheduled appropriately. Unless I am quite mistaken, nobody has ever backed out of the preliminary test and said "It's not working, something's wrong".

So, if your Ganzfeld researchers think they can beat random chance in some manner, have them formulate a claim for something they can _demonstrate_, and then apply. The same considerations mentioned above will be given to them, and the protocol will be appropriately tailored to be well within their claimed capabilities, while still well beyond the realm of success-via-random-chance.
 
The calculated probability of succeeding is, in effect, a perfectly accurate measure of the 'Hardness' of the task, whatever the claim

How hard the task is depends on whether the applicant has paranormal abilities and, if he has, on the precise nature of those abilities. The "calculated probability of succeeding" is calculated on the assumption that he hasn't any. But he believes that he has some. Otherwise he wouldn't apply. So he disagrees that the task's difficulty is accurately measured by that probability.
 
As opposed to your equally simplistic view which suggests that credit should be given for things which cannot be done either reliably or in a manner significantly more common than chance would indicate? I think not.

Wrong. The proper model for the paranormal is what the applicant claims he or she can do.

Strawman. Tiger Woods makes no claim that he will hit only good shots.

There is a reason that the claim is hand-tailored to each applicant, and that reason is that the Challenge tests what the claimant says he/she is able to do. If the claimant states that he can perform with 100% accuracy, that is the claimant's business; a protocol will then be negotiated that is well within the claimant's ability ACCORDING TO THE CLAIMANT, and is also sufficiently outside the range where success could be obtained by simple random chance.

While a claim that someone could perform "X" task with 54% accuracy might be worth entertaining, the number of tests required to rule out random chance alone would be large; and at some point the relevance of the task itself might be called into question, as well as the sincerity of the applicant. Bear in mind that nobody has successfully passed even the preliminary test; this indicates that any given claim is likely to be false or delusional, and thus endless patience is _not_ due for extremely time-consuming claims.

Also consider that where appropriate (see the dowsing protocols), each applicant is requested to confirm prior to the preliminary test that their ability is, indeed, "working" that day. If it is not working for some reason, the test could be rescheduled appropriately. Unless I am quite mistaken, nobody has ever backed out of the preliminary test and said "It's not working, something's wrong".

So, if your Ganzfeld researchers think they can beat random chance in some manner, have them formulate a claim for something they can _demonstrate_, and then apply. The same considerations mentioned above will be given to them, and the protocol will be appropriately tailored to be well within their claimed capabilities, while still well beyond the realm of success-via-random-chance.

I have the strangest feeling that someone is having a lend of us here.

The Ganzfeld researchers will never submit themselves to any kind of definitive test because the results would most likely see them lose their funding, and what shred of reputation they still possess.


M.
 
I have the strangest feeling that someone is having a lend of us here.

The Ganzfeld researchers will never submit themselves to any kind of definitive test because the results would most likely see them lose their funding, and what shred of reputation they still possess.
M.
Your position would be strengthened if the JREF were to encourage Ganzfeld experimenters to apply for the Challenge, and then no one does.
 
Only to those who, like you, have the simplistic view that the paranormal cannot be real unless it operates in an always on manner. I would suggest that the proper model for the paranormal is human performance. For example, on some days, even the best athlete in the world is way off his game. But, applying your logic, Tiger Woods should never hit a bad shot.
If Tiger Woods was the golfing equivalent of the Ganzfeld, he'd have hit only a couple of good shots, on training days, a long time ago, when no one else was really bothering to watch.

His sponsors would have dropped him because he was so boring, and he wouldn't be allowed to play with the pros. But there'd still be a couple of idiots hyping him up on message boards claiming he was the second coming of Christ.
 
If Tiger Woods was the golfing equivalent of the Ganzfeld, he'd have hit only a couple of good shots, on training days, a long time ago, when no one else was really bothering to watch.

His sponsors would have dropped him because he was so boring, and he wouldn't be allowed to play with the pros. But there'd still be a couple of idiots hyping him up on message boards claiming he was the second coming of Christ.
Your analogy is off-point. A golfer who hit more bad shots than good shots might be boring and not allowed to play with the pros, but the relevant question would be: Is his golfing performance better than that of an individual selected at random from the population at large? Ganzfeld experiments will never pass muster as nightclub entertainment, but if over time the results can beat odds of one in a million, they aren't random.
 
I've heard several times on these forums about the "1 in one thousand usual MDC standard for a preliminary test." ......So what, exactly, is required to pass the preliminary test?

The longish arguments on statistics are irrelevant.

Those who can prove or demonstrate the psychic or occult will not apply because under the rules of their status they cannot indulge, or mystify, or entertain a stranger, the merely curious, or the skeptical; and especially not for money.

Those who do apply will of course be those who cannot demonstrate the psychic or occult.

It is also highly unlikely that Mr. Randi actually knows or truly believes that psychic or occult events are necessarily fraud or impossible: It is all a sham and word-game.
 
The original purpose of this thread, was to discuss a particular type of claim -That in which only slightly above chance effects are claimed, so probability in this case becomes an overiding factor.

No it wasn't. Try going back and reading the OP again. Then try saying something that's actually relevant.

Only to those who, like you, have the simplistic view that the paranormal cannot be real unless it operates in an always on manner.

No. This is exactly what we have explained, and which you either refuse to understand or simply cannot. The whole point of the challenge is to test what people claim. If they claim to be 100% accurate, that is what is tested. If they claim to be 51% accurate, that is what is tested. The test is designed so that there is a very low chance of them passing by chance, but a very high chance of passing if they can actually do what they claim. Exactly what is meant by "very low" and "very high" is pretty much irrelevant, and the claim itself is completely irrelevant.

However, this whole thread is irrelevant since, as far as I know, no-one has ever applied with a claim of 51% accuracy. In fact, most applicants claim 100% accuracy and have to be persuaded to accept only something like 80% as a success - the JREF actually goes out their way to take into account that 100% accuracy is unlikely, thus demonstrating that your point is, in fact, pointless.
 
The original purpose of this thread, was to discuss a particular type of claim -That in which only slightly above chance effects are claimed, so probability in this case becomes an overiding factor.
No it wasn't. Try going back and reading the OP again. Then try saying something that's actually relevant.

Crikey! Very nice! You sound like a teacher!

In one of the very early posts, Rodney says -

I think your answer helps explain why there are so few non-delusional challengers for the prize. The idea that, for example, anyone can ALWAYS paranormally pick out a red card from a selection of face up cards is preposterous on the face of it (so to speak ;)). However, it may be possible for someone to pick out a red card at a significantly above chance rate, even if that rate is only 51-52% over thousands of trials. Would such a person apply for the prize, when there is not even a hint in the official rules of what level (s)he must perform at even to pass the preliminary test, let alone win the million dollars? The point is that, by not specifying an odds standard, serious challenges for the prize are discouraged. As a person involved in Ganzfeld experiments told me, the MDC is "so hedged and the criteria for success so arbitrarily set up and changeable at his [Randi's] whim that nobody will ever be able to pass his test."

So, as can be plainly seen, discussion of the odds standard, in relation to discouraging 'slightly above chance' claims. is clearly what this thread is about
'
It would help to say why you think something is irrelevant - To just say 'irrelevant', without giving a reason, comes over as rude - The whole point of dialogue, is to 'respond' to others' points, rather than just dismiss them as irrelevant

Only to those who, like you, have the simplistic view that the paranormal cannot be real unless it operates in an always on manner.
No. This is exactly what we have explained, and which you either refuse to understand or simply cannot.
(Bold added)
Indicates impatience. Being personal doesn't make for good dialogue.

The whole point of the challenge is to test what people claim. If they claim to be 100% accurate, that is what is tested. If they claim to be 51% accurate, that is what is tested. The test is designed so that there is a very low chance of them passing by chance, but a very high chance of passing if they can actually do what they claim. Exactly what is meant by "very low" and "very high" is pretty much irrelevant, and the claim itself is completely irrelevant.
(Bold added)

What is the point of saying eveything is irrelevant, without backing up that. It comes over as terribly opinionated.
The arguments have been carefully put forward, and to just say that everything is irrelevant, without giving the slightest reason, doesn't make for a very productive dialogue.
Giving another side, or disagreeing is alright, but just contradicting for no reason, and telling people that what they say is irrelevant, seems rude

However, this whole thread is irrelevant since, as far as I know, no-one has ever applied with a claim of 51% accuracy.

There you go again, saying things are irrelevant!

The argument given, was that this type of claim is discouraged -
Otherwise they might apply
 
Last edited:
Ganzfeld experiments will never pass muster as nightclub entertainment, but if over time the results can beat odds of one in a million, they aren't random.

And yet, if they cannot produce results that are significantly distinguishable from those derived from random chance, they are also not relevant.

However, I would certainly encourage the Ganzfeld researchers to apply. Formulate a concept of what they intend to demonstrate, why it will be significantly more than random chance, and then _do it_. The spotlight scrutiny focused on them will prove highly entertaining, I am sure.
 
What is the point of saying eveything is irrelevant, without backing up that.

OK, try again. Go back and actually read my post. See the one you quoted that says "Cuddles" on it? Good, that's the one. Now, you see all the words there that aren't the word "irrelevant"? Excelent. Those would be the ones explaining why the word "irrelevant" is there. Right, now you've managed all that, try actually addressing the points I have made instead of just complaining that you can't understand my posts if you don't read them.
 
However, this whole thread is irrelevant since, as far as I know, no-one has ever applied with a claim of 51% accuracy.
You might try re-reading post #6 on this thread, in which I stated that "it may be possible for someone to pick out a red card at a significantly above chance rate, even if that rate is only 51-52% over thousands of trials. Would such a person apply for the prize, when there is not even a hint in the official rules of what level (s)he must perform at even to pass the preliminary test, let alone win the million dollars? The point is that, by not specifying an odds standard, serious challenges for the prize are discouraged."
 

Back
Top Bottom