Merged Odds Standard for Preliminary Test

[pavel_do]

I know it is out of topic.. but still would like to post it..
There is a web site offering a phone readings to there clients.. and there is a top pay “psychic” that charges 20$ per MINUTE… I am shocked.. Nevertheless.. here is one of the comments.. from the client.. that made me laugh so I would like to share it with you.

“He started by saying things were really good for me & he was going to tell me why they were. He repeated that statement 3 times and then it sounded like he turned on the water and stepped into the shower--while still trying to talk to me. When I told him I could no longer hear him he said he would speak louder. Louder wasn't the problem--he sounded all garbled like he was drowning. Total waste of $$$$. If you don't have the time to do a proper reading don't be available for calls.”

 

[steenkh]

Exactly for that reason, 1 passed test can not show that person did not achieved it by luck.. same as if he fails, it does not mean he did not failed it by chance.. You need to have 2-3 test and if you repeat results.. that will be obvious that either you do have the ability or that just was a luck.. Just as Rodney have written in his post..

No, that is not correct. The success levels for the tests are designed to prove with very great certainty that you will not pass through sheer luck. You and Rodney's demand for successive tests just raises the chance of a false positive, which is, of course to your advantage.

If you have doubts about your ability to achieve 30 out of 40, you should have made that clear from the beginning. It is not up to the JREF to specify what you can do or not do. Their concern is to be sure that what you can do is not achieved by sheer luck, and it would be counterproductive to lower the odds or give you extra chances.

anyway, as the rules state, you can be retested in a year, which means that you actually do have a second chance. Unfortunately, the MDC is going to be withdrawn, so you will probably only manage a single retest.

 

[Moochie]

No, and if you read my post more carefully, you will see that what I was suggesting was continuing the preliminary test to someone who beat such odds. For example, let's suppose Pavel gets 26 out of 40 (65% hit rate vs. the expected 50%) in his preliminary test. The odds of doing that by chance are about 1 in 25. So why not retest him to see if he can repeat that result? If that were done and he performed only at a chance level, that would be the end of the testing. However, if he did repeat it, he would now have achieved 52 hits in 80 trials, which would happen by chance only about 1 in 200 times. So why not continue to test him? Again, if he performed only at a chance level, that would be the end of the testing. However, if he repeated that result again, he would now have achieved 78 hits in 120 trials, which would happen by chance only about 1 in 1,500 times. So finally, he would have beaten odds of better than 1 in 1000 and passed the preliminary testing. However, that would still not win him any money, but only qualify him for the final testing.


If demonstrating a paranormal ability were as simple as riding a bicycle, even Randi would now be convinced that there is such a thing as the paranormal.

Really think you're jumping the gun here, Rodney. Pavel hasn't achieved 26 out of 40, let alone anything better. When he has achieved this (if he achieves this) perhaps then would be the right time for special pleading.


M.

 

[Moochie]

Your aspirin anylogy seems flawed. Nevermind.

In all brevity - since we have very differing viewpoints and a discussion seems pointless - the "odd(s) discussion" will only be brought up by people like Pavel: People who seem to (for one part) modeling their claims around the required minimum odds of the MDC success criteria.

I understand the pragmatic part of that, obviously.

As do I. It's a crap-shoot.


M.

 

[Startz]

Some random thoughts

1. The idea of having to do two tests isn't exactly new. That's what JREF does. If one passes the preliminary, then one has to do it again to show it wasn't luck. (And I imagine for JREF to get another chance to look for cheating.)

2. The probability of passing the proposed 30 out of 40 test by random chance (false positive) is about 1 in 1,000. The chance that someone who has a 65 percent success rate will pass the test is only about 12 percent. The false negative rate would be 88 percent. (Mind you, that would depend on whether paranormal activities obey the usual rules of statistics.) To have a lower rate for the false negative would require longer testing or a different design.

3. It just isn't true that the applicant states an ability and JREF tests it under controlled conditions. There really is a negotiation. For example, Pavel made an initial proposal that was perfectly appropriate statistically. JREF declined to conduct a test according to his suggestions because they felt the protocol was too complicated. I'm willing to accept JREF's judgment because they know a lot more about this stuff than I do. More to the point, Pavel has also been willing to accept JREF's judgment in order to move the whole thing forward. Pavel has always said that because his power is imperfect he needs a somewhat longer test than JREF is willing to accommodate.

I realize that there have been many who have made ridiculous proposals and then whined when JREF wouldn't go along. But that's not what's happened with Pavel.

 

[steenkh]

3. It just isn't true that the applicant states an ability and JREF tests it under controlled conditions. There really is a negotiation. For example, Pavel made an initial proposal that was perfectly appropriate statistically. JREF declined to conduct a test according to his suggestions because they felt the protocol was too complicated.

As part of the negotiation, the claimant must stick to his guns if he knows his abilities. The reason why the JREF is dead against complicated protocols is that the more complicated, the more out of control they get.

For some reason which is less clear to me, the JREF is also struggling to keep down costs, even though it is clearly stated that all such costs would have to be paid by the claimant. They could theoretically accept long-winded protocols, even if this meant prolonged rental of facilities, payment of testers and security people to ensure that no one tampers with the test results or test object outside testing hours. But they do not.

 

[drkitten]

For some reason which is less clear to me, the JREF is also struggling to keep down costs, even though it is clearly stated that all such costs would have to be paid by the claimant.

The reason for this is that, even with costs being borne by the claimant, resources are still in short supply. Randi's time is limited to approximately 168 hours per week, and even less if you grant that he likes to sleep and eat. Most of the various tests are actually run and staffed by volunteer labor, who are paid in Randi's goodwill, and many of the participants are experts who are themselves busy.

Speaking only for myself, for example --- I would be happy to take an unpaid day out of my schedule to participate in a Challenge examination if my specialist skills would be useful. (I routinely do that for other scientific events such as science fairs.) I would be less happy taking a week (I have classes to teach, but I could probably arrange something if necessary). Taking a month would be right out. Even a paid month would be unacceptable; my students have to come first.

So, again speaking only for myself -- you can have six or eight hours in a good cause for free, but forty hours would cost you several thousand dollars, and two hundred hours is probably unavailable at any price you could afford to pay.

 

[petre]

No, and if you read my post more carefully, you will see that what I was suggesting was continuing the preliminary test to someone who beat such odds. For example, let's suppose Pavel gets 26 out of 40 (65% hit rate vs. the expected 50%) in his preliminary test. The odds of doing that by chance are about 1 in 25. So why not retest him to see if he can repeat that result? If that were done and he performed only at a chance level, that would be the end of the testing. However, if he did repeat it, he would now have achieved 52 hits in 80 trials, which would happen by chance only about 1 in 200 times. So why not continue to test him? Again, if he performed only at a chance level, that would be the end of the testing. However, if he repeated that result again, he would now have achieved 78 hits in 120 trials, which would happen by chance only about 1 in 1,500 times. So finally, he would have beaten odds of better than 1 in 1000 and passed the preliminary testing. However, that would still not win him any money, but only qualify him for the final testing.


If demonstrating a paranormal ability were as simple as riding a bicycle, even Randi would now be convinced that there is such a thing as the paranormal.

If the applicant only gets 26 out of 40 correct on the preliminary test, then why did the applicant claim the ability to get more correct in the protocol? Is the applicant not familiar enough with their own ability to make solid claims? The JREF challenge isn't for researching an ability, it is for testing one which the applicant should be comforable enough to define boundaries for.

Do you disagree that an applicant that cannot perform as they claim has not researched their ability enough before applying for the challenge?

 

[Rodney]

If the applicant only gets 26 out of 40 correct on the preliminary test, then why did the applicant claim the ability to get more correct in the protocol? Is the applicant not familiar enough with their own ability to make solid claims? The JREF challenge isn't for researching an ability, it is for testing one which the applicant should be comforable enough to define boundaries for.

Do you disagree that an applicant that cannot perform as they claim has not researched their ability enough before applying for the challenge?

Did you read Post #147 on this thread? There, Pavel states that he has averaged 28 hits per 40 trials, which equals only a 70% hit rate -- not the 75% hit rate that the JREF has specified in its protocol. When considering the pressure of the test, I think specifying a protocol involving a somewhat lesser hit rate than the claimed average makes sense. In any event, what could possibly be the justification for specifying a higher hit rate, other than the benefit to the JREF of a short test that the applicant is likely to fail?

 

[steenkh]

The reason for this is that, even with costs being borne by the claimant, resources are still in short supply. Randi's time is limited to approximately 168 hours per week, and even less if you grant that he likes to sleep and eat. Most of the various tests are actually run and staffed by volunteer labor, who are paid in Randi's goodwill, and many of the participants are experts who are themselves busy.

Of course, but experts can still be hired, and if a test can only be done by properly paid experts, this should be made clear to the claimant, and if s/he cannot carry the costs, the claim has to be rejected because of this. It seems wrong to force the claimant to rely on chance in order to keep down costs.

On the other hand, as we very well know it, these claimants are probably all deluded as to their abilities, and it also seems wrong to make them spend all their savings in the chase of a million dollars that they are unable to win.

 

[GzuzKryzt]

Of course, but experts can still be hired, and if a test can only be done by properly paid experts, this should be made clear to the claimant, and if s/he cannot carry the costs, the claim has to be rejected because of this. It seems wrong to force the claimant to rely on chance in order to keep down costs.

On the other hand, as we very well know it, these claimants are probably all deluded as to their abilities, and it also seems wrong to make them spend all their savings in the chase of a million dollars that they are unable to win.

Well said, steenkh.



And let's not forget that the JREF Challenge provides only one of the infinite ways to provide evidence for a so-called paranormal ability.

 

[petre]

Did you read Post #147 on this thread? There, Pavel states that he has averaged 28 hits per 40 trials, which equals only a 70% hit rate -- not the 75% hit rate that the JREF has specified in its protocol. When considering the pressure of the test, I think specifying a protocol involving a somewhat lesser hit rate than the claimed average makes sense. In any event, what could possibly be the justification for specifying a higher hit rate, other than the benefit to the JREF of a short test that the applicant is likely to fail?

Well I hope you'd agree that if an applicant claimed an ability that was successfuly 50.00001% of the time, when 50% is expected by chance, and each trial takes 10 years it is not practical to test such an ability to any degree of accuracy witin even several hundred lifetimes.

There are possible real paranormal abilites that simply cannot be tested in a reasonable time frame. The JREF has chosen a period of 1 day (something around 8 hours) for their threshold of 'reasonable'. If you are able to suggest a test that will allow a 70% hit rate and can be completed within the timeframe suggested, it may be of assistance in that protocol discussion. If there exists no way for the ability to be tested within that time, then it is simply not within the scope of the challenge.

 

[Moochie]

Well said, steenkh.



And let's not forget that the JREF Challenge provides only one of the infinite ways to provide evidence for a so-called paranormal ability.

Yes. As many have stated here before, claimants are at complete liberty to exhibit their "ability/ies" under proper observing conditions wherever and as often as they please. In fact, it would be damned helpful if they did, but they never do, do they?

I don't apologize for my skepticism regarding the present claimant -- all he's provided is some claim that he has been "tested" somewhere. Surely, given his claimed ability, he'd have a piano-roll of testimonials from accredited sources by now. Let him replicate his claimed successes before the scientific community a few times. At present all I see is a none-too-clever attempt to beat the odds.


M.

 

[Moochie]

Well I hope you'd agree that if an applicant claimed an ability that was successfuly 50.00001% of the time, when 50% is expected by chance, and each trial takes 10 years it is not practical to test such an ability to any degree of accuracy witin even several hundred lifetimes.

There are possible real paranormal abilites that simply cannot be tested in a reasonable time frame. The JREF has chosen a period of 1 day (something around 8 hours) for their threshold of 'reasonable'. If you are able to suggest a test that will allow a 70% hit rate and can be completed within the timeframe suggested, it may be of assistance in that protocol discussion. If there exists no way for the ability to be tested within that time, then it is simply not within the scope of the challenge.

Really? Care to submit some evidence of this? No one's stopping claimants, or anyone else for that matter, from being tested under proper observing conditions (ie, where all opportunity to cheat is eliminated), yet none of the claimants attempting the MDC have done so. The miscreants (Browne, Edward, et al) who make megabucks out of diddling the deluded simply don't care about the MDC -- their income is many multiples of a million.


M.

 

[Startz]

snip

I don't apologize for my skepticism regarding the present claimant -- all he's provided is some claim that he has been "tested" somewhere.

M.

snip

If I'm reading correctly that by "present claimant" you mean Pavel, note that he has provided an affidavit that his claims have been witnessed by two academics and that affidavit has been found acceptable by JREF.

 

[Moochie]

snipsnip

If I'm reading correctly that by "present claimant" you mean Pavel, note that he has provided an affidavit that his claims have been witnessed by two academics and that affidavit has been found acceptable by JREF.

"Two academics" is too broad a brush for my liking. I know several "academics" who reckon there's "something" to astrology. You now, extraordinary claims require extraordinary evidence. We never, ever, get such evidence. What's preventing "Pavel," et al from producing such evidence?


M.

 

[Startz]

"Two academics" is too broad a brush for my liking. I know several "academics" who reckon there's "something" to astrology. You now, extraordinary claims require extraordinary evidence. We never, ever, get such evidence. What's preventing "Pavel," et al from producing such evidence?


M.

Here's what RemieV posted in the official Challenge Application section.

For an academic witness, he provided an affidavit from the University of North Carolina Psychology Department with detailed information on a test they ran of his abilities.

I'm hoping that we'll soon get high quality evidence in the form of a JREF approved test.

 

[Moochie]

Here's what RemieV posted in the official Challenge Application section.


I'm hoping that we'll soon get high quality evidence in the form of a JREF approved test.

Well, we all know the "academics" who were totally convinced of Uri Geller's shenanigans. I don't expect these academics to be worth the paper their qualifications are written on. In fact, I think it would be a good idea to check their bona fides, if this hasn't already been done.

In any case, none of the claimant's "credentials" ought to be accepted without thorough scrutiny. The claimant's bona fides must be transparent, IMO.

I don't know why it must be repeated, but apparently it does: All aspects of the transactions between claimant and issuers of the MDC must, perforce, be 100% transparent. There simply can be no room for doubt on the part of either party if the proceedings are to have any credibility in the wider community.

Pavel would inestimably aid his claim if he had bothered to seek and establish a consistent record of successfully demonstrating his "ability." That he has not done so speaks volumes.

Hence, I smell a rather redolent rat. I am happy to be proven wrong.


M.

 

[steenkh]

In any case, none of the claimant's "credentials" ought to be accepted without thorough scrutiny. The claimant's bona fides must be transparent, IMO.

You seem to argue that the MDC which was designed to challenge supernatural claims, should not actually accept any claimants unless they have already proven beyond doubt that they would be able to win the challenge?

As far as I know, the "two academics and a journalist"-hurdle was set up in order to keep the mentally ill away, not in order to actually provide evidence for the paranormal.

 

[drkitten]

As far as I know, the "two academics and a journalist"-hurdle was set up in order to keep the mentally ill away, not in order to actually provide evidence for the paranormal.

Yeah.

Another way of looking at it is that if you're so messed up that you can't convince two academics and a journalist of your abilities (given how gullible academics and journalists are known to be), the JREF is not the place for you.

The problem is (and you can check the records on-line) that most of the claimants WERE sufficiently messed up that it seemed that the JREF was running an outpatient clinic, not an educational foundation.