Case Study: The IIG Protocol for VFF

[Uncayimmy]

Suddenly, once she focused on testing this particular claim, barriers that would block her from viewing people blocked the power (even if it was made out of the same material as the shirts she is seeing through). Dozens of protocols that she would be able to pass if she really had this power were proposed but she refused all except the protocol that was most expensive and involved the most people - and coincidentally had the most room for her lucky guesses.

This is why I am not so sure that she really believes she has powers as some people think. At some level she is making moves to avoid any test that is simple and airtight in terms of its results.

I don't disagree that Anita is to paranormal claims as Edison was to patents, but let's try to stay away from that in this thread except in the context you set out below.

So what are groups like the IIG to do? On the one hand they could refuse people pulling things like this, but in doing so Anita would claim the martyr mantle and use this refusal to spin that she really does have powers. On the other hand, we have tests within the questionable confines set out by the applicant, as we did here, which still prove what we already know - that Anita does not have powers - but have enough holes to let people like her spin away and find ways to claim victory.

If you look at the three broad groups of people before the test - True Believer, Non-Believer, and Undecided -and then check them after the test and all the spinning, what kind of movement was there, if any? I am very confident that there was a shift of from Undecided to Non-Believer. I think her efforts to "claim victory" actually resulted in more of the Undecided to move and probably made some of the True Believers cringe.

It seems to be no matter what happens, Anita wins in terms of getting the publicity she seeks. As long as she refuses to submit to testing conditions that are more rigorous, this is always going to happen.

I disagree that this "publicity" was positive for her. Can you name any up and coming psychics who used a failed challenge like this as a springboard? How about an existing psychic?

In my opinion the only person who successfully used a challenge to promote himself was The Professor. He played the skeptics here to perfection and managed to bring attention to himself within the same target audience he already appeals to. The fact that he did *not* take the test is what made it a success.

It sounds good to say that no test is better than a bad test, but I have yet to see any solid arguments that it's really the case. Failed tests, excuses, and transparently false claims of victory mean the challenge was successful as far as I'm concerned.

 

[Cuddles]

Who cares? What difference does it make? She talks to ghosts, too. None of those things will be confirmed or denied by the challenge because the challenge is not testing the claims on her website.

No, but the challenge was supposedly testing her claim to see inside people. Identifying which person out of six has a kidney missing was not the test of that, identifying exactly which kidney out of a possible 12 was missing was the test. I agree entirely with Joe that the wording "The Applicant claims to be able to detect which Subject in a group of six Subjects is missing a kidney" is misleading. That is not what she claims to do at all. Either that should not have been there, or it should have been put much more clearly.

If it is not disallowed, then it is permitted. Both sides agreed to it. I have no idea why you think that's a problem.

That's a rather odd attitude to take. You have said yourself that you don't like parts of the protocol. But obviously those parts were agreed on by both sides, so I have no idea why you think they were problems, right? No, obviously not right, given that the entire point of this thread is to discuss problems people have with the protocol. As it is, several people think that IIG should not have agreed to the protocol for various reasons, and no-one really cares what Anita agrees to because she's clearly incapable of recognising a decent protocol if it dances in front of her in a fluorescent jacket waving a big banner saying "I'm a decent protocol". Of course, that's the case with many paranormal claimants, which is why it's important that the people who supposedly know what they're doing actually get it right.

I agree that this is actually a very big problem here. One of the biggest issues Anita has had is with determining what is acceptable in terms of clothes, screens and the like. Yet the protocol leaves it completely up to the participants to decide what to wear. One of the critical parts for both having the claimed ability work and for preventing information leakage is left up to people who have actually deliberately had the details of the test and protocol kept from them. How can that not be a problem?

No one has yet to come up with a known method by which she could have gotten all three right.

I disagree. In fact, we have very good evidence that information on at least two of the subjects in the test was available to Anita. We have absolutely no idea if information on the others was available, but we can't rule out the possibility. The way the volunteers were chosen ensures that not only was there the possibility of information leakage, but also that it would be virtually impossible to know if such leakage actually occurred.

I think this is probably the biggest problem the test had. I understand that it may be difficult finding volunteers, especially when you need them with a particular rare condition. However, the fact remains that not only was it possible for the blinding to be broken, we know for a fact that it was broken. Personally I doubt that Anita used the information available and it was either just a lucky guess or used visual clues available during the test, but that's not the point. That protocol allowed information on the volunteers to be available to the claimant is a serious problem.

There is no science involved in these challenges anymore than there's science involved in my camping scenario.

I think you are using too narrow a definition of science. The scientific method is essentially just this:
1) Observe world;
2) Create hypothesis to explain observations;
3) Observe some more, preferably in a more controlled manner;
4) Check if new observations match hypothesis;
5) Goto 2

Tests such as this are very much science. The claimant has done steps 1) and 2). Most people think they have done them badly and that the hypothesis is clearly incorrect, but that is not relevant. Tests such as the JREF and IIG tests constitute part 3), and the success or failure of the claimant is part 4). Sure, it's not cutting edge research that's going to send us to other stars or cure cancer, and it may often be done to lower standards than might be expected of professionals, but it still very much follows the scientific method. See what happens, try to explain it, see if the explanation actually works. Repeat. That's all science is.

Edit: As for whether a bad test is worse than no test, I'm not sure there's really a clear answer to that. In this case, all the problems people see in the protocol make it easier to pass, yet Anita still failed. That should mean that it's not a problem at all, since it shows that even given extra help, her superpowers still obviously don't work. However, it does also open the floor for accusations of unfairness. Believers could claim that it is unfair not to do a retest since if she had passed, the problems mean we would likely have dismissed the results due to the flaws. It's silly, because it's not a symmetric situation and flaws that could mean a pass was meaningless do not similarly mean a fail is, but it would be better not to allow that kind of thing to be said at all. Sure, some people will always find something to make excuses about, but that doesn't mean we should make it easy for them. If we see a potential excuse we should try to cover it in advance. At least make the believers work for their excuses, and force them to take the weaker ones.

 

[wardenclyffe]

SNIP

In this case, all the problems people see in the protocol make it easier to pass, yet Anita still failed. That should mean that it's not a problem at all, since it shows that even given extra help, her superpowers still obviously don't work. However, it does also open the floor for accusations of unfairness. Believers could claim that it is unfair not to do a retest since if she had passed, the problems mean we would likely have dismissed the results due to the flaws.

SNIP

Of course, if she had passed, she would have gotten a retest for the money. Even if skeptics complained that it was a flawed test that allowed her to pass, she still would have earned her retest.

Ward

 

[Uncayimmy]

No, but the challenge was supposedly testing her claim to see inside people. Identifying which person out of six has a kidney missing was not the test of that, identifying exactly which kidney out of a possible 12 was missing was the test. I agree entirely with Joe that the wording "The Applicant claims to be able to detect which Subject in a group of six Subjects is missing a kidney" is misleading. That is not what she claims to do at all. Either that should not have been there, or it should have been put much more clearly.

What was described in the claim section was what Anita was agreed to do in the protocol. It was unnecessary to even have that in there. I liked it because it made the protocol easier to follow. The JREF doesn't do that, so when you read the protocol, it's confusing to try to understand what's supposed to happen.

That's a rather odd attitude to take.

Not at all. It's basic contract law.

You have said yourself that you don't like parts of the protocol. But obviously those parts were agreed on by both sides, so I have no idea why you think they were problems, right?

I explained what I did not like and why. I also acknowledged the logistical difficulty in doing what I suggested.

Saying "the subjects may wear hats" does not preclude all of the subjects wearing identical hats or even none of them wearing hats. The protocol is negotiated before you have all or even any of the subjects. You don't want to get down to the wire and find you have some lady with a beehive hairdo who refuses to wear a hat, so you leave yourself some flexibility. That's just being prudent.

I agree that this is actually a very big problem here. One of the biggest issues Anita has had is with determining what is acceptable in terms of clothes, screens and the like. Yet the protocol leaves it completely up to the participants to decide what to wear. One of the critical parts for both having the claimed ability work and for preventing information leakage is left up to people who have actually deliberately had the details of the test and protocol kept from them. How can that not be a problem?

What is not explicitly or implicitly excluded is by definition acceptable. Again, this is basic contract law (or even legislation). There are phrases you can use to easily accomplish this. In this protocol it said, "Each Subject will wear a light-weight cotton shirt or blouse with no pattern." This implicitly excludes dresses, swimwear, jackets, coats, and so forth. At the same time it allows for flexibility should the need arise.

I disagree. In fact, we have very good evidence that information on at least two of the subjects in the test was available to Anita. We have absolutely no idea if information on the others was available, but we can't rule out the possibility. The way the volunteers were chosen ensures that not only was there the possibility of information leakage, but also that it would be virtually impossible to know if such leakage actually occurred.

I said, "No one has yet to come up with a known method by which she could have gotten all three right." Let me amend that for clarity. Nobody has pointed out a known method which was possible for her to have used. Obviously she could have cheated, but she didn't. One of the IIG reps could have placed a big red X on the target because *that* wasn't excluded in the protocol, but that didn't happen.

How the IIG gathers people is entirely up to them and should not be part of the protocol anymore than it should be a requirement that Anita use her sooper powers. If she wanted to guess, that's her choice. If the IIG wanted to use all of Anita's friends and family as subjects and Dr. Carlson as one of the targets, that's their choice. We rely on the fact that the IIG wants to keep it's $50K and that Anita wants the $50K to ensure proper steps are taken.

The IIG does not owe it to you or anyone else to explain how they found their subjects.

That protocol allowed information on the volunteers to be available to the claimant is a serious problem.

It's only a serious problem if it puts the $50K at serious risk. I don't think it even came close to do that level.

I think you are using too narrow a definition of science. The scientific method is essentially just this:
1) Observe world;
2) Create hypothesis to explain observations;
3) Observe some more, preferably in a more controlled manner;
4) Check if new observations match hypothesis;
5) Goto 2

Tests such as this are very much science.

Using the scientific method does not make something science. What you describe is how a toddler learns that a pan is hot or that bees sting. Hell, it's essentially how a rat figures out how to get a pellet of food.

I don't call that science because it doesn't add to our body of scientific knowledge. No scientist is going to rely on this publicity stunt. It's not going to get into a textbook.

The claimant has done steps 1) and 2). Most people think they have done them badly and that the hypothesis is clearly incorrect, but that is not relevant.

I agree its irrelevant, but I suggest that most claimants reverse steps 1 and 2. That is, they form a hypothesis ("I'm special") and look for observations to support that premise.

Believers could claim that it is unfair

I don't care about believers because they don't care about science. Let them do their own tests. I don't care about scientists because they know the difference between a publicity stunt and research. I care about the Average Joe seeing yet another person making paranormal claims failing under controls they agreed to and then making a series of ridiculous excuses.

What do you think the purpose is of these challenges? The IIG knows they are publicity stunts. So does the JREF. They know believers won't be influenced by them. They know scientists thinks it's an amusing sideshow. Everything the IIG and JREF has done as far as I can see can be explained by these simple premises.

 

[LightinDarkness]

I disagree that this "publicity" was positive for her. Can you name any up and coming psychics who used a failed challenge like this as a springboard? How about an existing psychic?

In my opinion the only person who successfully used a challenge to promote himself was The Professor. He played the skeptics here to perfection and managed to bring attention to himself within the same target audience he already appeals to. The fact that he did *not* take the test is what made it a success.

It sounds good to say that no test is better than a bad test, but I have yet to see any solid arguments that it's really the case. Failed tests, excuses, and transparently false claims of victory mean the challenge was successful as far as I'm concerned.

All publicity is good publicity when you are catering to the woo audience. Remember, to woos the JREF and groups like the IIG represent something horrible which seeks to stop them from being special. That Anita went up against them will win her favor with the woos because of this, and because the woos are true believers little facts like her test failures will not matter. Although this test confirmed to the normal world what we already know - that Anita does not have superpowers - it will simply increase her credentials with the true believers.

I would find it hard to believe that anyone at this point - even before the IIG failure - could consider Anita as having real powers. Anyone with the real powers she claimed could have been easily tested with simple protocols suggested a year ago that would provide very concrete proof.

So what did this protocol do? Confirm the obvious and boost her credentials to the woos. Is that worth it? I am not sure yet. If Anita had agreed (as she said she would) that failing this test falsified her claim to xray superpowers, probably so (in my opinion). But obviously, she backtracked on that.

The problem with testing the paranormal is that, at some level, it requires the claimant to be genuine in their belief that they have paranormal powers and be genuine in their attempts to test it. Claimants who know better and are actively working to spin the odds and demand test conditions that allow for warm readings mess up that process and results in less-than-perfect tests.

 

[bookitty]

A small side note: I only know of two "undecideds" from Anita's test. One was the gentleman in the audience who thought there was significance in a coincidence outside the protocol. The other was my friend, who believes that there is "stuff" out there that we don't understand but who cares because real life is more important.

Obviously the first gentleman thought he saw something. It would be interesting to meet up with him now and see how he perceives the test. My friend still believes that we don't know everything but doesn't think Anita has the power to see into people. (Although he seemed to think that she believed it.)

I would be very curious see the results of a before/after questionnaire for any test that had a larger audience made up of those outside the skeptical community.

 

[Uncayimmy]

All publicity is good publicity when you are catering to the woo audience.

Evidence? I seriously doubt there is a single True Believer who does not believe that there are at least *some* frauds or innocently mistaken people. If you confronted the staunchest believer you know and said, "I can predict which card I will draw from a deck of cards" and proceed to fail five times in a row, will they still believe you? If so, they are hopeless anyway.

So, where is the evidence for claimants failing a challenge and using it as a springboard to successfully promote themselves in the woo community?

So what did this protocol do? Confirm the obvious and boost her credentials to the woos.

Again, where's your evidence that her credentials are boosted?

The problem with testing the paranormal is that, at some level, it requires the claimant to be genuine in their belief that they have paranormal powers and be genuine in their attempts to test it. Claimants who know better and are actively working to spin the odds and demand test conditions that allow for warm readings mess up that process and results in less-than-perfect tests.

Where is the evidence that this is a problem? Actually, just start with describing what exactly the problem is. By that I mean if they take the test and fail, how is the world worse off than if they didn't take the test at all?

I say that if someone approaches the likes of the IIG for a test and makes demands that seem reasonable to a layperson (not a skeptic or a True Believer) and the IIG refuses to concede because they want a "perfect" test, they look like cowards.

I also say it's much, much easier to build your credentials in the woo community by making unsupported claims and sharing anecdotes. A failed test works against the person and no test is merely neutral.

 

[Cuddles]

Not at all. It's basic contract law.

I have no idea what your point is here. The point that I, and Joe, are making has absolutely nothing to do with contracts or law. We are talking about constructing protocols, not the legalities of interpreting them afterwards. Saying that because this protocol was agreed it must be right because it was agreed to is just plain stupid.

I explained what I did not like and why. I also acknowledged the logistical difficulty in doing what I suggested.

Yes, and don't you find it just a little hypocritical to voice your own opinion on flaws in the test, but dismiss any others by saying that since they were agreed to it must be fine? Either flaws can be discussed and your bringing law into it is a pointless red herring, or they can't and your own OP should be dismissed according to your own argument.

Saying "the subjects may wear hats" does not preclude all of the subjects wearing identical hats or even none of them wearing hats. The protocol is negotiated before you have all or even any of the subjects. You don't want to get down to the wire and find you have some lady with a beehive hairdo who refuses to wear a hat, so you leave yourself some flexibility. That's just being prudent.

That's not being prudent, it's being extremely stupid. You cannot leave flexibility when it comes to things like blinding. That's the whole point of having a protocol. If you want to make sure there are no visual clues and everyone looks the same, you have to state that there are no visual clues and that everyone looks the same, you can't just leave it up to people to wear whatever the hell they feel like.

What is not explicitly or implicitly excluded is by definition acceptable.

No, it is not acceptable. Again, we are not talking about your nonsensical derail about law, we are talking about protocol design. By your argument a protocol that said "The claimant shall be given the correct answers beforehand" would be acceptable just because everyone agreed to it.

In this protocol it said, "Each Subject will wear a light-weight cotton shirt or blouse with no pattern." This implicitly excludes dresses, swimwear, jackets, coats, and so forth. At the same time it allows for flexibility should the need arise.

There can be no need for flexibility. The whole point of a protocol is to ensure all the conditions are defined exactly, so that there can be no mistakes and no wriggle room.

I said, "No one has yet to come up with a known method by which she could have gotten all three right." Let me amend that for clarity. Nobody has pointed out a known method which was possible for her to have used.

Now you're just being dense. Of course people have pointed out a known method it was possible for her to have used. You can deny that until you're blue in the face, but it's not going to magically change. The protocol left huge holes available for people (not just Anita) to cheat, and one of those holes can be seen to have actually happened. That it seems unlikely anyone actually took advantage of it does not mean it was not there.

There were clearly several big flaws with it. That none of these flaws appear to have resulted in any problems does not make them disappear. You started this thread supposedly to discuss the protocol, not the results. My concerns address the protocol. I don't really care, for this discussion, what did and didn't happen. What matters is what could have happened.

Obviously she could have cheated

And you don't see that as a problem? The protocol doesn't eliminate some very obvious, easily eliminated, ways of cheating, and that's absolutely fine just because it probably didn't happen?

but she didn't.

You have no idea that that is the case. It seems unlikely, but it is very much possible, and the fact that the only one she got correct was in fact the one for which we know information was available makes it much more than just a theoretical possibility.

One of the IIG reps could have placed a big red X on the target because *that* wasn't excluded in the protocol, but that didn't happen.

And because it didn't happen, it's perfectly OK to allow it to happen? What a strange attitude to take.

How the IIG gathers people is entirely up to them and should not be part of the protocol

Of course it should. Ensuring that no-one present knows who is who is absolutely fundamental to the blinding. As it was, Anita had contact with at least one person, and information was available with at least one more. Who knows what else may have been available if anyone had actually looked. This is exactly why these things need to be part of the protocol and not just made up as they go along.

If the IIG wanted to use all of Anita's friends and family as subjects and Dr. Carlson as one of the targets, that's their choice.

It may be there choice, but it would make it a completely invalid test, unless the blinding was specified sufficiently that she could have no way of knowing who was who. Which it wasn't.

We rely on the fact that the IIG wants to keep it's $50K and that Anita wants the $50K to ensure proper steps are taken.

And clearly we can't rely on that, since the proper steps weren't taken. I really am amazed that your entire argument pretty much boils down to saying that we should trust the IIG absolutely and that since nothing went wrong, nothing could have gone wrong.

The IIG does not owe it to you or anyone else to explain how they found their subjects.

They do if they want to be taken seriously. Of course, given that they've told us exactly how they found their subjects it appears they disagree with you here. The question is not how they found them, it's whether the method was a sensible one. While the method itself is not necessarily a bad one, the controls on who was eventually selected were certainly inadequate.

It's only a serious problem if it puts the $50K at serious risk. I don't think it even came close to do that level.

If Anita had wanted to cheat, she could easily have guaranteed herself a hit on the second round. Even assuming that there was no other information available anywhere else, that's a big increase in her odds of winning by chance. Given that the whole point of a good protocol is to eliminate possibilities like that, I'd say that's about as serious as a problem can get. The ends don't justify the means. Just because the results are what we expect does not make it a good test.

Using the scientific method does not make something science.

Well, see Athons post for some very good stuff on the subject of science. Suffice to say here that I think your definition of science is so narrow as to be completely worthless.

I agree its irrelevant, but I suggest that most claimants reverse steps 1 and 2. That is, they form a hypothesis ("I'm special") and look for observations to support that premise.

Well, veering off topic a little, but I disagree. In general, people will see something that leads them to believe something. They then take all later observations as supporting that, but it's rare, as far as I can see, for someone to become a believer before experiencing something they didn't understand.

I don't care about believers because they don't care about science.

But they do care about PR. If you give believers an opportunity to make themselves look good or skeptics look bad, they will take. If that opportunity can be even partially justified, they can easily sway the views of the average Joes you apparently care about. It's not enough to simply point out that someone failed a test. The average Joe often doesn't understand the testing any more than the believers do. But they certainly do understand unfairness, and given that skepticism hardly has a positive image anyway, it really doesn't help to give the believers more ammunition.

What do you think the purpose is of these challenges? The IIG knows they are publicity stunts. So does the JREF. They know believers won't be influenced by them. They know scientists thinks it's an amusing sideshow. Everything the IIG and JREF has done as far as I can see can be explained by these simple premises.

That's the whole point. It's a publicity stunt. If you end up giving the claimants better publicity than yourselves, that's not a good thing. That is exactly why they always strive to be as fair as possible, and to have the tests as reliable as possible. That's why Carina Landin, for example, is allowed a retest even though she actually did worse on the diaries she says are the ones her ability works on. In that case the protocol wasn't violated, it had just been left open enough in one place to make some complaints seem justified to those who didn't look at it in enough depth. And because the JREF understand that this is a publicity stunt, they understand that it looks much better to admit that things weren't perfect and try again, rather than just declaring that the protocol must have been fine because she failed.

 

[Uncayimmy]

I have no idea what your point is here. The point that I, and Joe, are making has absolutely nothing to do with contracts or law. We are talking about constructing protocols, not the legalities of interpreting them afterwards. Saying that because this protocol was agreed it must be right because it was agreed to is just plain stupid.

It has everything to do with contracts because that's what it was: a contract. It has similarities to a scientific test, but it's not done by scientists, it won't be used by scientists, and scientists won't judge its merits.

It is unequivocally a contract. If either party breached the contract, it would be resolved in a court of law. The language therefore must be viewed first and foremost as the language of business.

Will means that something must be done to consider the contract properly executed. Failure to do a will item can be considered a breach. If the breach is deemed a material breach, the court can force certain remedies.

May items are different. Depending on the wording, a may item might simply be an optional item or it might be a restrictive item. To say that the subjects may wear certain types of shirts would reasonably be interpreted as a restrictive item as in "they may only..." wear whatever.

As long as the shirts meet the restrictions given, that's acceptable to the contract. Since more details were not given, neither party has a solid foundation for protest. Having everyone in the same type of t-shirt is not a breach of the contract because the contract did not specify that this was not allowed either explicitly or implicitly.

Cannot/May Not items entirely restrictive. This means if the party does something they are not supposed to do, it's a breach.

Beyond those clauses, just about everything is fair game in a contract. Any disputes are resolved in court by looking at what is ordinary and customary. If that doesn't help, then the argument becomes about what would be reasonably be expected to give both parties a fair opportunity to execute the contract.

It's a challenge not a test, and the protocol is a contract. If there is a difference in interpretation of the protocol, the fact that it's a contract trumps any similarities to a scientific test.

Yes, and don't you find it just a little hypocritical to voice your own opinion on flaws in the test, but dismiss any others by saying that since they were agreed to it must be fine? Either flaws can be discussed and your bringing law into it is a pointless red herring, or they can't and your own OP should be dismissed according to your own argument.

That's a straw man.

That's not being prudent, it's being extremely stupid. You cannot leave flexibility when it comes to things like blinding. That's the whole point of having a protocol. If you want to make sure there are no visual clues and everyone looks the same, you have to state that there are no visual clues and that everyone looks the same, you can't just leave it up to people to wear whatever the hell they feel like.

Your inability to understand the logistics doesn't make the IIG stupid. If the IIG agrees to a will clause and for whatever reason is unable to fulfill that clause, that can be considered a breach of the contract. That opens up the possibility of a lawsuit.

They probably intended all along to have everyone in similar shirts and similar hats. The contract they signed did not preclude them from doing this. What they signed allowed them the flexibility to deal with logistical problems. If they say "identical" shirts and "identical" hats that opens the door for disputes in the definition of identical. Like I said, if they got stuck with some weirdo refusing to wear a hat, they had the flexibility to do a trial without hats. If they signed what you wanted them to sign, they would have risked a breach of contract.

Once again, you are treating the signed protocol like it's a protocol that scientists would describe when they publish a paper. That is simply wrong. It is a contract. It is in the best interest of the IIG to agree to as few will and may not clauses as possible because if for any reason they cannot meet those obligations, they will be in breach of the contract.

This is why the IIG did not describe the method they used to find subjects. I'm positive they took a number of steps to make the test as blind as possible, but they were smart not sign a contract that promised Anita how they would go about this. Why? Because it would be a breach of contract to use any other method. So, if their method failed, they would be stuck. If they found a better method, they would be prohibited from using it.

No, it is not acceptable. Again, we are not talking about your nonsensical derail about law, we are talking about protocol design. By your argument a protocol that said "The claimant shall be given the correct answers beforehand" would be acceptable just because everyone agreed to it.

Repeat after me: It's a contract. It is not a document explaining to the world what ingenious steps they took to construct a reliable test.

You do not know all the behind scenes efforts the IIG took. They have not published anything telling us what they did. All we know is what was written in the contract that Anita and the IIG signed. It spelled out the minimum requirements for each party as well as the restrictions.

They didn't write down a lot of stuff that they did or did they get too specific about what they agreed to do. For example, they said they were going to take photos, but they didn't say if it was a Polaroid or a digital photo printed on a inkjet or color laser printer. They agreed to just enough to satisfy both parties about the identification process.

There can be no need for flexibility. The whole point of a protocol is to ensure all the conditions are defined exactly, so that there can be no mistakes and no wriggle room.

Well, you're wrong. At the time they agree to the protocol and commence the process, they don't know if they can get 18 identical straw hats. Yes, 18, since some people are weird about wearing hats other people have worn, especially strangers. So, while they may intend to get 18 identical heights, they were smart enough not to agree to that in writing.

Now you're just being dense.

I think you need to unclench your butt cheeks.

Of course people have pointed out a known method it was possible for her to have used. You can deny that until you're blue in the face, but it's not going to magically change. The protocol left huge holes available for people (not just Anita) to cheat, and one of those holes can be seen to have actually happened. That it seems unlikely anyone actually took advantage of it does not mean it was not there.

There was no hole big enough for her have gotten all three correct. There were gaps that she might have been able to leverage to increase her odds beyond pure guessing.

There were clearly several big flaws with it. That none of these flaws appear to have resulted in any problems does not make them disappear. You started this thread supposedly to discuss the protocol, not the results. My concerns address the protocol. I don't really care, for this discussion, what did and didn't happen. What matters is what could have happened.

I'm not seeing "big flaws." I see gaps, and I see the logistical difficulties in closing every possible gap. I also see you and others thinking the published protocol is anything more than a contract and ignoring that the IIG was allowed to do anything and everything they could possibly think of as long as it didn't violate the contract.

And you don't see that as a problem? The protocol doesn't eliminate some very obvious, easily eliminated, ways of cheating, and that's absolutely fine just because it probably didn't happen?

The contract specifically says that Anita may not cheat.
"If there is any conscious attempt by the Applicant to cheat or thwart, circumvent or deliberately confuse any aspect of the protocol or final verification process the entire test will be suspended and possibly cancelled altogether."

You have no idea that that is the case. It seems unlikely, but it is very much possible, and the fact that the only one she got correct was in fact the one for which we know information was available makes it much more than just a theoretical possibility.

Fine. Maybe she cheated and got one right. Maybe it really wasn't a 1 in 4 chance.

And because it didn't happen, it's perfectly OK to allow it to happen? What a strange attitude to take.

It's not strange at all. What's strange is how blind you are to the obvious. The IIG is not going to go through all of this effort of putting together a challenge and offering $50K only to put a big red X on the target. That would be stupid on their part. Why should they write into the contract that they are not going to do something really stupid? They could just cut her a check instead.

The protocol negotiation is an adversarial process where the IIG wants to protect their money and prove a point while the claimant is trying to get that money and prove a point.

Of course it should.

I already explained why the IIG should not sign an agreement as to how they should gather the subjects. There is a big difference between taking a prudent course of action and signing an agreement that requires you take a specific prudent course of action.

It may be there choice, but it would make it a completely invalid test, unless the blinding was specified sufficiently that she could have no way of knowing who was who. Which it wasn't.

It's not a test. It's a challenge. And the IIG is not going to give away their $50K by using her friends and family.

And clearly we can't rely on that, since the proper steps weren't taken. I really am amazed that your entire argument pretty much boils down to saying that we should trust the IIG absolutely and that since nothing went wrong, nothing could have gone wrong.

That's not at all what I'm saying. What I am saying is that there is a huge difference between the published protocol, which is actually a signed contract, and the steps the IIG takes to protect their money. Furthermore, their goal is not to prove anything scientific. Its a publicity stunt.

I'm amazed that you don't seem to grasp these simple concepts.

They do if they want to be taken seriously.

Take seriously by whom? Their challenge is not subject to peer review. It's not going into textbooks. It's a publicity stunt where they took the steps necessary to prove their point. They don't need to convince anyone that they did the best blinding possible.

I'm all for discussing the blinding process and offering suggestions and criticisms. However, I don't believe for a moment that they owe it to anyone to explain themselves. I also judge the blinding process on how well it met the goals of the challenge, which is a different standard than proving something to add to the knowledge base we call science.

If Anita had wanted to cheat, she could easily have guaranteed herself a hit on the second round.

How? By reading all of Bookitty's posts and learning that she lived in Los Angeles, and then convincing Bookitty, who really can't stand Anita personally, to tell her what her friend looked like so that Anita would be able to identify him?

Yep, that's possible. I also think it was a mistake to use Bookitty's friend, but at the same time I don't know how hard it was to find subjects.

Well, see Athons post for some very good stuff on the subject of science. Suffice to say here that I think your definition of science is so narrow as to be completely worthless.

That's two straw men. I didn't define science. I gave a couple of examples of how toddlers and rats use the scientific method to point out that the use of the scientific method does not automatically mean science is being made.

But they do care about PR. If you give believers an opportunity to make themselves look good or skeptics look bad, they will take. If that opportunity can be even partially justified, they can easily sway the views of the average Joes you apparently care about. It's not enough to simply point out that someone failed a test. The average Joe often doesn't understand the testing any more than the believers do. But they certainly do understand unfairness, and given that skepticism hardly has a positive image anyway, it really doesn't help to give the believers more ammunition.

Who's giving ammunition to believers? I'm saying that I'm not worried about convincing them.

That's the whole point. It's a publicity stunt. If you end up giving the claimants better publicity than yourselves, that's not a good thing. That is exactly why they always strive to be as fair as possible, and to have the tests as reliable as possible. That's why Carina Landin, for example, is allowed a retest even though she actually did worse on the diaries she says are the ones her ability works on. In that case the protocol wasn't violated, it had just been left open enough in one place to make some complaints seem justified to those who didn't look at it in enough depth. And because the JREF understand that this is a publicity stunt, they understand that it looks much better to admit that things weren't perfect and try again, rather than just declaring that the protocol must have been fine because she failed.

According to Jeff Wagg, "We've now learned that the protocol might not have been followed properly by the testers. It seems that some of the diaries were older than was stipulated by the claimant's protocol."

 

[JoeTheJuggler]

Bit of a derail. . . .

According to Jeff Wagg, "We've now learned that the protocol might not have been followed properly by the testers. It seems that some of the diaries were older than was stipulated by the claimant's protocol."

I still say JREF was being generous on that one, which is certainly their prerogative.

I agree if the protocol wasn't followed, the results had to be scrapped and the test run again.

However, at least in the English translation of this protocol it said that diaries less than 100 years old (or whatever the figure was) should be used "if possible". (That's far from a strict requirement.) The fact that they weren't able to do the re-test with the diaries leads me to think that it wasn't possible to use all younger diaries (because there weren't enough), and therefore the protocol was followed.

 

[wardenclyffe]

UncaYimmy, I'm so glad you're back. You've saved me so much typing.

Ward

 

[Furcifer]

Interesting discussion. I must admit I'm leaning towards UY interpretation of the protocol. It really is a "challenge" not an "experiment".

If I've read everything right, the usual preliminary is set at 10 000:1 odds. Due to limitations by both sides it was agreed to reduce the preliminary odds to some 1800:1. The thing hasn't even begun and protocol is already broken if you want to be a stickler.

"I can shoot a fly off a sandwich at 300 yards, in a cross wind, blind folded, backwards and drunk"

"Here's a rifle, just shoot a bullseye at 100 yards and we'll talk"

That's a challenge. You're egging them on. You know if you put them to it they're going to make excuses, "I'm not drunk enough, this ain't my gun, that's not a sandwich" so you back off just so you can get them to bite.

It's positioning. It's posturing. It's about getting the person to quit talking and start chalking.

There's no reason to get all sciency about this unless the drunk bastard grabs the gun and hits the bull dead nuts center on the first shot.

 

[Uncayimmy]

Interesting discussion. I must admit I'm leaning towards UY interpretation of the protocol. It really is a "challenge" not an "experiment".

If I've read everything right, the usual preliminary is set at 10 000:1 odds. Due to limitations by both sides it was agreed to reduce the preliminary odds to some 1800:1. The thing hasn't even begun and protocol is already broken if you want to be a stickler.

"I can shoot a fly off a sandwich at 300 yards, in a cross wind, blind folded, backwards and drunk"

"Here's a rifle, just shoot a bullseye at 100 yards and we'll talk"

That's a challenge. You're egging them on. You know if you put them to it they're going to make excuses, "I'm not drunk enough, this ain't my gun, that's not a sandwich" so you back off just so you can get them to bite.

It's positioning. It's posturing. It's about getting the person to quit talking and start chalking.

There's no reason to get all sciency about this unless the drunk bastard grabs the gun and hits the bull dead nuts center on the first shot.

Well said.

 

[tsig]

Edit: As for whether a bad test is worse than no test, I'm not sure there's really a clear answer to that. In this case, all the problems people see in the protocol make it easier to pass, yet Anita still failed. That should mean that it's not a problem at all, since it shows that even given extra help, her superpowers still obviously don't work. However, it does also open the floor for accusations of unfairness. Believers could claim that it is unfair not to do a retest since if she had passed, the problems mean we would likely have dismissed the results due to the flaws. It's silly, because it's not a symmetric situation and flaws that could mean a pass was meaningless do not similarly mean a fail is, but it would be better not to allow that kind of thing to be said at all. Sure, some people will always find something to make excuses about, but that doesn't mean we should make it easy for them. If we see a potential excuse we should try to cover it in advance. At least make the believers work for their excuses, and force them to take the weaker ones.

I think paranormal tests actually aid and abet the woo cause.

Those who undergo the tests gain a line on their resume whether they win or lose and get the respect of (some)skeptics for the courage to take the test.

The mere existence of such tests lends credibility to the dowsers, astrologers, ect The fact that no one has won is used as proof they are fixed.

 

[Furcifer]

I think paranormal tests actually aid and abet the woo cause.

Those who undergo the tests gain a line on their resume whether they win or lose and get the respect of (some)skeptics for the courage to take the test.

The mere existence of such tests lends credibility to the dowsers, astrologers, ect The fact that no one has won is used as proof they are fixed.

Bolding mine.

Just don't mix the two. I can have respect for someone and not find them credible. The reverse isn't true, at least if I consider myself a skeptic.

I have the utmost respect for a family friend that's dedicated himself, his life, to serving the word of Jesus Christ. He's given his life's work to something he believes in. That's worthy of respect. If I shared the same convictions about almost anything I'd be further ahead as a person. A good person to boot.

I just don't find him credible. Perhaps even misguided. But who am I to judge?

I don't see anything judgmental in determining credibility. It shouldn't be a subjective process. When it does, it becomes a belief. Isn't that where we draw the distinction between the two?

 

[Uncayimmy]

I think paranormal tests actually aid and abet the woo cause.

By paranormal tests do you mean challenges like the IIG and JREF challenges? Or do you mean paranormal tests conducted by paranormal researchers?

Is your assertion based on the fact the most prominent woos have refused testing and that over the last decade only a handful of obscure people have ever actually gotten to the point of being tested?

Those who undergo the tests gain a line on their resume whether they win or lose and get the respect of (some)skeptics for the courage to take the test.

To whom are they submitting these resumes? Do you have evidence of these people using it to promote themselves?

The mere existence of such tests lends credibility to the dowsers, astrologers, ect The fact that no one has won is used as proof they are fixed.

Again, what evidence do you have of this?

Note: I'm getting deja vu all over again while writing this response.

Before you lay out your evidence supporting you assertions, please consider what I wrote upstream about the three broad audiences: skeptics, True Believers, and fence sitters. I argue that when it comes to True Believers there's nothing that can be said or done to convince them otherwise. Hell, Rodney is now effectively arguing that the "psi effect" is random and uncontrollable and hidden within, yet still separate from, the randomness of probability itself!

When I argue with Rodney, I'm not trying to convince him. I present my arguments in an effort to enlighten the huge group in the middle that is still undecided or unfamiliar with the concepts. I see challenges the same way. Sylvia Browne can come up with some excuse for not taking the test, and her Followers will believe it. So what? They will believe whether there is a challenge or not.

What's important is reaching those people who have some doubt. I believe the challenges go a long way in getting those people to think twice about whether these people are charlatans. In one-on-one conversations, which is by far the best way to influence people (ask the Mormons), you can open a lot of eyes talking about people who fail the challenge or who refuse to take it.

It's at the heart of skepticism and critical thinking to say, "If you can do it, prove it!" We look like idiots if in response to, "Fine! How do I prove it?" we go, "Gee. I dunno. Maybe, uh, maybe we could, uh, get some people to go over to Sylvia's house and like, you know, do a test or something."

 

[tsig]

By paranormal tests do you mean challenges like the IIG and JREF challenges? Or do you mean paranormal tests conducted by paranormal researchers?

Is your assertion based on the fact the most prominent woos have refused testing and that over the last decade only a handful of obscure people have ever actually gotten to the point of being tested?


To whom are they submitting these resumes? Do you have evidence of these people using it to promote themselves?


Again, what evidence do you have of this?

Note: I'm getting deja vu all over again while writing this response.

Before you lay out your evidence supporting you assertions, please consider what I wrote upstream about the three broad audiences: skeptics, True Believers, and fence sitters. I argue that when it comes to True Believers there's nothing that can be said or done to convince them otherwise. Hell, Rodney is now effectively arguing that the "psi effect" is random and uncontrollable and hidden within, yet still separate from, the randomness of probability itself!

When I argue with Rodney, I'm not trying to convince him. I present my arguments in an effort to enlighten the huge group in the middle that is still undecided or unfamiliar with the concepts. I see challenges the same way. Sylvia Browne can come up with some excuse for not taking the test, and her Followers will believe it. So what? They will believe whether there is a challenge or not.

What's important is reaching those people who have some doubt. I believe the challenges go a long way in getting those people to think twice about whether these people are charlatans. In one-on-one conversations, which is by far the best way to influence people (ask the Mormons), you can open a lot of eyes talking about people who fail the challenge or who refuse to take it.

It's at the heart of skepticism and critical thinking to say, "If you can do it, prove it!" We look like idiots if in response to, "Fine! How do I prove it?" we go, "Gee. I dunno. Maybe, uh, maybe we could, uh, get some people to go over to Sylvia's house and like, you know, do a test or something."

Guess you missed the part where I said "I think" .

It is my opinion and I have no intention of proving anything to you.

 

[Pup]

It's at the heart of skepticism and critical thinking to say, "If you can do it, prove it!"

And I think that's a necessary reaction, in order to advance knowledge. Investigating stuff is how the human race learns more about the world, after all.

But from a marketing point of view, there's also something to be said for the adage that no publicity is bad publicity. In general, people who want to believe in woo will believe in it. The only thing that can be affected is which particular woo they believe in. So publicity, whether it's good or bad, will bring attention to a particular woo, and give believers an opportunity to believe in that particular woo instead of some other that they've never heard of.

I don't think that's such a bad thing that challenges should be stopped. Looking at the big picture, I think it's better to investigate woo (up to a point) than to ignore it. Actually, one couldn't stop the human race from trying to investigate things even if one wanted to; it's what we do.

But looking at it from an individual woo's viewpoint, I expect that the energy of skeptics can be seen as a potential marketing engine that can be harnessed to their benefit, to make them stand out from other woos, if they can manipulate it right. The Professor was certainly trying that. It's just something that skeptics have to keep an eye out for, and try to minimize the impact of, while offering tests and challenges.

 

[Uncayimmy]

Guess you missed the part where I said "I think" .

It is my opinion and I have no intention of proving anything to you.

I fully realized it was your opinion when I asked you to support it, this being a discussion board centered around critical thinking and all. Are you under the impression that "it's my opinion" is a "get out of jail free" card or something?

 

[Rodney]

Hell, Rodney is now effectively arguing that the "psi effect" is random and uncontrollable and hidden within, yet still separate from, the randomness of probability itself!

I'm arguing all that? I thought that I was arguing the following: "It's certainly true that lotteries and casinos make money, but I'm not aware of any study done of either lotteries or casinos demonstrating that there is no psi effect . . . In casinos, there could conceivably be a negative psi effect that offsets a positive psi effect, wherein some people lose even more than would be expected by chance, offsetting those who do win. It also could be that the casino psi effect is very limited, and so, while some people don't lose as much as would be expected by chance, they can't overcome the house odds advantage." See http://www.internationalskeptics.com/forums/showpost.php?p=5368099&postcount=72