Case Study: The IIG Protocol for VFF

[Uncayimmy]

I am starting a thread to discuss the protocol design for the IIG test for VisionFromFeeling because I think it's worthy of its own discussion. Let's try to keep the VFF stuff in the appropriate threads and only discuss her in the context of her influence on the negotiations. Hopefully Derek from the IIG will engage us.

I don't see the $50K challenge as a scientific test. It's a "put up or shut up" proposition. If the ability is there, the protocol should allow it to shine through. If it's not real, then the controls should ensure that other factors (information leakage, lucky guesses) will not result in a passed test.

Some argue that a poorly designed test is not better than no test at all. I think that's a point worth discussing in this thread. Part of that discussion should be to compare/contrast test vs challenge. If "poorly designed" favors the claimant, yet the IIG is still confident that their money is safe, is that a bad thing?

I didn't like that there were only 6 people per trial because it gave Anita a ~23% chance of getting one right in three tries. The reason the number was that low is because Anita refused to go any lower than 4.5 minutes per person. This presents a major logistical problem.

Even if they had used 10 subjects, she would have had a ~15% of getting one right, which is a little less likely than guessing a single roll of a die. This is the part of the challenge that I hate the most because if she gets one right, it seems like something special happened. It's hard to convince the average mope on the street that one success is nothing. However, I don't see how it could be avoided.

I also didn't like that variety of people in each trial. Based on my research kidney donations happen at virtually the same rate per age group meaning that the number of 20 year olds donating a kidney in a given year is about the same as for 40 year olds. What this means, though, is that 20X more people who are 40 have donated kidneys than those who are 20. In other words each year each age group from about 18 to 55 gains about the same number of donors - more years = more donors. And, of course, the older a group is, the more people who have suffered diseases or accidents that might have resulted in a kidney removal.

So, in an ideal world you find the people missing kidneys first. You then assemble a group of controls of the same sex and about the same age with similar body characteristics. This, too, creates a hell of a logistical problem. It's hard enough finding volunteers as it is.

That's enough to get the ball rolling. I trust others will chime in.

 

[JoeTheJuggler]

Thanks for starting the new thread. (Sorry about continue posting this OT stuff in the other thread.)

I understand these kinds of challenges are not scientific tests, but I don't know that I agree there is such a big distinction. After all, everyone calls them "tests", and they're generally modeled on scientific tests.

But yes, at heart they're PR challenges for educational purposes.

At any rate, I'm one of the people crowing that doing a poorly designed test is not better than no test at all. For one reason, for paranormal claimants, there's almost no such thing as bad publicity. So if we're going to give them the limelight, we should have an ultra-tight case of failure.

I agree with your points on the protocol. Chances were good that she'd get something right. (And for all of Rodney's wrong comments, he is correct that the wording of the protocol expressed the claim as a twofold thing where one part was just naming the person missing a kidney. Since this has no correlation to Anita's claimed ability, it was a problem.)

Also, not specifying how subjects were chosen and assigned was a big problem. Someone in the other thread made a great analogy to encrypting data--keeping the protocol secret was a pretty good sign of possible information leakage.

I thought the protocol was also pretty vague on some of the nuts and bolts of the test itself, though the execution of the test was very good (subjects "may wear a hood or broad-brimmed hat. . ."). I missed the beginning and first round, but I believe they did an "open" round, where she was able to "demonstrate" her ability on a subject whose kidney situation she knew and then asked if there was any condition or issue that would prevent her ability from working in the test itself. (Again, very good stuff, but it should have been in the protocol.)

In praise of the protocol, one huge redeeming feature was this bit:

For this Preliminary Demonstration to be considered successful, the Applicant must correctly identify which Subject from each trial group of six was in fact missing a kidney and correctly identify which kidney (left or right) was missing.

And believe it or not, despite all the bunk Anita has been saying since the test, this part worked.

She at least admits that she failed. And for that I commend the IIG.

 

[Audible Click]

<snip>She at least admits that she failed. And for that I commend the IIG.<snip>

She admits that she failed but posits that her claim was not falsified and now she has added that she has a new claim: "I know beforehand if I'm wrong or if I'm right."

 

[bookitty]

<Snipped for focus>
At any rate, I'm one of the people crowing that doing a poorly designed test is not better than no test at all. For one reason, for paranormal claimants, there's almost no such thing as bad publicity. So if we're going to give them the limelight, we should have an ultra-tight case of failure.

You can't design a test to be bullet-proof against chance. Chance happens. For example, if this test had included more subjects and 6 trials instead of 2, then VFF could have gotten two subjects correct completely by chance. (Actually, this particular test makes for a fun game of extrapolation - If the test was more like X, than the excuse would have been like Y.)

Are there examples of a claimant seeing the results of their test, admitting to failure and not applying a post-test spin?

 

[Kariboo]

Statistics is not my best subject so maybe someone can do the calculations for this:

If they had used the same number of people (18, 3 of whom would have been missing a kidney) and time per subject (4.5 min), but had presented the subjects one at a time and having VFf make a forced choice with each subject ( 1 or 2 kidneys) and randomizing the people with one kidney in that lineup (they are randomized between the 15 others).

Wouldn't have that 1) made a huge difference in having the result be due to chance or 2) picking the one that looks most likely to be missing a kidney of 6 subjects at a time?

While at the same time conforming to what VFf claimed she could do?

 

[ynot]

I thought the IIG test was poorly conceived but perhaps not as poor as the Connie Sonne test. I don’t see any reason why Anita needed to know that one of the test subjects definitely had a missing kidney, and I don’t know why she didn’t test just one person at a time.

Simple test - “Has this person got a missing kidney?” - yes or no (and then -"are you absolutely sure? ;-). Repeat enough times to reduce the chance of success by lucky guess. If she gets it wrong in any attempt she has failed and the test is over. Simple and definitive.

 

[JoeTheJuggler]

You can't design a test to be bullet-proof against chance. Chance happens. For example, if this test had included more subjects and 6 trials instead of 2, then VFF could have gotten two subjects correct completely by chance. (Actually, this particular test makes for a fun game of extrapolation - If the test was more like X, than the excuse would have been like Y.)

Are there examples of a claimant seeing the results of their test, admitting to failure and not applying a post-test spin?

I wasn't suggesting there is a test that completely rules out chance. I mean a test where in a failure is clearly seen as a failure.

The point of the statistics is to rule out success due to chance with some level of confidence. In that regard, I have no problem with this protocol.

The standard was 100% correct, or fail. I think that was the redeeming feature of this protocol, and it's what I've been confronting Anita with over and over on the other thread. She said that if she failed the test she would admit that her claim is false. She did fail the test (and she acknowledges that), but she refuses to admit her claim is false. I understand that there is no protocol that could get her to admit that.

Again, on this point--perhaps the most important one--I have no problem with the protocol.

The point I was making is that it's not better to do a poorly designed test than no test at all. I see no reason to have given in to her demands that were inconsistent with her claims. Forcing the subjects to sit for extended periods of time, for example, when her claim has consistently been an instantaneous ability.

 

[ynot]

The point I was making is that it's not better to do a poorly designed test than no test at all. I see no reason to have given in to her demands that were inconsistent with her claims. Forcing the subjects to sit for extended periods of time, for example, when her claim has consistently been an instantaneous ability.

I completely agree.

ETA - The person making a paranormal claim should define exactly what the parameters of the claim are and then they should demonstrate the claim. The only sceptical test that needs to be done is on the validity and honesty of the demonstration. If the demonstration isn’t valid or honest it shouldn’t be given any attention or limelight.

 

[Uncayimmy]

I understand these kinds of challenges are not scientific tests, but I don't know that I agree there is such a big distinction. After all, everyone calls them "tests", and they're generally modeled on scientific tests.

Scientific tests are the result of following the scientific method, which by definition includes research. These challenges are nothing more than one person saying, "I can do something science says is impossible" and the other side saying, "Fine, let's see you do it when we set up a situation where there is no other rational explanation."

I agree with your points on the protocol. Chances were good that she'd get something right. (And for all of Rodney's wrong comments, he is correct that the wording of the protocol expressed the claim as a twofold thing where one part was just naming the person missing a kidney. Since this has no correlation to Anita's claimed ability, it was a problem.)

You're both just playing games of semantics. The only reason it was expressed in that manner is because of the limitations of the English language. It would have sounded stupid and kinda confusing to say it any other way, especially since the protocol itself (due to using kidneys stored inside living humans) required two steps to make the identification.

If you have a better way of writing it, let's hear it.

Also, not specifying how subjects were chosen and assigned was a big problem.

Big problem for whom and in what sense?

I thought the protocol was also pretty vague on some of the nuts and bolts of the test itself, though the execution of the test was very good (subjects "may wear a hood or broad-brimmed hat. . .").

How is that pretty vague? Seems clear to me.

I missed the beginning and first round, but I believe they did an "open" round, where she was able to "demonstrate" her ability on a subject whose kidney situation she knew and then asked if there was any condition or issue that would prevent her ability from working in the test itself. (Again, very good stuff, but it should have been in the protocol.)

It was in implicit in the statement, "The IIG will ask the Applicant whether she is comfortable and if she feels her abilities are working adequately." How can she know this without first encountering a person missing a kidney?

 

[bookitty]

I wasn't suggesting there is a test that completely rules out chance. I mean a test where in a failure is clearly seen as a failure.

The point of the statistics is to rule out success due to chance with some level of confidence. In that regard, I have no problem with this protocol.

The standard was 100% correct, or fail. I think that was the redeeming feature of this protocol, and it's what I've been confronting Anita with over and over on the other thread. She said that if she failed the test she would admit that her claim is false. She did fail the test (and she acknowledges that), but she refuses to admit her claim is false. I understand that there is no protocol that could get her to admit that.

Again, on this point--perhaps the most important one--I have no problem with the protocol.

The point I was making is that it's not better to do a poorly designed test than no test at all. I see no reason to have given in to her demands that were inconsistent with her claims. Forcing the subjects to sit for extended periods of time, for example, when her claim has consistently been an instantaneous ability.

I agree with everything you say here.

I'm just trying to figure out if there is any way to get claimants to accept the statistics. Not as an argument but as a question. Outrageous odds don't seem to work.

 

[Uncayimmy]

I agree with everything you say here.

I'm just trying to figure out if there is any way to get claimants to accept the statistics. Not as an argument but as a question. Outrageous odds don't seem to work.

Honestly, forget the claimants. There is little or even no evidence that these challenges change their minds. Even if it was likely, it's entirely way too much effort to expend on just one person (in my opinion).

The audience you're trying to reach isn't skeptics either. For them the results are a foregone conclusion. Otherwise, why put up the $50K or $1M?

The audience you want to reach is made up of all those people who really don't think about this stuff all that much. They don't understand the statistics very well (most people don't). Chances are they are only vaguely familiar with the claim. They probably think there *might* be something there but aren't sure. They figure that lots of these people are frauds or deluded but not all.

That's why you need to make compromises. Who cares, besides us, that Anita says she can usually do it very quickly? Lots of people we want to reach would find it unreasonable to make her do it 30 seconds. If the IIG refuses to accept what *appears* to be a reasonable amount of time, then they look like cowards. When that happens, the claimant wins.

It's a judgment call that requires balancing resources against public perception by the unwashed masses. Forget about the skeptics and forget about the True Believers and concentrate on those people you can reach. I think that's what the IIG tried to do.

 

[bookitty]

Very good points, UY. I was interpreting Joe's questions as if they related to the claimant, not the intended audience.

 

[JoeTheJuggler]

You're both just playing games of semantics. The only reason it was expressed in that manner is because of the limitations of the English language. It would have sounded stupid and kinda confusing to say it any other way, especially since the protocol itself (due to using kidneys stored inside living humans) required two steps to make the identification.

If you have a better way of writing it, let's hear it.

From the protocol:

The Claim:

The Applicant claims to be able to detect which Subject in a group of six Subjects is missing a kidney, to further identify which kidney (left or right) is missing in her selected Subject, and to be able to do this with 100% accuracy in three consecutive trials.

How I might have worded it:

The applicant claims to be able to see inside a person to be able to detect which kidney of which of six subjects is missing and to be able to do this with 100% accuracy in three consecutive trials.

I don't think the first part of the claim in the protocol, "The Applicant claims to be able to detect which Subject in a group of six Subjects is missing a kidney," corresponds at all to what her claim was. Here's how it's worded on her website:

"When I look at people, I see images in my mind of the inside of their bodies. I see organs, tissues, cells, and chemicals, and even what I call the vibrational level inside the atoms. Not only pictures, I also feel things. I feel texture, density, temperature, and more, and I can feel the pain and discomforts that others have."

How is that pretty vague? Seems clear to me.

It sounds to me like it was up to each subject to decide what kind of head covering, if any, he or she would wear. I think what they actually did--all of them wearing the same head covering--made good sense, but that's not what the protocol called for.

It was in implicit in the statement, "The IIG will ask the Applicant whether she is comfortable and if she feels her abilities are working adequately." How can she know this without first encountering a person missing a kidney?

Again, her claim was to be able to see a person's insides when she looks at them, not anything about detecting which person is missing an unspecified kidney. At any rate, I don't see how that statement implied an open trial run of her skills. (The open trial run is usually specified in MDC protocols--as with dowsers. It was also spelled out in the Connie Sonne protocol.)

Scientific tests are the result of following the scientific method, which by definition includes research. These challenges are nothing more than one person saying, "I can do something science says is impossible" and the other side saying, "Fine, let's see you do it when we set up a situation where there is no other rational explanation."

I understand the point you're making--there's no need to treat Anita's claim, for example, as if it were a legitimate scientific hypothesis--, but you surely can't deny that these tests are modeled after scientific testing. The way she fails is a failure to reject the null hypothesis. That's still what we're trying to do.

 

[Maia]

I'm sure that there's been a link already posted somewhere... so this may be a silly question... but could someone re-post the link to the actual protocol itself? I'd like to read it in order to be sure that I know what everyone is talking about.

 

[Uncayimmy]

The applicant claims to be able to see inside a person to be able to detect which kidney of which of six subjects is missing and to be able to do this with 100% accuracy in three consecutive trials.

If a kidney is missing, how, exactly, does she detect it? For all we know it's in a some other person halfway around the world? Besides, you have the word "which" twice in there, which makes it sound like there are two things going on. Furthermore, your description is not an accurate description of what happens during the test. What happens in the test is exactly what they wrote in the claim.

I don't think the first part of the claim in the protocol, "The Applicant claims to be able to detect which Subject in a group of six Subjects is missing a kidney," corresponds at all to what her claim was. Here's how it's worded on her website:

There is actually no need whatsoever for the claim to be there in the first place. If you took it out, nothing at all would change. Well, you and a few others wouldn't be as worked up, but as a practical matter nothing would change.

"When I look at people, I see images in my mind of the inside of their bodies. I see organs, tissues, cells, and chemicals, and even what I call the vibrational level inside the atoms. Not only pictures, I also feel things. I feel texture, density, temperature, and more, and I can feel the pain and discomforts that others have."

Who cares? What difference does it make? She talks to ghosts, too. None of those things will be confirmed or denied by the challenge because the challenge is not testing the claims on her website.

Anita and the IIG disagreed over whether she could do what was outlined in the protocol. How they got there is irrelevant. What's relevant is that the IIG was so confident that she would fail, they were willing to risk their money. Anita was seemingly so confident that she spent nearly a thousand dollars to go for it. You want it to mean something that it doesn't.

Suppose you're out camping with a group of guys. One guy claims he's been trained as a Navy SEAL, Green Beret, a sniper, and FBI agent. You think he's full of ****, so you say, "Okay, you think you could hit six out six beer bottles with a rifle from 20 yards for $50?" He says he can, so you grab a .22 out of the trunk and say, "Here ya go."

If you totally misread the guy and he hits all six bottles, you still don't know if any of that stuff is true. That requires a whole different approach to verify. But if he fails, he looks like the jackass he is. That's all these challenges are, so quit trying to make them mean something they are not.

It sounds to me like it was up to each subject to decide what kind of head covering, if any, he or she would wear. I think what they actually did--all of them wearing the same head covering--made good sense, but that's not what the protocol called for.

If it is not disallowed, then it is permitted. Both sides agreed to it. I have no idea why you think that's a problem.

Again, her claim was to be able to see a person's insides when she looks at them,

It sure was. So what? The challenge made blind luck an extremely unlikely factor. It made educated guesses very unlikely but still a tad better than blind luck. No one has yet to come up with a known method by which she could have gotten all three right. If she had passed, it would not have proven that she could see a person's insides.

I understand the point you're making--there's no need to treat Anita's claim, for example, as if it were a legitimate scientific hypothesis--, but you surely can't deny that these tests are modeled after scientific testing. The way she fails is a failure to reject the null hypothesis. That's still what we're trying to do.

Yes, I can most certainly deny that these are scientific tests because if she were, in fact, to turn the world upside down, that spectacle is not at all indicative of how a scientist would research her abilities. The only thing they have in common is eliminating other possible methods to achieve a set of results. If she had passed, how could you possibly differentiate between seeing inside the body, smelling concentrations of kidney cells, or hearing kidneys working? The last two are actually far more likely.

There is no science involved in these challenges anymore than there's science involved in my camping scenario.

 

[Uncayimmy]

I'm sure that there's been a link already posted somewhere... so this may be a silly question... but could someone re-post the link to the actual protocol itself? I'd like to read it in order to be sure that I know what everyone is talking about.

No.

 

[Furcifer]

If the claimant is going to make outrageous statements before the test, there's certainly no way of preventing them from doing so afterwards. The hand waving is to be expected, plain and simple.

If anyone at the IIG could predict what kind of excuses a claimant would make after the test, and design protocol to prevent them from doing so, is themselves a mind reader and worthy of the $50K.

 

[LightinDarkness]

At any rate, I'm one of the people crowing that doing a poorly designed test is not better than no test at all. For one reason, for paranormal claimants, there's almost no such thing as bad publicity. So if we're going to give them the limelight, we should have an ultra-tight case of failure.

This is a interesting challenge though with people like Anita. Although Anita's claims to paranormal superpowers are wide and vast, she really zoomed in on the "I can see missing kidneys" things and absolutely refused to submit to any of the very simple and airtight tests that would prove the core ability behind the missing kidneys claim (that of being able to see past physical barriers to detect something inside..in this case, the presence or absence of a organ).

Suddenly, once she focused on testing this particular claim, barriers that would block her from viewing people blocked the power (even if it was made out of the same material as the shirts she is seeing through). Dozens of protocols that she would be able to pass if she really had this power were proposed but she refused all except the protocol that was most expensive and involved the most people - and coincidentally had the most room for her lucky guesses.

This is why I am not so sure that she really believes she has powers as some people think. At some level she is making moves to avoid any test that is simple and airtight in terms of its results.

So what are groups like the IIG to do? On the one hand they could refuse people pulling things like this, but in doing so Anita would claim the martyr mantle and use this refusal to spin that she really does have powers. On the other hand, we have tests within the questionable confines set out by the applicant, as we did here, which still prove what we already know - that Anita does not have powers - but have enough holes to let people like her spin away and find ways to claim victory.

It seems to be no matter what happens, Anita wins in terms of getting the publicity she seeks. As long as she refuses to submit to testing conditions that are more rigorous, this is always going to happen.

 

[Uncayimmy]

If the claimant is going to make outrageous statements before the test, there's certainly no way of preventing them from doing so afterwards. The hand waving is to be expected, plain and simple.

If anyone at the IIG could predict what kind of excuses a claimant would make after the test, and design protocol to prevent them from doing so, is themselves a mind reader and worthy of the $50K.

Seriously, though, would you want that? I wouldn't. If you keep in mind that the goal is to reach the fence-sitters, the more wriggling the claimant does on the hook, the worse she looks and by extension the worse other woos look. I'd rather hear about how the fat guy blocked her abilities and how the tattoos were a distraction than for her to quietly slip away with a mea culpa. It shows her true colors and how narcissistic and delusional these people really are.

I would have preferred three misses but not at the expense of all the histrionics we've gotten as a result of her getting one correct. Protocols should prevent winning by ordinary means and allow for success by paranormal means. Beyond that, let the excuses fly after the test. It's highly educational, not to mention entertaining.

 

[Furcifer]

Seriously, though, would you want that? I wouldn't. If you keep in mind that the goal is to reach the fence-sitters, the more wriggling the claimant does on the hook, the worse she looks and by extension the worse other woos look. I'd rather hear about how the fat guy blocked her abilities and how the tattoos were a distraction than for her to quietly slip away with a mea culpa. It shows her true colors and how narcissistic and delusional these people really are.

I would have preferred three misses but not at the expense of all the histrionics we've gotten as a result of her getting one correct. Protocols should prevent winning by ordinary means and allow for success by paranormal means. Beyond that, let the excuses fly after the test. It's highly educational, not to mention entertaining.

No, the way it stands now is pretty perfect. She performed exactly as the Gods of probability wanted her too. And that's with the SNAFU with Test 2. She essentially clawed her way back to average with that one.

The excuses are so very entertaining. I'm not sure if the ones made before the test for not taking one are sweeter then the ones made after.