The Ganzfeld Experiments

[Ed]

If you study and know statistics then you know that the meta-analysis is severely flawed due to the lack of demographic matching and satadardization.

I agree that there is an effect and so far there is no proof that it is not an arftifact of a flawed design

Basically, that's it in a nutshell. The focus on any one aspect of this study seems to be misdirection. The broader issue is "why"?

They had to know. It was their business to know. It reminds me of that quote from the Wired article on Targ "we have a result". The question is is getting support for some paranormal something worth compromising one's intellectual integrity? And is focussing on sullied results, with full knowledge that they are suspect, just as bad? Are these things inescapable in any belief system?

 

[T'ai Chi]

If you study and know statistics then you know that the meta-analysis is severely flawed due to the lack of demographic matching and satadardization.

So does that apply to critics (of 'psi') meta analyses... ?

 

[T'ai Chi]

And you can't control for the non standard and standard by just rating things on a scale Tai, as a person versed in statitics you should know that is just a smoothing of already flawed or potentialy flawed data.

So... I'm having a hard time understanding why you believe the standard ratings to be flawed I guess.

Not to even discuss Paul's contenetion that at times the reseachers may have deliberately chosen targets to influence the results.

I'm not interested in Paul's speculation.

 

[Dancing David]

Basically, that's it in a nutshell. The focus on any one aspect of this study seems to be misdirection. The broader issue is "why"?

Tai people get goofy results all the time, it is imporatant to weed out why they got the goofy results. If someone wants to claim that there is a psi sending in the ganfeld, them it is crucial to eleiminate all such artifacts.
It was noise in microwave transmissions that led to the discovery of the cosmic microwaves. So finding sources and potential sources of noise is important.

If the potential exists that an artifact of the design created the results, then the results are inconclusive. that is science. It is not the misdirection, science is reductioneist, if you want to scientificaly prove that psi exists then you have to eleiminate all possible sources of error.

That is true of all science and especialy the social sciences.


They had to know. It was their business to know. It reminds me of that quote from the Wired article on Targ "we have a result". The question is is getting support for some paranormal something worth compromising one's intellectual integrity? And is focussing on sullied results, with full knowledge that they are suspect, just as bad? Are these things inescapable in any belief system?

Yes, bad research is practised in all fields, the others in the field should police the results, and any one should be able to replicate the results.

The issue is that with a flawed study you get flawed data.

It is true of all sciences, you don't think that everybody just agreed when that woman discoved the chiarlity of weak particles , do you? No they had to go and repliacte her results before they beileived them to be true.

That is the nature of science, I don't have to beleive that water is comprised of two hydogens and an oxygen. If I wish I can use electrolysuis to confirm the proportion of the gases and other assays to replicate the results.

All research should be subjected to scepiticism, there is fraud and malfeisance in all areas of research.

 

[Dancing David]

So... I'm having a hard time understanding why you believe the standard ratings to be flawed I guess.

Ersby has pointed out a very interesting artifact, can you just set the midpoint?
What about inter rater reliability?
If you know how meta analysis works then you know that demographic matching of the studies is crucial to the results. To compare unrelated methods is questionable, to pretend to go back and control for it is technicaly called 'fudging the data'.



I'm not interested in Paul's speculation. [/B]

You should be, any accusation of fraud is always very serious!

 

[T'ai Chi]

if you want to scientificaly prove that psi exists then you have to eleiminate all possible sources of error.

Eliminating all possible sources of error isn't even possible in physics, dude. Think "measurement error" for one. We can only eliminate all that we can.

 

[T'ai Chi]

You should be, any accusation of fraud is always very serious!

Seriously just speculation, unless there is good evidence for that. Evidence that Paul has not presented a good case for, IMO.

 

[Anders W. Bonde]

Originally posted by T'ai Chi:

Eliminating all possible sources of error isn't even possible in physics, dude. Think "measurement error" for one. We can only eliminate all that we can.

At the very least, psi-ists need to reduce measurement error to the point where measurement error alone can be entirely ruled out as the cause for what may appear to be statistically significant results. The lack of replication of experiments indicates to me that they have not been doing that so far - methinks they need to change their methods - which will render all past studies what they are, namely useless.

 

[CFLarsen]

At the very least, psi-ists need to reduce measurement error to the point where measurement error alone can be entirely ruled out as the cause for what may appear to be statistically significant results. The lack of replication of experiments indicates to me that they have not been doing that so far - methinks they need to change their methods - which will render all past studies what they are, namely useless.

Oh, believe me, they will change their methods, but they will not agree that their past studies are useless.

If they do, they will bury it so deep that only the most observant reader will discover it. Case in point:

And deep within that PEAR paper, right after the final and most searching analysis results were posted, is the following admission that the results simply did not appear:

"Once again, there was reasonably good agreement among the six scoring recipes, but the overall results were now completely indistinguishable from chance."
(PEAR, p227, Distributive Scoring)

Full article here

 

[Ersby]

So... I'm having a hard time understanding why you believe the standard ratings to be flawed I guess.

The problems, as I see it, with the standardness criteria are...

First, it means that the Honorton meta-analysis of 1985 can no longer be compared to the 1999 “standarded” m-a, since the second uses a different criteria for choosing which experiments to include. Thus, when Radin lumps them together for his incomplete overview of ganzfeld work since 1974, he is making a mistake.

Second, “standardness” consists of the autoganzeld protocol set-up as laid down in 1987, and also some other notes according to the PRL results. For example, they suggest considering meditators or artists as “standard” since they got good results in the PRL experiments. But these suggestions seem to have been rather arbitrary. If good results in the PRL database should indicate “standardness”, then dynamic targets should be standard and static targets non-standard. Yet there’s an experiment using static targets at the highest end of the range, so obviously dynamic/static was not given as a criteria for “standardness” which, considering the large gap in results between them in the PRL work, I find odd.

Plus, certain experiments that used the autoganzfeld quite strictly and consider themselves as a replication to the PRL work (such as Bierman’s, see one of my posts above, or maybe the last page, I don’t know!) has been given a very low rating. This is especially so with Willin’s work. The differences seem to be the target nature (music) and the randomisation process (manual shuffling). Other than that it’s a pretty standard ganzfeld set up. Yet it’s rated at a lowly 1.33. This seems somewhat harsh.

Third, having given a new order to the list of ganzfeld experiments, the mid-point of the scale has been set at the median: 4. Now this is, in fact, the only place along the range where the hypothesis regarding standard results to non-standard results is maintained. The mean (5.33) and the mode (6.67) are no good. In fact, if you take the mean as the average and use the full results of Wezelmen and Bierman’s series 6 experiments, the results of standard and non-standard are less than one per cent different. Amherst has insisted that the median is the only worthy average to take, but this hasn’t convinced me. It implies that the standardness rating actually has a numerical value. That standardness 2 is half as standard as standardness 4. Nor are those experiments of the same standard homogenous.

I have a dilemma just now that illustrates the point neatly. On my hard drive I have a collection of stuff about the ganzfeld, but not all. I am missing an experiment by Symmonds and Morris from ‘93. Looking at the m-a I can see it scored 4. What does that tell me? Nothing. I cannot even guess what the protocol of Symmonds’ experiment was by looking at the other experiment that scored 4 (Bierman’s series 5).

Fourthly, I don’t think there’s much of a liner regression of results according to standardness. See my graph of page 10... or 11 [edit: page 12!].

But may I ask you a question, T'ai Chi? You know more about statistics than me, so maybe you can help. One of the effect sizes quoted in the m-a seems peculiar to me. How do you calculate the effect size? In particular, an experiment that lasted 10 sessions, with a 25% hit rate expected by chance, yet got only one hit (10%). What would the effect size of that experiment be?

 

[Paul C. Anagnostopoulos]

T'ai said:
Eliminating all possible sources of error isn't even possible in physics, dude. Think "measurement error" for one. We can only eliminate all that we can.

Yes, but, you see, physics doesn't rely entirely on protocol and statistics to show effect, whereas psi does. That means psi experiments have to be perfect, which is impossible.

That's why we keep asking for some psychic to just do it.

~~ Paul

 

[dharlow]

That's why we keep asking for some psychic to just do it.

~~ Paul

Could you be more specific? Any transmission of information without known senses will have to have an identifiable target...simply "reading" someone's thoughts could not be considered permissable evidence. What one person might consider to be an accurate description of that target may not be so accurate to someone else, hence the use of statistics to introduce some form of objectivity to any correlations.

The Ganzfeld often relies on the premise (which may be wrong) that psi operates to some degree or another in all individuals, and multi-subject experiments also reduce the potential for cheating, assuming no relation between sender and agent. As far as tests with specific individuals obtaining high scores, there are quite a few in the parapsychological literature...with the debate then centering around the potential for cheating.

 

[Ed]

Could you be more specific? Any transmission of information without known senses will have to have an identifiable target...simply "reading" someone's thoughts could not be considered permissable evidence. What one person might consider to be an accurate description of that target may not be so accurate to someone else, hence the use of statistics to introduce some form of objectivity to any correlations.

Hence the use of statistics to obfuscate. Why not forced choice? Why not Zener cards? Why not experiments that are designed to be clear and unequivical? It appears that clarity and lack of equivication are antithetical to the demonstration of the paranormal.

 

[dharlow]

Hence the use of statistics to obfuscate. Why not forced choice? Why not Zener cards? Why not experiments that are designed to be clear and unequivical? It appears that clarity and lack of equivication are antithetical to the demonstration of the paranormal.

Forced choice experiments (which often used zener cards in the Rhine era) relied on statistics, and there is not much of a difference (aside from some statistical assumptions that need to be changed) between those experiments and the Ganzfeld. The main difference is in target material. It was believed that targets more resembling day-to-day imagery might better simulate "real-life" psi (for those who believed in such). These experiments still rely on a theoretical baseline against which to judge the results,
and are essentially a forced-choice experiment, although when giving the mentation, the subject often doesn't know the target nature, decoys, etc...

The Ganzfeld technique and accompanying statistical analysis is pretty straight forward. Where things seem more complicated (perhaps 'obfuscated' as you said) is in the use of meta-analysis to pool the results of these experiments. Meta-anaylsis should not be used to prove or disprove an effect (as has been done, respectively, by Radin, and Milton/Wiseman). Meta-analysis should only be used as an identifier of tentative correlations in the data which can then be retested, and it unfortunately being misused by a few proponents and skeptics in the parapsychology debate.

 

[Paul C. Anagnostopoulos]

dharlow said:
Could you be more specific?

I want some psychic to come forward who can just do stuff. In an obvious manner. Fairly reliably. Then we can dispense with the statistics.

The Ganzfeld technique and accompanying statistical analysis is pretty straight forward.

I disagree. Because the hypotheses of the experiments have to do with the protocol and statistical analysis themselves, rather than some theory of how psi works, the protocol and statistics have to be more or less perfect. Not only do known mundane means of information transfer have to be eliminated, but so do all possible mundane means, which is hopeless. The statistics may be trickier than people imagine: Is the simple 25% by chance model too naive? What else could be going on to skew the chances, even without information leaks? Why can some experimenters apparently replicate results, while others cannot? What is this experimenter effect, anyway?

These experiments still rely on a theoretical baseline against which to judge the results, and are essentially a forced-choice experiment, although when giving the mentation, the subject often doesn't know the target nature, decoys, etc...

Why speak and record the mentation at all? This seems like a bag hacked on to the side of the protocol.

~~ Paul

 

[dharlow]

I want some psychic to come forward who can just do stuff. In an obvious manner. Fairly reliably. Then we can dispense with the statistics.

It isn't that simple. Any psychic whose claimed abilities consist of mental (i.e. non physical) phenomena must be subject to strict controls employed either by a number of independent magicians or scientists with conjuring knowledge. Derren Brown performs 'psychic feats' on an almost daily basis and fairly reliably. Of course, he is not using any psychic abilities here, but he could claim to if he wanted, as Geller does. Certainly some claimed subjects have obtained success in experiments where one need not have much knowledge of statistics to know that the result is due to chance. One then must determine if controls are adequate to prevent cheating.

I disagree. Because the hypotheses of the experiments have to do with the protocol and statistical analysis themselves, rather than some theory of how psi works, the protocol and statistics have to be more or less perfect. Not only do known mundane means of information transfer have to be eliminated, but so do all possible mundane means, which is hopeless. The statistics may be trickier than people imagine: Is the simple 25% by chance model too naive? What else could be going on to skew the chances, even without information leaks? Why can some experimenters apparently replicate results, while others cannot? What is this experimenter effect, anyway?

Well, these standards should apply to any scientific experiment, especially ones involving humans as subjects. Any kinds of errors and outside factors must be eliminated in order for the result to stand up. I agree that they are even more important to a phenomena which is apparently sporadic and seems to contradict other forms of knowledge. This is why these results need replication, and certainly some experiments do achieve independent significance in the Ganzfeld, even in the most recent database despite refinements in technique to eliminate artifact. As far as the other questions you asked, they're good questions, and they've been asked for 70 years ever since Rhine's results came about. In particular, it does seem that certain experimenters achieve more success than others. Experimenter effects are becoming more known throughout science, especially in psychology. Interacting with human subjects can influence the results to some degree and it may be no different here. In particular, it is believed that experimenters who put the subject to ease and express a positive viewpoint that psi exists to the subjects will obtain significant results, while those who interact little with the subject and do not express this view obtain the null. Of course there is the psi experimenter effect, a hypothesis in which the experimenter's psi influences the result. However, as this hypothesis use as an explanation the very phenomena under question, it will not be much help to those skeptical of psi.

[/QUOTE]
Why speak and record the mentation at all? This seems like a bag hacked on to the side of the protocol.

~~ Paul [/QUOTE]

As far as I know, these records are often used as a way of seeing how psi may manifest in the mind of the operator. For instance, does it work through symbolism? Vague or striking imagery?...etc. Parker discusses some of the uses of this material in the Psi Wars special issue of the Journal of Consiousness Studies.

 

[Paul C. Anagnostopoulos]

Dharlow said:
It isn't that simple. Any psychic whose claimed abilities consist of mental (i.e. non physical) phenomena must be subject to strict controls employed either by a number of independent magicians or scientists with conjuring knowledge.

Yes, of course, I understand that. I'm just asking for the guy to be able to do something reliably, so we can test him a few times and all agree that he's the genuine article. Ain't never gonna happen.

Well, these standards should apply to any scientific experiment, especially ones involving humans as subjects. Any kinds of errors and outside factors must be eliminated in order for the result to stand up. I agree that they are even more important to a phenomena which is apparently sporadic and seems to contradict other forms of knowledge.

And they are absolutely critical to phenomena that are nothing more than statistical analysis of the rejection of the null hypothesis that results will be at chance. People just don't seem to understand what it means that the hypotheses in psi experiments are not about an underlying physical theory. Any other so-called science that operates the same way has the same problems.

~~ Paul

 

[CFLarsen]

I'm just asking for the guy to be able to do something reliably, so we can test him a few times and all agree that he's the genuine article. Ain't never gonna happen.

Let's break that up, so we get a more accurate assessment of the situation.

I'm just asking for the guy to be able to do something reliably...

This is easy: Psychics run their businesses based on that they are able to do something reliably.

...so we can test him a few times...

This is much more difficult: We know that the vast majority of psychics shun anything that looks like a serious test.

...and all agree that he's the genuine article.

That would require a clear, unambiguous result, which we are still waiting for. Do. Not. Say. "Arizona".

Ain't never gonna happen.

So far, nope. The onus is on the psychics, not the skeptics. I don't care much for statistical mumbo-jumbo from tests where there is no clear definition of what researchers are looking for. That is simply a smoke-screen. Let's get those who claim to talk to dead people, or predict earthquakes or lottery numbers. They are out there, but they don't want to come out into the light.

Feh.

 

[Interesting Ian]

Yes, of course, I understand that. I'm just asking for the guy to be able to do something reliably, so we can test him a few times and all agree that he's the genuine article. Ain't never gonna happen.

Quite possible it ain't never going to happen. I think we should be wary, albeit not totally dismissive, of those who claim they can produce marked "in yer face" anomalous cognition/perturbation on demand. Such "in yer face" stuff tends to be inherently unpredictable so far as I am able to understand.

But what about not concentrating on the authenticity question in a direct way, but attempting to discern certain characteristic patterns in these meta-analyses, and then seeing if they can be reproduced?? If they can that would be interesting would it not??.

Then perhaps we could devise some hypothesis about how these effects operate. If further experimentation were then commensurate with the hypothesis in question this would not only settle the authenticity question, but will also furnish us with a theory about how psi operates.

Also this method would be less susceptible to accusations of cheating and of artifacts skewing the results.

 

[Ed]

]

Forced choice experiments (which often used zener cards in the Rhine era) relied on statistics, and there is not much of a difference (aside from some statistical assumptions that need to be changed) between those experiments and the Ganzfeld. The main difference is in target material.

No. The interposition of raters is a serious flaw in control, particularly when more than one is used. Why even go down that road? A pool of x stimuli, one transmitted, pick from the pool. The notion that a rater can pick more accurately what a receiver "sees" than the receiver is ludicrious. Simple, clear, unambigious.

It was believed that targets more resembling day-to-day imagery might better simulate "real-life" psi (for those who believed in such).

Based on what? That well controlled experiments did not produce the desired results so the design becomes more slippery? Even here, the lack of self selection of the stimulous is unwarrented and suspicious. I somehow find it laughable that one could describe a complex scene more easily than a simple figure. In any event, objective complex figures could be constructed that leave less wiggle room. How about simply using one of the primary colors as backrground for each card? Why is a rater necessary if not to interpolate additional wiggle? And how can one talk about the characteristics of real life psi without a demonstration of psi in the first place? So it can't be demonstrated yet the characteristics are known? Are you saying that if you did 30,000 trials on an "adept" with a simple figure interterspersed with blanks that person would not perform? At all? Nothing? Nada? But when it gets complex he can? Can he with a simple experimental design?


These experiments still rely on a theoretical baseline against which to judge the results,
and are essentially a forced-choice experiment, although when giving the mentation, the subject often doesn't know the target nature, decoys, etc...

Perhaps there is but what could be simple and direct is obfuscated.

The Ganzfeld technique and accompanying statistical analysis is pretty straight forward.

[It could be a hell of a lot better.

Where things seem more complicated (perhaps 'obfuscated' as you said) is in the use of meta-analysis to pool the results of these experiments. Meta-anaylsis should not be used to prove or disprove an effect (as has been done, respectively, by Radin, and Milton/Wiseman). Meta-analysis should only be used as an identifier of tentative correlations in the data which can then be retested, and it unfortunately being misused by a few proponents and skeptics in the parapsychology debate.

I suspect that T'ai will take issue with you, he being a statistition and all. But I agree. The effects seem to be an artifact of the stats. [/B]