The Ganzfeld Experiments

[amherst]

I cannot explain the results of the PRL.

Ersby, since you readily admit to not having an explanation for the evidence, why do you continue to believe that psi has not yet been satisfactorily proven? Surely you are aware that the experiments done at PRL have been replicated by (as of late 97) six different labs, Edinburgh(289 session), Amsterdam(164), Cornell(25), Rhine Center(590), Gothenburg(90), Utrecht(232) all with above chance hit rates consistant with the PRL work, and that the 355 PRL experiments were in fact a replication of the original 762 ganzfeld sessions published in 85. So again, my question to you is, exactly what more do you need?

Looking forward to your reply,

amherst

 

[Darat]

Ersby, since you readily admit to not having an explanation for the evidence, why do you continue to believe that psi has not yet been satisfactorily proven?

..snip...t

Do you mind me asking what definition you use for "psi"?

 

[Lucianarchy]

Read the discussion section of the Bierman et al paper. Page seven, I believe. Basically, the "random" number generator used by the original experimentor didn't give an equally populated sample set (unsurprisingly), but when Bierman corrected for the actual sample set that had been generated, the results became "non-significant."

The next-to-last paragraph is particularly interesting, as he specifically cautions his readership to look for this flaw in other ganzfeld databases. In other words, the flaw is probably systemic in Bierman's opinion.

Thanks. I can't seem to find the discussion you refer to though. I know a lot of stuff has been lost recently.

 

[drkitten]

Here is the original reference to the Bierman paper. As with most scientific papers, the real meat is towards the end, in the section labelled "Discussion" :

So what exactly is your problem with the ganzfeld? As you well know, the analyses done initially by Bem and then furthered by Bierman et.al. http://a1162.fmg.uva.nl/~djb/publications/1998/AutoGF_set20effect.pdf show that the results can not be due to response bias. [N.b. the authors specifically distance themselves from this statement within the paper, see page 7] This strongly suggests that any future successful experiments will not be due to it either.

 

[Paul C. Anagnostopoulos]

Amherst said:
Ersby, since you readily admit to not having an explanation for the evidence, why do you continue to believe that psi has not yet been satisfactorily proven?

I believe that psi has been proven, assuming the definition of psi is "stuff we don't understand how it works."

~~ Paul

 

[amherst]

Here is the original reference to the Bierman paper. As with most scientific papers, the real meat is towards the end, in the section labelled "Discussion" :

I've already corrected you once but you foolishly ignored me. The quote which you (for some absurd reason) think is an admittance of the PRL work being nonsignificant, is actually and quite clearly referring to the difference in success between static and dynamic targets.

I suggest that anyone truly interested in the matter at hand simply ignore "drkitten" because he is hopelessly confused and seems quite content to stay that way.

Originally posted by Darat
Do you mind me asking what definition you use for "psi"?

"...anomalous processes of information or energy transfer, processes such as telepathy or other forms of extrasensory perception that are currently unexplained in terms of known physical or biological mechanisms."
http://comp9.psych.cornell.edu/dbem/does_psi_exist.html#Introduction


amherst

 

[JMA]

Hi,

If Psi do exist, how do you explain Richard Wiseman failling in his mass participation experiment?

If the Ganzfeld IS accurately detecting a power, then I have to wonder what use it is. Sure, IF it's real then it's 'above chance' - but only just.[/B]

I agree with that...

And most of the psychic are not in a "ganzfeld" state when doing there reading, so what's the point?

See you,

 

[Dancing David]

. So again, my question to you is, exactly what more do you need?

Looking forward to your reply,

amherst

The standard deviation of the data in ganfeld effetct and a reason to understand why a 33% hit rate should be considered to be above the rate of standard deviation. I would really like to see some more recent studies that involve thousand of participants and runs.

 

[Darat]

...snip...


"...anomalous processes of information or energy transfer, processes such as telepathy or other forms of extrasensory perception that are currently unexplained in terms of known physical or biological mechanisms."
http://comp9.psych.cornell.edu/dbem/does_psi_exist.html#Introduction


amherst

Thanks for that. And sorry for this but can you explain your definitions "telepathy" and "extrasensory perception"? The reason for asking is that whilst I know how I would use them quite often here I've noticed others define them in a different way.

 

[Ersby]

Ersby, since you readily admit to not having an explanation for the evidence, why do you continue to believe that psi has not yet been satisfactorily proven? Surely you are aware that the experiments done at PRL have been replicated by (as of late 97) six different labs, Edinburgh(289 session), Amsterdam(164), Cornell(25), Rhine Center(590), Gothenburg(90), Utrecht(232) all with above chance hit rates consistant with the PRL work, and that the 355 PRL experiments were in fact a replication of the original 762 ganzfeld sessions published in 85. So again, my question to you is, exactly what more do you need?

Looking forward to your reply,

amherst

Some of the experiments you referenced were later culled from the Running Head meta-analysis for being "non-standard". In particular the Amsterdam ones. So are they acceptable experiments or not.

The Rhine Centre work, I'll admit, I'm not familiar with. Any links would be much appreciated. I'll take a closer look at the others you reference later. Just now, time's pressing. I'll leave you with something I knocked up on Word earlier...

In an homage to Loki's post, here are my problems with the ganzfeld database.

While we’re listing the problems with the pro-psi explanation, here’s my list:

1.. Black Magic!

(only joking)

1.. Standardness not applied to both meta analyses that showed a positive result. The significant effect gained after re-evaluating the data with a new criteria is assumed to be a replication of the old Honorton result. But the Honorton data did not have this new criteria attached to it. Until this is done (and it may well have no impact on the effect size) it is inaccurate to say that the Bem et al meta-analysis replicates the Honorton one.

2.. This is a minor quibble, but I’m in a sharing mood. In the Running Head paper, the “standardness” was measured from 7 to 1. Thus the average was set at 4. Why would this be? The value of standardness is just a construct. It has no intrinsic value of itself. In other words, an experiment valued with a standardness of 6 is not “twice as standard” as one that scored 3. In the same way, if Manchester Untied beat Arsenal 2-1, it doesn’t mean they played twice as well. The difference may be very small. Additionally, the standardness scale seems to be absolute. Willin’s work with music is rated 1.33 and below that there’s really very little room for a less standard protocol. But I’m sure there must be.

3.. The meta analyses don’t cover all the experiments. Ganzfeld trials have been carried out since 1973, and the most recent results I know of are from 2003. So that’s thirty years. Honorton’s m-a cover 12 of those (’73-’85) and Bem’s covers ’91 to ’97 (with two Kanthamani experiments from 1988 thrown in, which is odd since there’s no reference to other work from ’88 to ’90) Which leaves 12 years unaccounted for when people boldly talk about m-a and replication. I’ve already mentioned Bierman’s bleak data regarding 1985-93, and from what’s been released since 1998, the most recent experiments taken as a whole don’t seem to demonstrate an effect.

 

[amherst]

Originally posted by Ersby
Some of the experiments you referenced were later culled from the Running Head meta-analysis for being "non-standard". In particular the Amsterdam ones. So are they acceptable experiments or not.

Non-standard replications are acceptable if the objective is to understand psi. They are not acceptable (and were never intended) as a proof gathering method:

"Perhaps there is some merit in continuing to conduct exact replications of the ganzfeld procedure, but genuine progress in understanding psi rests on investigators' being willing to risk replication failures by modifying the procedure in any way that seems best suited for exploring new domains or answering new questions." http://comp9.psych.cornell.edu/dbem/updating_the_ganzfeld_data.htm

The Rhine Centre work, I'll admit, I'm not familiar with. Any links would be much appreciated. I'll take a closer look at the others you reference later. Just now, time's pressing. I'll leave you with something I knocked up on Word earlier...

In an homage to Loki's post, here are my problems with the ganzfeld database.

While we’re listing the problems with the pro-psi explanation, here’s my list:

1.. Black Magic!

(only joking)

1.. Standardness not applied to both meta analyses that showed a positive result. The significant effect gained after re-evaluating the data with a new criteria is assumed to be a replication of the old Honorton result. But the Honorton data did not have this new criteria attached to it. Until this is done (and it may well have no impact on the effect size) it is inaccurate to say that the Bem et al meta-analysis replicates the Honorton one.

There was no new criteria, I don't know where you got this from. The blind raters were instructed to rate the standardness of the studies according to the degree with which they conformed to the criteria set out by Bem and Honorton in the original Psychological Bulletin paper:

"Rather than provide our own ad hoc definition, we had the raters read the general description from the section labeled "The Ganzfeld Procedure" in Bem and Honorton's (1994, pp. 5-6) report as well as most of the detailed method section describing the computer-controlled autoganzfeld procedure used in Honorton's Psychophysical Research Laboratories (PRL) published in the the Journal of Parapsychology (Honorton et al., 1990, pp. 102-110)."
http://comp9.psych.cornell.edu/dbem/updating_the_ganzfeld_data.htm

2.. This is a minor quibble, but I’m in a sharing mood. In the Running Head paper, the “standardness” was measured from 7 to 1. Thus the average was set at 4. Why would this be? The value of standardness is just a construct. It has no intrinsic value of itself. In other words, an experiment valued with a standardness of 6 is not “twice as standard” as one that scored 3. In the same way, if Manchester Untied beat Arsenal 2-1, it doesn’t mean they played twice as well. The difference may be very small. Additionally, the standardness scale seems to be absolute. Willin’s work with music is rated 1.33 and below that there’s really very little room for a less standard protocol. But I’m sure there must be.

I really don't see the what your (admittedly minor) quibble is all about. Likert type scales are used widely in the social sciences and in this case it gives a very clear indication of how far the raters felt an experiment deviated from standardness.

3.. The meta analyses don’t cover all the experiments. Ganzfeld trials have been carried out since 1973, and the most recent results I know of are from 2003. So that’s thirty years. Honorton’s m-a cover 12 of those (’73-’85) and Bem’s covers ’91 to ’97 (with two Kanthamani experiments from 1988 thrown in, which is odd since there’s no reference to other work from ’88 to ’90) Which leaves 12 years unaccounted for when people boldly talk about m-a and replication. I’ve already mentioned Bierman’s bleak data regarding 1985-93, and from what’s been released since 1998, the most recent experiments taken as a whole don’t seem to demonstrate an effect.

As I've mentioned before, in his 1997 book The Conscious Universe, Dean Radin summarized the results of all the 2,549 ganzfeld studies which had been published up to that time. When the hit rates of all the studies were combined, the average was 33.2%. This is an "astronomically significant" result and demonstrates a strong and replicable phenomenon.


amherst

 

[Paul C. Anagnostopoulos]

I wonder if a lot of the results of ganzfeld experiments is due to selection bias problems?

~~ Paul

 

[Dancing David]

posted by Amhearst
When the hit rates of all the studies were combined, the average was 33.2%. This is an "astronomically significant" result and demonstrates a strong and replicable phenomenon.

So what is the standard deviation and is thirty three percent higher than that?

 

[thaiboxerken]

If you do a meta-analysis of ALL the ganzfield experiments published, I would think you should get a higher than "chance" average. How often to these people published failures, after all? What about the results that are high because of improper protocols and outright cheating, are those included as well?

 

[Ersby]

Surely you are aware that the experiments done at PRL have been replicated by (as of late 97) six different labs, Edinburgh(289 session), Amsterdam(164), Cornell(25), Rhine Center(590), Gothenburg(90), Utrecht(232) all with above chance hit rates consistant with the PRL work, and that the 355 PRL experiments were in fact a replication of the original 762 ganzfeld sessions published in 85.

You know, I’m having a hard time tracking these figures down, mostly because I tend to remember experiments by the person who carried them out, not the Institute where they were held. I’ve got records of 236 sessions from Amsterdam (Beirman’s sessions 1-6) with an average hit rate or 29.2%. From Utrecht (which covers Wezelman’s work, although Beirman collaborates too, correct me if I’m wrong) I have 164 sessions and a hit rate of 26.4%. In short, unless I’m missing something here, I can’t agree they all had hit rates consistent with the PRL results.

The work from Gothenburg (Parker), I agree, is the best work to date with the most robust hit rate. Although, you omitted Durham (Broughton et al) who did a replication of the PRL trails and scored 25.8% hit rate over 209 sessions.

Still can’t find those 590 sessions by Rhine. The only name I know associated with that institute is Palmer, and I can only find a small fraction if this 590. Any ideas?

Next, about Radin’s analysis. I’ll admit, I was surprised when you first mentioned it, since I’ve read quite a lot about ganzfeld and remote viewing, and Radin is hardly ever mentioned. A closer look at the figures you supplied have given me a clue as to why.

Radin’s m-a covers 2,549 sessions (not 2,549 studies, surely. That’d mean an average of over 30 ganzfeld studies a year!). Honorton’s m-a covered 762 sessions. The Palmer, Bem et al covered 1,661. The PRL studies are 354. Add these together, and we’ve already surpassed the 2,549 in Radin’s m-a, and we haven’t even touched the studies from ’83 to ’91. I think the claim that it covers all studies from ’73-’97 isn’t wholly accurate. So my point about missing data stands.

Lastly,

There was no new criteria, I don't know where you got this from. The blind raters were instructed to rate the standardness of the studies according to the degree with which they conformed to the criteria set out by Bem and Honorton in the original Psychological Bulletin paper

I meant it was new in that the previous m-a had not used this criteria. We simply do not know if, under the same standards, we would get the same results. Remember, Hyman’s m-a (covering the same period as Honorton) scored experiments according to flaws in the protocol, and found no effect.

 

[amherst]

You know, I’m having a hard time tracking these figures down, mostly because I tend to remember experiments by the person who carried them out, not the Institute where they were held. I’ve got records of 236 sessions from Amsterdam (Beirman’s sessions 1-6) with an average hit rate or 29.2%. From Utrecht (which covers Wezelman’s work, although Beirman collaborates too, correct me if I’m wrong) I have 164 sessions and a hit rate of 26.4%. In short, unless I’m missing something here, I can’t agree they all had hit rates consistent with the PRL results.

The work from Gothenburg (Parker), I agree, is the best work to date with the most robust hit rate. Although, you omitted Durham (Broughton et al) who did a replication of the PRL trails and scored 25.8% hit rate over 209 sessions.

Still can’t find those 590 sessions by Rhine. The only name I know associated with that institute is Palmer, and I can only find a small fraction if this 590. Any ideas?

Next, about Radin’s analysis. I’ll admit, I was surprised when you first mentioned it, since I’ve read quite a lot about ganzfeld and remote viewing, and Radin is hardly ever mentioned. A closer look at the figures you supplied have given me a clue as to why.

Radin’s m-a covers 2,549 sessions (not 2,549 studies, surely. That’d mean an average of over 30 ganzfeld studies a year!). Honorton’s m-a covered 762 sessions. The Palmer, Bem et al covered 1,661. The PRL studies are 354. Add these together, and we’ve already surpassed the 2,549 in Radin’s m-a, and we haven’t even touched the studies from ’83 to ’91. I think the claim that it covers all studies from ’73-’97 isn’t wholly accurate. So my point about missing data stands.

I quote Radin at length:

"Figure 5.4 summarizes all replication attempts as of early 1997. As before, the graph shows the hit-rate point estimates and 95 percent confidence intervals. The left-most line records the results from the 1985 meta-analysis (indicated as "85 MA"), and the next line to the right shows the Psychophysical Research laboratories (PRL) autoganzfeld results. The numbers in parentheses after each label refer to the number of ganzfeld sessions contributed by the various investigators. Thus, the 1985 meta-analysis hit rate was based on a total of 762 separate sessions."

He then shows a graph listing all the sessions from each lab, their point estimates, and their 95% confidence intervals:

(762 sessions) 85 meta-analysis at 37% with a 95% confidence interval from about 33% to 39%, (355) PRL at 34% with a 95% confidence interval from about 30% to 37%, (289) Edinburgh at about 39% with a 95 percent confidence interval from about 34% to 44%, (164) Amsterdam at about 29% with a 95% confidence interval from about 23% to 35%, (25) Cornell at about 35% with a 95% confidence interval from about 19% to 53%, (590) Rhine Center at about 27% with a 95% confidence interval from about 24% and 30%, (90) Gothenburg at about 30% with a 95% confidence interval from about 23% to 38%, (232) Utrecht at about 28% with a 95% confidence interval from about 24% to 34%.

The combined results of all the sessions show an overall hit rate of 33.2% with a 95% confidence interval between about 31% and 34%.

Radin continues:
"The next replications were reported by psychologist Kathy Dalton and her colleagues at the Koestler Chair of parapsychology, Department of Psychology, University of Edinburgh, Scotland. The Edinburgh experiments, conducted from 1993 through 1996 (and still ongoing) consisted of five published reports and 289 sessions using an improved, fully automated psi ganzfeld setup. It was based on Honorton's original autoganzfeld design and implemented in stages first by Honorton, then by psychologist Robin Taylor, then by me, and finally by Kevin Dalton. Other replications have been reported by Professor Dick Bierman of the Department of Psychology at the University of Amsterdam; Professor Daryl Bem of Cornell University's Psychology Department; Dr. Richard Broughton and colleagues at the Rhine Research Center in Durham, North Carolina; Professor Adrian Parker and colleagues at the University of Gothenburg, Sweden; and doctoral student Rens Wezelman from the Institute for Parapsychology in Utrecht, Netherlands.
While only the 1985 meta-analysis, the autoganzfeld study, and the Edinburgh study independently produced a hit rate with 95% confidence intervals beyond chance expectation, it is noteworthy that each of the six replication studies (after the autoganzfeld) resulted in point estimates greater than chance. The 95 percent confidence interval at the right end of the graph is the combined estimate based on all available ganzfeld sessions, consisting of a total of 2,549 sessions. The overall hit rate of 33.2% is unlikely with odds against chance beyond a million billion to one."

So unless you think Radin is lying or somehow mistaken about how many sessions had been reported up to that time, and/or lying/mistaken about their results, then this is compelling proof that the ganzfeld experiments have been replicated.

A few more things:
1.Where did you get "The Palmer, Bem et al covered 1,661" from?
2. The Rhine Center is located in Durham.

Lastly,



I meant it was new in that the previous m-a had not used this criteria. We simply do not know if, under the same standards, we would get the same results. Remember, Hyman’s m-a (covering the same period as Honorton) scored experiments according to flaws in the protocol, and found no effect.

Are you suggesting the PRL studies reported in the original Psych Bull paper did not use the criteria Bem/Honorton said they did? Because this is the information the raters used to rate the experiments Milton and Wiseman used in their meta-analysis.

Further, in the response to the Milton/Wiseman paper, the raters weren't scoring for flaws, and were completely blind as to the outcome of each experiment (unlike Hyman).

And as far as Hyman's original meta-analysis goes, you do know that:
"None of the commentators agreed with Hyman, while two statisticians and two psychologists not previously associated with this debate explicitly agreed with Honorton."(Radin,97)?

amherst

 

[Ersby]

I quote Radin at length:

"Figure 5.4 summarizes all replication attempts as of early 1997. As before, the graph shows the hit-rate point estimates and 95 percent confidence intervals. The left-most line records the results from the 1985 meta-analysis (indicated as "85 MA"), and the next line to the right shows the Psychophysical Research laboratories (PRL) autoganzfeld results. The numbers in parentheses after each label refer to the number of ganzfeld sessions contributed by the various investigators. Thus, the 1985 meta-analysis hit rate was based on a total of 762 separate sessions."

He then shows a graph listing all the sessions from each lab, their point estimates, and their 95% confidence intervals:

(762 sessions) 85 meta-analysis at 37% with a 95% confidence interval from about 33% to 39%, (355) PRL at 34% with a 95% confidence interval from about 30% to 37%, (289) Edinburgh at about 39% with a 95 percent confidence interval from about 34% to 44%, (164) Amsterdam at about 29% with a 95% confidence interval from about 23% to 35%, (25) Cornell at about 35% with a 95% confidence interval from about 19% to 53%, (590) Rhine Center at about 27% with a 95% confidence interval from about 24% and 30%, (90) Gothenburg at about 30% with a 95% confidence interval from about 23% to 38%, (232) Utrecht at about 28% with a 95% confidence interval from about 24% to 34%.

The combined results of all the sessions show an overall hit rate of 33.2% with a 95% confidence interval between about 31% and 34%.

Radin continues:
"The next replications were reported by psychologist Kathy Dalton and her colleagues at the Koestler Chair of parapsychology, Department of Psychology, University of Edinburgh, Scotland. The Edinburgh experiments, conducted from 1993 through 1996 (and still ongoing) consisted of five published reports and 289 sessions using an improved, fully automated psi ganzfeld setup. It was based on Honorton's original autoganzfeld design and implemented in stages first by Honorton, then by psychologist Robin Taylor, then by me, and finally by Kevin Dalton. Other replications have been reported by Professor Dick Bierman of the Department of Psychology at the University of Amsterdam; Professor Daryl Bem of Cornell University's Psychology Department; Dr. Richard Broughton and colleagues at the Rhine Research Center in Durham, North Carolina; Professor Adrian Parker and colleagues at the University of Gothenburg, Sweden; and doctoral student Rens Wezelman from the Institute for Parapsychology in Utrecht, Netherlands.
While only the 1985 meta-analysis, the autoganzfeld study, and the Edinburgh study independently produced a hit rate with 95% confidence intervals beyond chance expectation, it is noteworthy that each of the six replication studies (after the autoganzfeld) resulted in point estimates greater than chance. The 95 percent confidence interval at the right end of the graph is the combined estimate based on all available ganzfeld sessions, consisting of a total of 2,549 sessions. The overall hit rate of 33.2% is unlikely with odds against chance beyond a million billion to one."

So unless you think Radin is lying or somehow mistaken about how many sessions had been reported up to that time, and/or lying/mistaken about their results, then this is compelling proof that the ganzfeld experiments have been replicated.

A few more things:
1.Where did you get "The Palmer, Bem et al covered 1,661" from?
2. The Rhine Center is located in Durham.

Are you suggesting the PRL studies reported in the original Psych Bull paper did not use the criteria Bem/Honorton said they did? Because this is the information the raters used to rate the experiments Milton and Wiseman used in their meta-analysis.

Further, in the response to the Milton/Wiseman paper, the raters weren't scoring for flaws, and were completely blind as to the outcome of each experiment (unlike Hyman).

And as far as Hyman's original meta-analysis goes, you do know that:
"None of the commentators agreed with Hyman, while two statisticians and two psychologists not previously associated with this debate explicitly agreed with Honorton."(Radin,97)?

amherst

Thanks for such a detailed response. I'll take the easy bits first.

I'll have to search about for reactions to Hyman's m-a. I wasn't aware of that.

I'm not saying that PRL didn't use the criteria they said. I'm saying that the Honorton m-a didn't use "standardness" as a criteria when sorting the experiments for inclusion in the m-a. Until this is done, it cannot be said with any certainty that the two corrspond.

I got the 1,661 figure by adding together the no. of sessions in the experiments used in the m-a. Simple, really.

I didn't know that Durham=Rhine. That accounts for 209 sessions attributed to Rhine but those results were at chance.

I do think that Radin may have got the numbers wrong. Plus, from the figures you provide, it's clear he is missing data. I still maintain that his assertion "each of the six replication studies (after the autoganzfeld) resulted in point estimates greater than chance" is correct (see my above comments on Beirman, Wezelman, Broughton)

 

[Ersby]

At the end of my last post I said “correct”. I did, of course, mean “incorrect”. Silly me.

About the stuff about Ray Hyman (and let’s not get too deeply into this. It’s dull enough throwing quotes back and forth, without having to defend/attack the honour/competence of people we’ve never met) I found this:

“Hyman found a highly significant overall effect in the database, but concluded that this effect was negated as he found a significant relationship between the study outcomes and procedural and statistical flaws contained in the studies. However, Hyman's flaw categorisations were severely criticised by Honorton, and a psychometrician, Saunders, found faults in Hyman's statistical analyses.” (“Experimental Evidence Suggestive of Anomalous Consciousness Interactions” Deborah L. Delanoy)

So it looks like the statistical procedures by Hyman were suspect. Perhaps. As for his rating of successful experiment to flawed experiment, only Honorton disagrees, and we already knew that. The other statistician you mentioned is probably Rosenthal, about whom Hyman says “Because Harris and Rosenthal did not themselves do a first-hand evaluation of the ganzfeld experiments, and because they used Honorton's ratings for their illustration, I did not refer to their analysis when I wrote my draft for the chapter on the paranormal. Rosenthal told me, in a letter, that he had arbitrarily used Honorton's ratings rather than mine because they were the most recent available. I assumed that Harris and Rosenthal were using Honorton's sample and ratings to illustrate meta-analytic procedures. I did not believe they were making substantive contribution to the debate.” (Hyman’s comment In response to the paper “Replication and Meta-Analysis in Parapsychology” by Jessica Utts.)

I can’t find anything about the two psychologists, but then I personally feel psychology is a soft science like parapsychology so can’t really take them too seriously. If they had no previous knowledge of the subject, then who’s to say if there input is of any use?

 

[amherst]

Thanks for such a detailed response. I'll take the easy bits first.

I'll have to search about for reactions to Hyman's m-a. I wasn't aware of that.

I've already posted quite a bit of information on this in a previous post. Here it is again:

In his essay, Rhetoric Over Substance: The Impoverished State of Skepticism, Charles Honorton wrote:

"The next line of criticism concerned the effects of procedural flaws on the study outcomes. In our meta-analysis of the ganzfeld studies, Hyman and I independendently coded each study's procedures with respect to potential flaws involving sensory cues, randomization method, security, and so on. Here Hyman and I did not agree: my analysis showed no significant relationship between these variables and study success, while Hyman claimed that some of the flaw variables, such as the type of randimization, did correlate with results. In his initial assessment, Hyman claimed there was a nearly perfect linear correlation between the number of flaws in the study and its success (Hyman, 1982); this analysis contained a large number of errors that Hyman later attributed to typing errors (communication to Honorton, November 29, 1982). Later, Hyman (1985) claimed a significant relationship between study flaws and outcomes based on a complex multivariate analysis. However, an independent psychological statistician described the analysis as "meaningless" (Saunders 1985). Finally, Hyman agreed that "the present data base does not support any firm conclusion about the relationship between study flaws and study outcome" (Hyman & Honorton, 1986, p. 353). Were our differences in flaw assessment simply reflections of our respective biases? Perhaps, but independent examination of the issue by non-parapsychologists has unanimously failed to support Hyman's conclusions (Atkinson, Atkinson, Smith & Bem. 1990; Harris & Rosenthal, 1988a, 1988b; Saunders, 1985; Utts, 1991). In an independent analysis using Hyman's own flaw codings, two behavioral science methodologists concluded, "Our analysis of the effects of flaws on study outcomes lends no support to the hypothesis that Ganzfeld research results are a significant function of the set of flaw variables" (Harris & Rosenthal, 1988b, p. 3)."

I'm not saying that PRL didn't use the criteria they said. I'm saying that the Honorton m-a didn't use "standardness" as a criteria when sorting the experiments for inclusion in the m-a. Until this is done, it cannot be said with any certainty that the two corrspond.

The criteria I'm talking about is the standardness criteria. Unlike the Milton/Wiseman meta-analysis, the studies from the 94 autoganz meta-analysis were homogeneous . They all followed the standard criteria, set out by Bem and Honorton in their paper. This standardness criteria is what the blind Cornell grad students used to rate the standardness of the Milton/Wiseman database. And there was no "sorting the experiments for inclusion in the m-a." Everything Honorton did at PRL was reported in the original paper.

I got the 1,661 figure by adding together the no. of sessions in the experiments used in the m-a. Simple, really.
I didn't know that Durham=Rhine. That accounts for 209 sessions attributed to Rhine but those results were at chance.
I do think that Radin may have got the numbers wrong. Plus, from the figures you provide, it's clear he is missing data.

It's not at all clear to me that he is missing data. What data do you think he is missing? Show me the studies he failed to include.

I still maintain that his assertion "each of the six replication studies (after the autoganzfeld) resulted in point estimates greater than chance" is (in)correct (see my above comments on Beirman, Wezelman, Broughton)

Why do you still stand by that assertion? All the studies from Beirman, Wezelman, and Broughton were included in Radin's analysis. It is really starting to become hard to understand what your criticisms are.

amherst

 

[amherst]

At the end of my last post I said “correct”. I did, of course, mean “incorrect”. Silly me.

About the stuff about Ray Hyman (and let’s not get too deeply into this. It’s dull enough throwing quotes back and forth, without having to defend/attack the honour/competence of people we’ve never met) I found this:

“Hyman found a highly significant overall effect in the database, but concluded that this effect was negated as he found a significant relationship between the study outcomes and procedural and statistical flaws contained in the studies. However, Hyman's flaw categorisations were severely criticised by Honorton, and a psychometrician, Saunders, found faults in Hyman's statistical analyses.” (“Experimental Evidence Suggestive of Anomalous Consciousness Interactions” Deborah L. Delanoy)

So it looks like the statistical procedures by Hyman were suspect. Perhaps. As for his rating of successful experiment to flawed experiment, only Honorton disagrees, and we already knew that.

"only Honorton disagrees"
??? Read again the excerpt from Honorton's essay which I reprinted in my last post.

The other statistician you mentioned is probably Rosenthal, about whom Hyman says “Because Harris and Rosenthal did not themselves do a first-hand evaluation of the ganzfeld experiments, and because they used Honorton's ratings for their illustration, I did not refer to their analysis when I wrote my draft for the chapter on the paranormal. Rosenthal told me, in a letter, that he had arbitrarily used Honorton's ratings rather than mine because they were the most recent available. I assumed that Harris and Rosenthal were using Honorton's sample and ratings to illustrate meta-analytic procedures. I did not believe they were making substantive contribution to the debate.” (Hyman’s comment In response to the paper “Replication and Meta-Analysis in Parapsychology” by Jessica Utts.)

Jessica Utts:
"In discussing the paper and postscript by Harris and Rosenthal, Hyman stated that "The alleged contradictory conclusions [to the National Research Council report] of Harris and Rosenthal are based on a meta-analysis that supports Honorton's position when Honorton's [flaw] ratings are used and supports my position when my ratings are used." He believes that Harris and Rosenthal (and I) failed to see this point because the lower power of the test associated with their analysis was not taken into account.

The analysis in question was based on a canonical correlation between flaw ratings and measures of successful outcome for the ganzfeld studies. The canonical correlation was 0.46, a value Hyman finds to be impressive. What he has failed to take into account however, is that a canonical correlation gives only the magnitude of the relationship, and not the direction. A careful reading of Harris and Rosenthal (1988b) reveals that their analysis actually contradicted the idea that the flaws could account for the successful ganzfeld results, since "Interestingly, three of the six flaw variables correlated positively with the flaw canonical variable and with the outcome canonical variable but three correlated negatively" (page 2, italics added). Rosenthal (personal communication, July 23, 1991) verified that this was indeed the point he was trying to make. Readers who are interested in drawing their own conclusions from first-hand analyses can find Hyman's original flaw codings in an Appendix to his paper (Hyman, 1985, pages 44-49)."
http://anson.ucdavis.edu/~utts/91rmp-r.html

I can’t find anything about the two psychologists, but then I personally feel psychology is a soft science like parapsychology so can’t really take them too seriously. If they had no previous knowledge of the subject, then who’s to say if there input is of any use?

Huh? No previous knowledge of the subject? The subject is the proper use of meta-analysis, something psychologists like Atkinson, Atkinson, Smith & Bem, and Harris & Rosenthal are extremely well versed in.

"I personally feel psychology is a soft science like parapsychology so can’t really take them too seriously."

You do know that Hyman, Alcock, Blackmore, most of CSICOP's more prominent skeptics, are psychologists?

By the way, Bem did graduate work in physics at MIT.
http://homepage.mac.com/dbem/bem_bio.htm

amherst