"I've been reading two of the PEAR papers:
Information and Uncertainty: 25 Years of Remote Perception Research and
Evidence for Consciousness-Related Anomalies in Random Physical Systems
I haven't finished yet; the latter paper is only sixteen pages but contains statistical analysis which requires checking, while the former is 74 pages and is heavy on jargon. I can at this point raise a few concerns:
In Evidence for Consciousness:
1. Several key figures are missing from the paper. This may have occurred when it was prepared for the web; whatever the reason, it means the paper is seriously flawed as it stands.
2. The paper discusses a methodology for judging the quality of a parapsychological experiment, and then moves on to drawing conclusions based on this evaluation. The actual results of this evaluation are nowhere to be found.
3. The paper focuses on three studies; the one it finds most interesting is a study performed by Jahn and Dunne. Jahn and Dunne are colleagues of the authors of this paper and have published a number of joint papers. Curiously, the authors, Radin and Nelson, neglect to mention this, even though you find one of these papers referenced in the footnotes.Quite frankly, if the paper had been submitted for publication in this form, any conscientious referee would have rejected it.
With Information and Uncertainty, the problems are slightly different. Even in the abstract we are bombarded with jargon; for a moment I thought I was reading Alan Sokal's Transgressing the Boundaries: Toward a Transformative Hermeneutics of Quantum Gravity again.
quote: The possibility that this increased emphasis on objective quantification of the phenomenon somehow may have inhibited its inherently subjective expression is explored in the contexts of contemporary signal processing technologies and ancient divination traditions.
Well, many scientific papers are hard to read; this is not to say they may not be valuable. But one soon begins to wonder if the experimenters are deliberately trying to produce bad science:
quote: The agents, who in all the formal experiments were known to the percipients...
In other words, the test was not blinded in any way.
quote: While no explicit tactical instructions were given, an attitude of playfulness was encouraged and emphasis was placed on enjoyment of the experience, rather than on achievement per se.
Playfulness having been well-demonstrated as a useful experimental too.
quote: No systematic records were maintained on the relative effectiveness of the various personal strategies deployed by the participants in their approach to the task, or on any of their psychological or physiological characteristics.
Not keeping records also being a popular tool of experimental science.
Here's a doozy:
quote: Furthermore, although limiting the extracted information to the 30 specified binary descriptors minimized the reporting task for the participants, it precluded utilization of other potentially relevant information in the transcripts, such as specific colors, textures, architectures, or any other details not covered by the questions. These shortcomings were partially offset by the continued requirement that percipients first generate free-response descriptions from which the descriptor responses were then derived. In this way, the data were preserved in a format suitable for subsequent human judge evaluation as a complementary method of assessment; it also was intended to retain the spontaneity of the PRP experience. Nonetheless, it became evident that after several experiences with the descriptor utilization, many participants tended to limit their attention and descriptions to those features that they now knew were specific to the questions. Despite these acknowledged limitations, the program continued to collect additional data, and also instituted means of assessing the effectiveness of the individual descriptors in constructing the scores. This was accomplished by incorporating a variety of subroutines into the scoring algorithms that compounded descriptor performance in the form of sums of the number of the times each descriptor was correctly answered “yes†and how many times it was correctly answered “no,†again adjusted for a priori probabilities of occurrence.
In other words, the experiment was not proceeding the way the experimenters wished, so they changed the rules and kept going! Then things start to get really bad:
quote: The failure of the FIDO experiments to enhance the declining PRP yield prompted the PEAR researchers to look elsewhere for the source of the problem. One of the laboratory’s human/machine studies had indicated that operator pairs of opposite sex, working together with a shared intention, produced stronger effects than same-sex pairs or individual operators. This, in turn, had led to a comprehensive examination of nine of PEAR’s human/machine databases, which were found to display significant gender-related differences in individual operator performance.
The old grasping-at-straws approach, as exemplified by Galileo and Milliken.
quote: Although there was reasonably good agreement among the six scoring recipes, the overall results of these distributive data were totally indistinguishable from chance, despite the fact that from an impressionistic standpoint several of the individual trials appeared to be successful.
In other words: nothing was happening, even though it looked like it was if you had no understanding of probability or experimental method.
quote: The substantial difference between the yields of the ex post facto and ab initio data, however, did raise some concern that the former, on which the descriptor questions and methodology initially had been based, could have introduced a spurious score inflation into the composite database.
But, even with these concerns:
quote: Given the systematic deterioration of the experimental yield,
that is, as they tightened the experimental protocol, the number of positive hits dropped off
quote: further generation of data was suspended pending a clearer understanding of the underlying problem. Our approach to this challenge was to return to the original ex post facto data, which had survived the onslaught of a battery of human judges and an armada of analytical methods without appreciable damage, and use it as a basis for comparing the relative effectiveness of the various encoding and analytical techniques on the apparent yield of a given set of trials.
Since the high-quality data doesn't bear out their expectations, they decide to use to low-quality data instead. They also seem to be surprised that subjectivity affects the result:
quote: If anything, the subjectivity inherent in the distributive encoding actually appeared to be confounding the experimental yield without benefit to the analytical process.
Or possibly not. Can anyone tell me with any certainty what that sentence is supposed to mean?
quote: It also is curious that these three judges agreed on the correct match of only one of the 31 trials.
Yes, "curious" is one term for this result. Here's the worst I've found so far:
quote: The evidence acquired in the early remote perception trials had raised profound questions in the minds of the PEAR researchers, similar, no doubt, to those of the countless others who, over the course of history, had experienced first-hand the validity of Paracelsus’ remarkable claim.
This simply has no place in anything purporting to be a scientific paper. But despite this alarming bit of brain-failure, they proceed:
quote: The possibility that ordinary individuals can acquire information about distant events by these means, even before they take place, challenges some of the most fundamental premises of the prevailing scientific world view. Yet, difficult as it may be at times, the true spirit of science requires humility in the face of experimental evidence, even when, or especially when, that evidence suggests that our existing models of reality may be wrong, or at least incomplete.
In other words, they are assuming the existence of the effect they have set out to test. This is not generally regarded as helpful. Actually, maybe the "Paracelsus" quote wasn' the worst. Here's something that brought me up short:
quote: Yet, like so much of the research in consciousness-related anomalies, replication, enhancement, and interpretation of these results proved elusive. As the program advanced and the analytical techniques became more sophisticated, the empirical results became weaker. It appeared as if each subsequent refinement of the analytical process, intended to improve the quality and reliability of the “information net,†had resulted in a reduction of the amount of raw information being captured. This diminution of the experimental yield prompted extensive examination of numerous factors that could have contributed to it, but after exploring and precluding various possible sources of statistical or procedural artifact, we concluded that the cause of the problem most likely lay somewhere in the subjective sphere of the experience.
In other words:
1. We performed an experiment with lousy controls and indifferent analytical methods and got a strong positive result.
2. Every time we tighten the controls or refine the analysis, the result gets statistically weaker.
3. Therefore the problem lies "somewhere in the subjective speher of the experience".
I dunno. Ya think? You don't think it may indicate that the positive results are somehow related to poor experimental control and poor analytical methods? Evidently not:
quote: As we pondered this paradox, we became cognizant of a number of subtler, less quantifiable factors that also might have had an inhibitory effect on the experiments, such as the laboratory ambience in which the experiments were being conducted. For example, during the period in which the FIDO data were being generated, we were distracted by the need to invest a major effort in preparing a rebuttal to an article critical of PEAR’s PRP program. Most of the issues raised therein were irrelevant, incorrect, or already had been dealt with comprehensively elsewhere, and had been shown to be inadequate to account for the observed effects. Notwithstanding, preparation of a systematic refutation deflected a disproportionate amount of attention from, and dampened the enthusiasm for, the experiments being carried out during that time. Beyond this, in order to forestall further such specious challenges, it led to the imposition of additional unnecessary constraints in the design of the subsequent distributive protocol. Although it is not possible to quantify the influence of such intangible factors, in the study of consciousness-related anomalies where unknown psychological factors appear to be at the heart of the phenomena under study, they cannot be dismissed casually. Neither can they be interpreted easily.
In other words: while we were busy addressing our previous stuff-ups, our latest experiment went to Hell. Further, tight experimental control places "unnecessary constraints" on our research, and this "intangible factor" is inhibiting the "phenomena under study" .No, look, I'm sorry, you're wrong. If this is supposed to be an example of good parapsychology, the bad stuff must be truly mind-boggling!
__________________Read my shiny new blog! It's still utterly pointless and boring, but it looks much nicer. Report this post to a moderator | IP: Logged
12-13-2002 08:56 PM"
