Version two
The question of what disease to target is still an open one. I've yet to hear any suggestions...
Changes: more specifics, the addition & elaboration of the selection-bias prevention step, and dropping the use of information as a controlled variable (since I can't think of an explanation that would have getting more information make the results worse, and some worldviews by which the opposite would be true, I'm assigning it as a baseline).
Enjoy.
One clarification: for the purposes of the challenge, all of part one is considered to be irrelevant. The only part that is actually under question is part two, measured as described.
Of course, JREF is welcome to be involved through the entire course of the study if it so desires.
[copying current version of the design draft below]
---------------------------------------------------------
A new study on remote healing - DRAFT
Sai Emrys, 8/13/05
Summary
There have been studies done before on the efficacy of prayer. They have, however, often suffered from some important flaws that render them nearly useless for answering the two main initial questions on the topic: does it work, and if so, how can it be made to work better? (The question of how it works is obviously a rather more difficult one, and in no way addressed by this.)
Previous studies have made mistakes:
• being too vulnerable to a purely “placebo effect†explanation;
• having far too small a sample size to be worth anything statistically;
• having significant breaches of double-blind;
• not tracking potentially important variables that can then be pointed to as confounds;
• not being controlled, or being purely retrospective;
• admitting numerous biases - primarily selection bias - for the measurements;
• having unmeasurable results.
The number of studies that manage not to have one (or many) of these flaws is simply too few. So, I want to run a study that will address this.
First, I want to be clear on what this study is designed to examine: remote prayer. “Remote healing†is a more accurate but less intuitive description. Simply put, it will measure whether – and if yes, how much – people with serious illnesses are affected by people praying for them, in a remote location, having no communication between the two sets of subjects. More specifically, it will only address the question of whether the prayer that occurs as a result of this study has any impact; see below for details.
There is no anticipated possibility of harm to any participant – except perhaps religious self-doubt in the event that prayer does not work for the purposes of this experiment. No participants or researchers will be remunerated in any way. Any participant is free to abandon the experiment at any time they wish by notifying the researcher; any data from their participation will be removed from the experiment, and they will not be involved in any further trials.
Any donations will go towards covering costs of running this study. Any excess remaining at the end of the study will be donated to Médecins sans Frontiers.
Review of previous attempts
(pending completion of literature research)
Procedure
People involved
The recipients will be a group of people drawn from various hospitals, who all have similar, serious, terminal conditions. They will be asked to answer a short survey (demographics, views on religion, etc. – see below), give informed consent to participating in the study, and choose how they want their information to be used. In no event will they know whether they were selected to be prayed for or not, or by whom, unless they abandon the study or the study comes to an end; nor will their doctors.
Potential disease candidates include: late-stage AIDS; cancer; etc. Ideally, the condition chosen will be something that has some chance of cure or spontaneous remission; few acute (vs. ongoing) medical treatments; low life expectancy (~50%); and easily rated progress. Having a moderate amount of various trackable characteristics – e.g. pain, typical days spent in hospital, typical quality-of-life ratings, etc. – will make observing any change as a result of the study be easier.
The healers (as a generic and semantically unambiguous form of “pray-ersâ€) will be drawn from various religious groups, preferably spread across the world and from a diverse pool of religious beliefs and practices. Each will be assigned to pray for one recipient at a time (and each recipient will have one or zero people assigned to pray for them), once a week, for six weeks. They will be asked to keep a log – to verify that they did in fact carry out their duties – and of course to note any particular unusual events or experiences they wish to record.
Each healer would be given personally unidentifiable but personalizing information about their assigned recipient, such as demographics; first name; general location (e.g. state); detailed description of their diagnosis; and if possible, photo and detailed updates on the recipient’s status, to the extent that these are available.
Overall procedure:
All patients will be tracked as to the course of their disease, their symptoms, etc. (See below.)
This preliminary study will be run in two parts. Each part will involve at least 50 “healers†– or whatever portion thereof remain, if participants drop out and are not replaced – praying, one time a week for four weeks each, in succession, for at least four “recipientsâ€. (I.e. recipient A for one month, then recipient B for one month, etc.)
The first part will involve four subgroups under the main “test†group (informed x directed, as described below), and will not have a predefined outcome measure. It will be used solely to determine what an optimal outcome measure would be.
Part one:
Each recipient will be randomly placed into one of two groups: control (50%), and test (50%). The test group will be further split into two equal parts: directed and undirected. (These latter groups do not affect the central question of whether prayer works at all; all of them will receive an equal amount of prayer, just in different ways. They are there solely as a speculative inclusion from the results of other studies, to be used as below.)
Those in the “directed†group will have their healer instructed to pray for specific benefits. E.g., for the cure of their disease, reduction in pain, prolongation of life, and general happiness etc. Those in the “undirected†group will have their healer instructed to pray in a purely general way: e.g. along the lines of “may God’s (Universe / Mother’s / etc) will be doneâ€. This will stay the same for any particular recipient, but will change – 50-50 – for each healer.
Each “healer†would be assigned to two recipients from both of these groups, in blocks. (i.e. two of one, then two of the other.) The order will be randomized – half of the healers will be instructed to “directed†first, the other half “undirectedâ€.
Health measurements taken from the beginning of the study until one month after the end of this phase, will be used to decide which of the four trial groups appears to show the best result, and what combination and weighting of tracked disease statistics produces the greatest difference between control and test groups. This may include items such as subjective symptom reports, doctor prognoses, days in hospital, money spent on treatment, survival length, etc.
This “data massage†will be at the discretion of the researcher. (One alternative to choosing a fixed option on directedness would be to direct “healers†to use whichever they are more used to / attracted to. This preference would be polled, to determine if there is any crossover effect between healer preference and the direction given.)
The recipients and/or healers may also be trimmed by particular properties, if this is determined to have a significant benefit to the effect gained – e.g. by only using those of a certain religion, experience level, praying style, belief in prayer, etc. That is, these traits would become selection criteria for participating in the second part of the study. This will, again, only be done if it is expected (based on part one data) that doing so will increase the effect.
In any event, the final measure decided upon will be a single, numeric expression per “recipientâ€, based solely upon specific data gathered about their health condition, and with clear admission criteria for both recipients and healers.
Thus ends the complicated part. There will be a two month (or longer, if necessary to process data) pause in the study to give the researcher time to compile existing data and determine optimal measures, instructions, etc., to use for the second part.
Part two:
Crucially, the second part will involve one trial group – that is, there will be no controlled variables other than the 50-50 split of controls vs. non-controls, which is newly and randomly assigned from an “untouched†pool of recipients. All healers will be given the same instructions – that is, either to engage in directed, non-directed, or by-preference prayer (as determined in part one). The measure fixed upon at the end of part one will be used to judge the data from part two, and will in no event be modified after the commencement of part two. Lastly, all data from part one – that is, all data that went into determining what measure to use – will be discarded.
Preferably, part two will last longer and involve more participants. This would be determined by resource and practicality constraints, however; the minimum would be the same as for part 1 – 50 people praying for 4 recipients each, for one month each, with each recipient being assigned zero or one healers.
The measure determined at the end of part one would then be applied to the information gathered from part two. This will be used as the basis for determining whether there is a statistically significant difference between the control and test groups.
All participants will be required to sign a release which:
• states their understanding of the facts presented in this document, and willingness to participate;
• swears that all information they provide is true to the best of their knowledge; and
• describes wishes in regard to the disclosure of their confidential information (e.g. being released to their paired partner, to other researchers, to media inquiries, etc.).
That’s it, really. If recipients and healers mutually wish to have their contact info released to each other, they will be introduced; if not, not. Everyone will get a copy of the overall results, as well as a copy of the results of their own trials.
Potential confounds & arguments against validity
1. It is possible that prayer does not work, for some essential reason, when it is under study in this manner.
This, and all other apologist arguments in the same vein, is by definition not controllable. If prayer does indeed have such a limitation, then this study is moot; if not, then it is not a concern.
2. It is possible – probable even – that “recipients†will be prayed for by people other than their assigned “healersâ€.
This variable – and all other otherwise uncontrolled variables – is controlled by the process of randomization; there is no reason why it would be different between the control and test groups. It will, however, be tracked – by polling the recipients as to whether they, or people they know of, will be praying for their betterment. It is expected that this measure will be statistically equal between groups, as with other statistics such as average age, gender distribution, etc.
3. Falsification on the part of participants, e.g. “healers†not carrying out their assigned duties
The only effect this can have on the outcome of the study is to result in a false negative, or in a result that is smaller than might otherwise be produced. Given the practical difficulties that would be involved in doing anything but taking people at their word – and the fact that this could in no way produce a false positive – this will be ignored.
4. “Data miningâ€, a.k.a. selection bias
The dual tier design shields the development of admission criteria, health measure, etc., from tainting the results. Once these criteria are determined, they will be final; since they are determined before the obtainment of any data that goes into the final results, there is no opportunity for selection bias.
5. Significance
As this is intended solely to be a preliminary, exploratory study, a statistical significance of 95% or higher (p<0.05) will be considered a positive outcome.
A result – that is, a statistically significant difference between the average scores in the control vs. test groups – at that level, of any magnitude, will be considered justification for a full study of identical design as the second part, with a new set of “recipients†and “healersâ€, and with a significantly larger pool of participants.
6. Violation of double-blind
For the nitpicker: provision of personal information about the recipient to the healer could be considered a breach of complete double-blinding. However, one would be required to come up with an explanation of how this would have an effect on the patient despite the total lack of communication path from healer to recipient – something not possible except by admitting the existence of a non-mundane means of communication between the two. Thus, this can be considered a moot point.
Information Tracked
Info gathered on recipient:
• General:
o Name
o Random ID number
o Random confirmation number (disclosed only to recipient)
o Demographics
 Gender
 Age / date of birth
 Religion
• Type
• Fervency / activity
• Length of time practicing
o This religion
o Anything seriously
 Socioeconomics
• Ethnicity
• Income
• Parents’ income
 Location
o Picture
o Belief in the efficacy of prayer
o Recipient of prayer otherwise
 Self-prayer
• Frequency
• Style
• Length
 Known recipient (e.g. church, family)
• Ditto
o Personality (intro/extroversion etc)
• Disease
o Prognosis at start (e.g. expected survival rate, life expectancy, etc)
o Dated progress notes
o Numeric measures
 Amount of meds used by type (e.g. anesthetics)
 # days in hospital
 # “incidents†(positive or negative)
 Self-reported pain / quality-of-life scores
 $ spent in treatment
 Doctor skill (e.g. # years practicing medicine)
 Length of survival
Info gathered on healer:
• Same demographics etc
• Usual praying style
o Directed / undirected
o Ritual / group / appeal to ___ / …
o Duration of one “prayerâ€
• Status as a celebrity or professional related to remote healing in any way (e.g. reiki practioner, “psychicâ€, monk, professional priest, etc.)
o Years claiming this
Variables:
• Prayed for or not
• Directed vs. undirected prayer
Variables to try controlled testing later:
• # times prayed for (1 vs. 8)
• Personal information about recipient given to healer (e.g. name, gender, age, photo, detailed description of current problem / past history, status updates, general location, vs. minimal)
• Religion / method of healer
• Religion / belief in remote healing (“prayerâ€) of recipient
• IF measuring against an event (e.g. surgery):
o Synchronization WRT prayer (i.e., before, after, during, etc)
Use of Information – Privacy Policy
There are three classes of information gathered on participants in this study: not personally identifiable, personalizing, and confidential. They are treated differently, as below.
Not personally identifiable information includes all medical information about a particular trial when associated only with an ID number (and the ID number itself), and all aggregated information about participants by category (e.g. average age, gender proportions, etc). This information may be used and released freely; there shall be no restriction on their use.
Personalizing information is that which is given to the healers about their assigned recipients. That is, basic demographics, first name (NOT last), physical description, full diagnosis, location (no more specific than state), and (if available) photo and detailed status updates. This will only be released (under condition of non-disclosure) to the assigned healer.
All other information will be considered confidential. Such information – e.g. full name, contact information, etc. – will only be released with the explicit consent of the participant involved. If both “recipient†and “healer†in a particular trial indicate a wish to have contact with each other, they will be given each others’ contact information at the end of the study, or when either decides to abandon the study.
Of course, the recipient may not know anything about their status in the study – e.g. whether they are in the control or test groups, or anything about the “healer†assigned to them if there is one – until the study is over. Likewise for the “healerâ€, aside from the clause above.
