• Quick note - the problem with Youtube videos not embedding on the forum appears to have been fixed, thanks to ZiprHead. If you do still see problems let me know.

Need Help with Randomizing for Experiment

Amapola

Unregistered
A
Joined
Oct 6, 2005
Messages
8,254
I'm trying to set up a proper experiment. A friend of ours believes that praying for water will make it "better" and that plants watered with prayed for water will do better than plants that are watered with normal water - ie, not prayed for water.

His original suggestion was to have two plants, water one with the prayed for water and the other with regular water. However that did not seem like enough plants to me.

I went out and bought 20 identical pots. I have filled them all with the same type of dirt - it is aged manure cleaned from my animal pens and then allowed to sit all winter. I have soaked all 20 pots with water and planted each pot with two bean seeds. These are pole beans - the variety is Kentucky Wonder. I put two seeds in each pot, spaced apart, and will re-seed if one or more fails to come up.

According to our friend we wait until the plants are up and thriving, and then we can begin the experiment.

What I am proposing to do is water 8 with the prayed for water; 8 with "normal" water, and the remaining 4 I will water with "cursed" water. (Hey - if praying works, why not cursing?) I need to go over my cursing protocol with our friend to make sure it meets his standards of "cursed".

My question is this: How do I properly decide which pots get the cursed water, which the prayed for water and which the normal water? I want to be certain I am doing this correctly. Is there a way to randomly decide which pot gets which water? Only I will know. My husband and our friend certainly will not know and I do not plan on telling anyone else until the experiment is done. So how do I randomize it? Please help, and thanks!
 
Last edited:
Amapola said:
I'm trying to set up a proper experiment. A friend of ours believes that praying for water will make it "better" and that plants watered with prayed for water will do better than plants that are watered with normal water - ie, not prayed for water.

His original suggestion was to have two plants, water one with the prayed for water and the other with regular water. However that did not seem like enough plants to me.

I went out and bought 20 identical pots. I have filled them all with the same type of dirt - it is aged manure cleaned from my animal pens and then allowed to sit all winter. I have soaked all 20 pots with water and planted each pot with two bean seeds. These are pole beans - the variety is Kentucky Wonder. I put two seeds in each pot, spaced apart, and will re-seed if one or more fails to come up.

According to our friend we wait until the plants are up and thriving, and then we can begin the experiment.

What I am proposing to do is water 8 with the prayed for water; 8 with "normal" water, and the remaining 4 I will water with "cursed" water. (Hey - if praying works, why not cursing?) I need to go over my cursing protocol with our friend to make sure it meets his standards of "cursed".

My question is this: How do I properly decide which pots get the cursed water, which the prayed for water and which the normal water? I want to be certain I am doing this correctly. Is there a way to randomly decide which pot gets which water? Only I will know. My husband and our friend certainly will not know and I do not plan on telling anyone else until the experiment is done. So how do I randomize it? Please help, and thanks!
Click to expand...

I think the experiment is interesting but it lacks statistical power and wouldn't be a fair test of prayer. I'd drop the n=4 cursed plants and go with 10 versus 10, though I still don't think failure would prove anything (though success would).

How large of an effect does your friend think prayer has on plant growth?

80% power is considered an acceptable minimum. For the study you propose, you would need 393 plants in each group to detect a small effect (.2 standard deviations bigger growth in the prayer group); 64 in each group for a medium effect (.5 sds) and 26 in each if the effect were large (.8).

With the 8 and 8 comparison as you have it, and assuming a medium effect, you only have .26 power.

This means that even if prayer works, there's a 74% chance your experiment won't be sensitive enough to show it.

This calculator lets you play around with values to see how many plants you'd need to get reasonable power:

http://www.dssresearch.com/toolkit/spcalc/power_a2.asp (bumping it to 10 per group gets you about .30 power)

Perhaps changing the experiment would be a good idea. For example, instead of charting plant growth, see if your friend can pick which of the 10 got prayer and which didnt after so many days or weeks of watering. You're testing a slightly different hypothesis, but you would certainly have power here to see if a believer can spot the effects of his/her prayer working.
 
Amapola said:
I'm trying to set up a proper experiment. A friend of ours believes that praying for water will make it "better" and that plants watered with prayed for water will do better than plants that are watered with normal water - ie, not prayed for water.

His original suggestion was to have two plants, water one with the prayed for water and the other with regular water. However that did not seem like enough plants to me.

I went out and bought 20 identical pots. I have filled them all with the same type of dirt - it is aged manure cleaned from my animal pens and then allowed to sit all winter. I have soaked all 20 pots with water and planted each pot with two bean seeds. These are pole beans - the variety is Kentucky Wonder. I put two seeds in each pot, spaced apart, and will re-seed if one or more fails to come up.

According to our friend we wait until the plants are up and thriving, and then we can begin the experiment.

What I am proposing to do is water 8 with the prayed for water; 8 with "normal" water, and the remaining 4 I will water with "cursed" water. (Hey - if praying works, why not cursing?) I need to go over my cursing protocol with our friend to make sure it meets his standards of "cursed".

My question is this: How do I properly decide which pots get the cursed water, which the prayed for water and which the normal water? I want to be certain I am doing this correctly. Is there a way to randomly decide which pot gets which water? Only I will know. My husband and our friend certainly will not know and I do not plan on telling anyone else until the experiment is done. So how do I randomize it? Please help, and thanks!
Click to expand...

perhaps the simplest approach would be to get your husband and friend to divide the plants into 3 groups that they agree are reasonably similar. Securely label them with a hidden label, so that the group can be revealed at the end of the experiment. Then randomly number them in sequence and keep a secret record of the group to which each plant belongs. keep all plants in the same area and maybe arrange to move their relative positions every couple of days to minimise variations ddue to their exact position. Watering would best be done in such a way that if all plants are watered at the same time, no plant is likely to dry out or drown. Sit back and wait.
 
Label the pots A through T. Make corresponding slips of paper. Put the slips of paper into a hat. After shaking the pieces of paper thoroughly, draw the pieces of paper out, assigning one to each group until all pots are assigned. Arrange the pots alphabetically, preferably as close together as possible, so as to exclude any differences in terms of light, music, and other ambient characteristics, and water each according to your list.

I really don't have a clue how to do it mathematically, but that's how I would randomize...
 
I'd say skip the cursing, and just go 10 to 10 with holy vs unholy water.

As to the randomizing, I'd say let your friend mix up the pots while you're not in the room (to eliminate any suspicion later that you put "puny" seeds in one set), then number the pots. Do this now, while they're all just indistinguishable dirtbags.

Once the pots are fairly numbered, flip a coin to decide whether odd or even pots will be favored with the good water.

To be completely fair, a third party should handle the coin flip and the watering, so that neither you nor your friend will know which plants got which water. At the agreed-upon time, the numbers can be covered, and your friend can try to identify which plants were favored by prayer. I'd say if he correctly guesses 8 of the 10, prayer wins.
 
maybe use grass seed and bump up the n size dramatically. Plant 100 in identical containers and start from the first watering with prayer versus non. Honestly, the 10/10 comparison is not worth doing as the certain failure can be written off as poor power.

One thing to consider though is how fun it might be to measure the length of 200 different blades of grass...
 
It would be the binomial distribution for the guessing game.

Each trial would be graded right or wrong. I don't think it would be fair to analyze the probability on just the ones the person picked as prayed for, as there are 10 other "trials" also (the 10 not prayed for).

With n = 20, p = .5 the probability of getting at least

10 right is .59
11 right is .42
12 right is .25
13 right is .13
14 right is .057
15 right is .02

14 right would do it.


If you did do it on just the 10 prayed for then 8 out of 10 would be the magic number, but I'm not sure that's appropriate.

http://rockem.stat.sc.edu/prototype/calculators/index.php3?dist=Binomial
 
sorry for mass posting here. If anyone knows:

would calculating the p on just the 10 picked be kosher? This might be the only type of experimental design where it seems ok (despite what I said above) to analyze only half the observations. 8 of 10 has the same p as 14 of 20.

And, after picking any 10, the fate of the other 10 is sealed (if 8 of 10 picked were correct, than 8 of 10 not picked were correct too).

Edited!

but wait then, that would be 16 of 20 correct which has a much smaller p value than 8 of 10.

This is like the monty hall problem / question. Fascinating, or maybe I need to go to bed.

Hey you could also do signal detection analysis to separate your prayer's true sensitivity at detecting god's work from his/her personality (is she conservative when picking-- must really really look like god touched it before picking it, or liberal-- as long as it sorta looks like god's work, she'll pick it).

The hits would be the % correct on the prayed for ones.
False alarms would be the % (out of 10) where she picked one that wasn't prayed for.

Sensitivity would be hits minus false alarms and you'd predict it to be zero.

Personality would be hits + false alarms. Values greater than 1 suggests she's the "just needs it to sorta look like god's work for me to pick it" type believer.
 
Last edited:
Yeah, doing it the 8/10 way would unfairly penalize the prayer person.

You think she's performing at p=.05 when correctly picking 8 out of 10. But, translating that to 16 out of 20 to account for all observations actually has a p value of .0059. That's 10 x less likely than you would have concluded her performance to be!
 
Amapola said:
My question is this: How do I properly decide which pots get the cursed water, which the prayed for water and which the normal water? I want to be certain I am doing this correctly. Is there a way to randomly decide which pot gets which water?
Click to expand...

Label the pots with the numbers 1 through 20. Then, generate a list of the numbers 1 through 20, in random order, and use the first four, the next eight, and the last eight, respectively.

I agree with others in this thread that splitting the twenty pots into ten and ten is probably better than into four, eight and eight. If you and your friend agree that cursing might be worse than doing nothing but wouldn't be better, you could simply use ten cursed pots and ten blessed ones.
 
bpesta22 said:
It would be the binomial distribution for the guessing game.
Click to expand...

I don't think so. The guesses aren't independent. The guesser knows that ten pots were blessed and ten weren't, so there's no chance that he will guess otherwise.

Here's what I've come up with: The number of correct guesses, out of twenty, will certainly be even. (Every wrong guess that a cursed pot was blessed will be accompanied by another wrong guess that a blessed pot was cursed, because the guesser knows that there are ten pots of each kind.) Assuming that there's no real difference between the pots, the probability of making exactly 2n correct guesses is [latex]$\binom{10}{n}^2/\binom{20}{10}$[/latex], so the probability of at least 14 correct guesses is 0.0894, and of at least 16 is 0.0115.
 
bpesta22 said:
and wouldn't be a fair test of prayer.
Click to expand...

How do you design a fair test for prayer experiments?

I just prayed for all of the pots. Some more than others. Sorry, amapola!!

bokonon said:
As to the randomizing, I'd say let your friend mix up the pots while you're not in the room (to eliminate any suspicion later that you put "puny" seeds in one set)
Click to expand...

Never leave anyone alone with the setup. If there is suspicion that one can fiddle, then all can fiddle.

Who will supervise the whole set while growing? I can make the right ones wither from a distance - just give me 5 seconds and a syringe filled with battery acid. Won't be detected.
 
bpesta22 said:
sorry for mass posting here. If anyone knows:

would calculating the p on just the 10 picked be kosher? This might be the only type of experimental design where it seems ok (despite what I said above) to analyze only half the observations. 8 of 10 has the same p as 14 of 20.
Why do that? Keep it simple and stick to the design you proposed originally. One minor quibble
is that with 14 hits out of 20, p is not less than .05. It is .05765914. Go for 13 hits where p= .02069473.
Click to expand...
 
I think it would be useful to allow the experiment to proceed without blinding - that is, give your friend the opportunity to 'see' dramatic differences in the growth of the plants watered with prayed for water. Then when he is convinced he sees a difference, present the plants to him one-by-one in a blinded manner and ask which group the plant belongs to. You may have to remove the plant from the pot and present it in a different position (such as upside-down) in order to avoid any subtle clues (like scratches on the pot or the shape of the plant). What you want is for him to understand the difference between what we see when our biases are allowed to influence our perceptions and what we see when they are not.

It is difficult to guarantee identical growing conditions (I spent several summers working at an agricultural research centre growing winter wheat so I have (too much :)) direct experience with this problem), so randomization is necessary to try to counteract that effect. I agree with others to drop the idea of cursed water. And there is no necessity to use 10 of each if you are simply asking him which group each plant falls in to at the end (rather than comparing groups). Any method of generating a random result will do. Simply roll the die for each pot and label it with the water it is to receive. You should end up with a reasonably balanced number of prayed-for and normal.

There would need to be a number of other rigorous controls if this were a real experiment (like making sure the two kinds of water are stored in the same manner), but it sounds like the point is really to teach your friend something about perception.

Linda
 
Dodge; thanks for your argument.

I still think it's the binomial distribution though as the probability of guessing right on any of the 20 trials is still .50.

Even if you know it's 10 prayed /10 not, and stop at trial 18 because you've picked 10 that you think were prayed for, the probability that 19 and 20 were not prayed for is still .5 each.

Your strategy would affect which plants you pick (i.e., guess no to all plants after guessing yes on the 10th) but not the probability of being right on any trial.

jc; I agree, though I think you mean at least 15?
 
Jeff Corey said:
Bragging about the significance of your results is known as "p waving".
Click to expand...

What's it mean that my p is bigger than yours? Whatever; lets make sure we keep our p's orthogonal to each other...:boxedin:
 
Last edited:
bpesta22 said:
Dodge; thanks for your argument.

I still think it's the binomial distribution though as the probability of guessing right on any of the 20 trials is still .50.

Even if you know it's 10 prayed /10 not, and stop at trial 18 because you've picked 10 that you think were prayed for, the probability that 19 and 20 were not prayed for is still .5 each.

Your strategy would affect which plants you pick (i.e., guess no to all plants after guessing yes on the 10th) but not the probability of being right on any trial.

jc; I agree, though I think you mean at least 15?
Click to expand...
Yeah, I saw that too late to edit it. It's Saturday AM, after all.
 
You must log in or register to reply here.

Back
Top Bottom