Design a double-blind experiment

[RichardR]

The point Stimpy was making was to hypothesize a mechanism, not to know a mechanism. The two-slit experiments you mention, for example, each hypothesized a mechanism. The researcher said "I think it acts this way. If that is true, and I think I should be able to distinguish that mechanism from other possibilities in the following way." Then he designs and executes the experiment to see how it behaves and whether that behavior refutes or corroborates his hypothesized mechanism.

I’m still a little puzzled. What mechanism was hypothesized? And what mechanism was hypothesized to explain gravity?

 

[RichardR]

Randi doesn't describe this as double-blind in that commentary. He also doesn't specify enough details to know whether it is or isn't double-blinded. If, for example, Randi planned to tape the one-sided reading of the selected sitter and then phone the other ten and play Sylvia's reading to them as if it were for them, that would give equal confirmation bias to each sitter.

OK. If the sitter was not allowed to say anything (utilizing some kind of one-way only phone), and if Randi phoned the other nine and played back the Sylvia tape so they thought it was Sylvia on the line for them, it would work.

 

[BillHoyt]

I’m still a little puzzled. What mechanism was hypothesized? And what mechanism was hypothesized to explain gravity?

RichardR,

There are far too many two-slit experiments to answer that question for all. Which do you want? Young's original experiment? At the time of his experiment (1801), the wave theory of light was a nearly dead issue. Young revived it with the original two-slit experiment design. The experiment was designed to distinguish the wave hypothesis from the particle hypothesis. Young's experimental conclusion was that light was wave-like.

Cheers,

 

[BillHoyt]

OK. If the sitter was not allowed to say anything (utilizing some kind of one-way only phone), and if Randi phoned the other nine and played back the Sylvia tape so they thought it was Sylvia on the line for them, it would work.

Does anybody know where the detailed protocol is on the JREF site? I'd like to see exactly what Randi and Sylvia agreed to .

Cheers,

 

[xouper]

BillHoyt: Does anybody know where the detailed protocol is on the JREF site? I'd like to see exactly what Randi and Sylvia agreed to .

I posted the two links earlier in this thread. That's as detailed as it gets, as far as I know (but I could be wrong).

Stimpson J. Cat: My point was that even if such an experiment were done, with proper controls, and proper data analysis, that it still wouldn't say jack squat about psi.

In the other two threads where this is also being discussed, I have been trying to make that very point.

May I ask, when you say "proper controls", are you referring to proper protocols, or are you referring to having proper test subjects acting as controls?

 

[Stimpson J. Cat]

RichardR,

This puzzles me a bit. Surely we don’t know the mechanism for how quantum mechanics works. When we perform the two slit experiment, and add an electron detector at one of the slits, we find the electron behaves like a particle. But we don’t know how this works, do we? Why is this different?

BillHoyt's clarification is correct. You have to remember that QM did not just spring full grown into the world. It was built from the ground up, through a long process of hypothesis testing. But at each step, each experiment was designed to test a specific, falsifiable hypothesis. That is what is needed here.

When I say "mechanism" I am not talking about an explanation in terms of other known phenomena (which is what is lacking in QM). Such explanations are not even scientifically necessary, since it is a given that whatever the most fundamental part of our scientific knowledge is (currently QM and GR), it is going to be purely descriptive in nature, and lacking such an explanation.

When I say "mechanism" in this context, I am talking about a clear, unambiguous, and falsifiable hypothesis, which both accurately describes some currently non-understood phenomenon, and which makes clear testable predictions.

Of course, before that stage can ever be reached with parapsychology, they must first find some non-understood phenomenon. Hence all the data mining for anomalous statistical results.


Xouper,

May I ask, when you say "proper controls", are you referring to proper protocols, or are you referring to having proper test subjects acting as controls?

I was referring to proper protocols. As someone else mentioned, I do not think that control subjects are even feasible for this type of analysis, since in order for control subjects to be used, it must be the case that if the null-hypothesis is true, then the control subjects should be indistinguishable from the test subjects. Only if this is true, does a statistically significant difference between the test and control subjects allow one to reject the null-hypothesis.

Dr. Stupid

 

[Paul C. Anagnostopoulos]

Dr. Cat said:

I was referring to proper protocols. As someone else mentioned, I do not think that control subjects are even feasible for this type of analysis, since in order for control subjects to be used, it must be the case that if the null-hypothesis is true, then the control subjects should be indistinguishable from the test subjects. Only if this is true, does a statistically significant difference between the test and control subjects allow one to reject the null-hypothesis.

Why wouldn't the control subjects and test subjects be indistinguishable under the null hypothesis? Gee, now that I think about it, I'm not sure what the null or experimental hypotheses are.

~~ Paul

 

[jj]

Hypothesis:

"Clod Readers, Hot Readers, Conjourers, or Trickster experts are unable to replicate the same results as Laurie Campbell John Edward, Suzane Northrop, George Anderson and Anne Gehman, collectively known as 'research mediums' for the HESL, under the conditions and protocols of the research of Dr Schwartz."

Testing the hypothesis will measure one way or the other whether known scientific (mundane) methods are being employed.

We don't need to worry about dead people here, what's important is finding out if mundane means are responsible for the results.

That's not a hypothesis. What constitutes "replication"?

The holes and room for abuse in that one make it hopeless.

 

[Stimpson J. Cat]

Paul,

Why wouldn't the control subjects and test subjects be indistinguishable under the null hypothesis? Gee, now that I think about it, I'm not sure what the null or experimental hypotheses are.

That's exactly the problem. What Luci presented isn't a null-hypothesis at all. A null-hypothesis would be something like "John Edward is just doing cold reading". You could then attempt to test this null-hypothesis by getting a bunch of cold readers together, and having them each read a bunch of subjects, without the subjects knowing who was reading them. The subjects could then report the quality of the readings.

Unfortunately, even if all the possible problems with such an experiment were controlled for, there would be the problem that this experiment does not test the null-hypothesis. Even if John Edward gives better readings than all of the control set (cold readers), that still doesn't rule out the possibility that John Edward is just a better cold reader than they are.

In other words, even if the null-hypothesis is true, and John Edward is just using cold reading, that does not imply that we should not expect him to be better at cold reading that the control subjects.

An experiment using a control population can only test the null-hypothesis is the validity of the null-hypothesis implies that the test subject should be indistinguishable from the control set, but in this experiment, this is not at all implied. Such an experiment is thus pointless.

Next null-hypothesis Luci?

Dr. Stupid

 

[Lucianarchy]

Hypothesis:

There is no known scientific mechanism that can replicate the same results as Laurie Campbell, John Edward, Suzane Northrop, George Anderson and Anne Gehman, collectively known as 'research mediums' for the HESL, under the conditions and protocols of the research of Dr Schwartz.

 

[jj]

Hypothesis:

There is no known scientific mechanism that can replicate the same results as Laurie Campbell, John Edward, Suzane Northrop, George Anderson and Anne Gehman, collectively known as 'research mediums' for the HESL, under the conditions and protocols of the research of Dr Schwartz.

That's not a falsifiable hypothesis to start with, and putting in the "conditions... Dr. Schwartz" only makes it worse.

"no known scientific mechanism" is a negative (not null) hypothesis.

 

[BillHoyt]

That's not a falsifiable hypothesis to start with, and putting in the "conditions... Dr. Schwartz" only makes it worse.

"no known scientific mechanism" is a negative (not null) hypothesis.

jj,

You don't understand. All such science is defined by Schwartz' lab!

Cheers,

 

[Lucianarchy]

Hypothesis:

Known scientific mechanisms can replicate the same results as Laurie Campbell, John Edward, Suzane Northrop, George Anderson and Anne Gehman, collectively known as 'research mediums' for the HESL, under the conditions and protocols of the research of Dr Schwartz.

 

[Paul C. Anagnostopoulos]

Now you're getting somewhere, Luci. Next you must expand "same results as" to specifically list what those results are.

I was talking to my wife about this thread and she had a good chuckle. Then she reminded us to make sure we control for ESP and/or mediumship abilities on the part of the sitters.

~~ Paul

 

[jj]

Hypothesis:

Known scientific mechanisms can replicate the same results as Laurie Campbell, John Edward, Suzane Northrop, George Anderson and Anne Gehman, collectively known as 'research mediums' for the HESL, under the conditions and protocols of the research of Dr Schwartz.

*sigh* I do think you're trying hard. You need to work in the chance of statistical success or failure, I think, and while you can say something like "in this experiment", you have to realize that that limits the applicability of the results.

I'd have proposed something if I had more experience, HOWever, I'm far too able to see holes in this, the previous, etc...

You have to realize that you have to arrange for some way to actually test the possibility of "outside" (non-psi) information, etc, by having controls (which are not people, but are test conditions that have a known behavior) for both psi and non-psi hypotheses. It' s hard to do that for psi hypotheses, of course, so we could drop that... BUT... when we drop that, then you can't judge the sensitivity of your test.

That's always true of a "first" test, though, so it's not a terminal problem.

Somebody with a lot more experience as a stage magician or mundane mentalist ought to step up here with some advice, I KNOW I'm not experienced enough in the misdirection/sleigh of hand end of things to propose a set of test conditions, and until I know about that, I can't really propose a hypothesis without having some concern that it's flawed.

 

[Lucianarchy]

No, but thanks for the comments, that'll do just fine as a hypothesis. The results should be self evident.

 

[TLN]

Now you're getting somewhere, Luci. Next you must expand "same results as" to specifically list what those results are.

Whenever you're ready Luci. Oh, that right, we're not allowed to see Schwartz's data.

 

[Paul C. Anagnostopoulos]

Known scientific mechanisms can replicate the same results as Laurie Campbell, John Edward, Suzane Northrop, George Anderson and Anne Gehman, collectively known as 'research mediums' for the HESL, under the conditions and protocols of the research of Dr Schwartz.

Okay, so let's say we know what the "same results" are, and we know what the conditions and protocols are. So we pick some bozo off the street and have him use a "known scientific mechanism" for doing a reading, for example, cold reading (assuming that doesn't conflict with the conditions or protocols). What would it mean if he (a) got the same results; (b) didn't get the same results?

~~ Paul

 

[jj]

No, but thanks for the comments, that'll do just fine as a hypothesis. The results should be self evident.

You have to have a solid, agreed version of what constitutes "success", Luci.

Otherwise evaluators can do anything they want. It only takes a tiny BIT of skew to get either far positive or far negative relative to chance!

 

[BillHoyt]

You have to have a solid, agreed version of what constitutes "success", Luci.

Otherwise evaluators can do anything they want. It only takes a tiny BIT of skew to get either far positive or far negative relative to chance!

Implicit in this, jj, is the a priori definition of the success criteria.
And that these criteria must be objective.

Cheers,