Question about statistical significance

[Rolfe]

Don't take this as the final word, because I'm not that well up on the subject. But it's a sad fact that pretty much all pharmacies carry a stock of OTC homoeopathic remedies, with a little laminated book to look up and see what you should be taking. Most of these preparations probably have something in them, as they're not that "potent", but you could probably count the molecules. Of course, we're constantly told by the real woo-woos that this isn't really homoeopathy, and that you have to pay them for a 1-hour individual consult to see what the right remedy for what ails YOU is, but heck, the pharmacies seem to like to stock the stuff.

The pharmacy where I live only has a small display tucked away at ground level (well, it's not as if there's any safety issue...), and they laugh nervously if I raise the topic, but it's there.

Now I don't know about them actually making up and dispensing the high-potency (high-dilution) stuff the woo-woos prescribe individually, but I think it is occasionally done. I should ask the pharmacists what the score is. I suspect it's theoretically within their remit, and some may do it if they have a lazy homoeopath nearby, but I suspect most wouldn't keep the equipment these days.

The homoeopathic hospitals have "pharmacies", of course. And some of the more historical homoeopathic literature implies that it was once relatively common to have remedies made up in an ordinary pharmacy - in the days when a lot of medicines were prepared like this. Also have a look at this page here - this seems to be a relatively recent account of a pharmacy being involved, with perhaps unexpected consequences. Not sure which country that was in, though.

There was never any classical requirement that the person making up the remedy shouldn't be a sceptic (so long as they actually did the procedure of course!), but now that we're seeing so much quantum entanglement being preached suggesting that the scam works because of an effect of the intent of the practitioner, I do wonder just how long it will be before we're told that the bench-work has to be done by a true believer.

Rolfe.

 

[daver]

Will he be preparing the homeopathic solution?
If so, he might put a trace, detectable chemical inside without your knowing about it. Are you willing to watch him make the stuff?

You could compensate for a bit of this stuff. He's talking about a C30 mix, right? At C20, have him give a ref his mixture and a sample of his diluting water; the ref will pour his mix into two bottles (X1 and X2), the diluting mix into two others (Y1 and Y2) (X1, X2, Y1, and Y2 are random labels). X1 and Y1 are given to a lab to analyze, X2 and Y2 are returned to the homeopath. If the lab can determine gross differences between X1 and Y1, the homeopath presumably cheated, and put some marker in either the "medicine" or the diluting fluid. Regardless, the homeopath then goes on to prepare his concoction. He doesn't know which of X2 and Y2 are the real "medicine", but presumably his efforts will produce a curative in one case and well shaken water in the other.

 

[Rolfe]

You could compensate for a bit of this stuff. ..(snip).. He doesn't know which of X2 and Y2 are the real "medicine", but presumably his efforts will produce a curative in one case and well shaken water in the other.

I don't think it's worth it. I run a lab, and I couldn't swear to find a totally unknown "something" in any sample. I think the JREF will agree that everything has to be done to ensure no tampering.

I'm pasting below the current draft of the letter I'm preparing to send to the journal putting forward my suggested method. Many thanks to T'ai Chi especially (never thought I'd be saying that, but I mean it) for the statistical assistance. Any constructive comments welcome. (I'm assuming the applicant can only test one shot at a time, i.e. he only has one team of provers - of course if he could do 20 trials simultaneously it could be very quick, but I don't think that's on - after all, he won't want to share the million bucks with any more people than he can help! )

No doubt he'll have some excuses (maybe the stuff doesn't store, maybe he won't want it sent through the post which would be my preferred means of dispensing, maybe all those sceptics watching will stop the magic working), but there should be ways round them. We'll see.

Dear Editor,

I'm immensely flattered that Mr. Hoare would "love" to have my help in designing a protocol to test his claims regarding homoeopathic provings. However, every time I think I've got my brain round these peculiar assertions, he informs me that no, I just don't understand the subject.

In fact, I'm only concerned to know whether he can indeed distinguish a 30C homoeopathic preparation of his own choice from a sham. How he does it would undoubtedly be interesting, but I certainly wouldn't want to restrict his protocol in any way.

So, how about this?

1. Mr. Hoare selects his 30C remedy.
2. The real remedy and the sham are prepared.
This is the most difficult part of the whole exercise. Does Mr. Hoare want to prepare his own materials, or have this done by a member of his staff? Or would he agree to a third party (perhaps a university pharmacologist?) doing the preparation? However it's done, this stage will have to be scrutinised in minute detail so that all parties are comfortable that there has been no opportunity for any jiggery-pokery.
3. The preparations are turned over to an honest, unbiassed third party (the dispenser) - again perhaps a university colleague would oblige. This person assigns heads or tails to the remedy and the sham, and keeps this assignment constant throughout the exercise.
4. The dispenser tosses a coin, and dispenses whichever the result indicates, remedy or sham, in as large a quantity as Mr. Hoare feels he needs to ascertain which he's been given. Needless to say, the stock preparations and the identity of what has been dispensed must be closely guarded. Contact between the dispenser and Mr. Hoare should be the minimum necessary.
5. Mr. Hoare does whatever he likes to ascertain whether he's been given the active remedy or the sham, taking as much time as he needs.
6. He reports his decision, perhaps to a completely separate third party, who merely makes a secret note of the decision and informs the dispenser that the next trial can now go ahead.
7. After waiting however long Mr. Hoare feels is necessary, return to step 3, toss the coin again and repeat the exercise.

It would be wise to take independent statistical advice on this, but I imagine that 18 correct out of 20 trials (p<0.001) would impress pretty much anybody.

This might be quite a prolonged exercise, depending on how long it takes Mr. Hoare to make up his mind each time, but in a sense that doesn't matter. So long as the preparation, storage and dispensing of the materials are carried out with scrupulous honesty and the record of what has been dispensed each time is completely secret, there's no need at all to supervise the actual testing. I imagine Mr. Hoare intends to set up some sort of "proving", using either himself (surely even one person can tell whether he's been given a random 10% of the pieces of a jigsaw of the Mona Lisa or of a blank canvas, to return to his own analogy), or as large a group of volunteers as he requires. The people involved can confer all they like, consult whatever literature or expert sources they like, and take as long as they like. Nothing is forbidden - NMR spectroscopy, or X-ray diffraction, or dowsing; ask NASA, or Porton Down, or Russell Grant; even give it to selected patients if that is deemed appropriate. What could be fairer than that?

Of course, a trial along these lines would certainly be acceptable as a protocol for a preliminary test for the JREF Million Dollar Prize. I see no point whatsoever in going to all this trouble unless it is an official attempt at the prize. I would therefore urge Mr. Hoare to look at www.randi.org/research/index.html, read through the rules at www.randi.org/research/challenge.html (these look perfectly fair to me), and get his application in. The JREF will then provide advice on the final details of the protocol, including how to ensure that the critical steps are adequately secure, and what level of performance should be accepted as a statistically valid demonstration of the claimed ability.

I think this is a simple and fair protocol to test Mr. Hoare's claim. I would be very happy to help him refine it further to meet the requirements of the JREF, to approach academic colleagues who might be mutually acceptable as unbiassed keepers of the preparation, dispensing and recording steps, and even to be one of his proving group if he really wants me - though if I were him, I'd stick to experienced homoeopaths who understand what's going on and know what they're looking for. I look forward to hearing from him.

Yours faithfully,

Rolfe.

 

[T'ai Chi]

Many thanks to T'ai Chi especially (never thought I'd be saying that, but I mean it) for the statistical assistance.

You're welcome.

Don't let Hoyt know about the assistance- he might explode.