I was referring to the judgments stating that the JREF's response was 'timid'.
Pear Cable CEO Calls James Randi's $1 Million Offer a Hoax
- Thread starter jhuntington
- Start date
-
- Tags
- pear cable
I was referring to the judgments stating that the JREF's response was 'timid'.
This is a perfect example of why designing protocols shouldn't be taken lightly, and not be left to people with no idea what they are doing.
When a protocol raises more questions than it answers, it is a worthless protocol.
E.g.:
- Is it 10 or 20?
- What will this mean for the statistical analysis?
- E.g., how many can merely guess in order to change the outcome of the test?
- How are the 10/20 people picked? "Random" - does that mean closing your eyes, open a telephone book and point your finger?
- Are there any CDs that are more suitable for a test than others?
- Why different machines, if it isn't the machines that are being tested? This alone invalidates the test.
- Why no baseline test? This alone invalidates the test.
- How long will a test take?
- Where, exactly, will the test take place?
- How will the results be recorded?
- What about the sign-off procedure?
- Who will be present?
- How will the test be documented?
- Will electronic equipment be used?
- If so, will that have any effect on the result?
Not a "piece of cake".
Yes, indeed. "He'll...sooner or later" is not future tense, but past tense. Always listen to the Grammar Stalin....
yairhol
Graduate Poster
- Joined
- Jan 16, 2007
- Messages
- 1,401
Finally The JREF is getting to be recognized by more people. It takes a scandal to be famous.
http://acapella.harmony-central.com/forums/showthread.php?t=1763185
http://www.gearslutz.com/board/so-m...end-cable-got-good-ears-make-million-usd.html
Regards,
Yair
http://acapella.harmony-central.com/forums/showthread.php?t=1763185
http://www.gearslutz.com/board/so-m...end-cable-got-good-ears-make-million-usd.html
Regards,
Yair
Last edited:
rjh01
Gentleman of leisure
One other issue. Is there any law in USA that says you must be able to prove your advertising claims? I know in Australia companies have been in trouble for such things. If there are such laws Pear Cable company could be prosecuted.
As to who is harmed, anyone who buys such cables. They will be losing $,000. Gaining nothing. Maybe not their life savings, but so what?
As to who is harmed, anyone who buys such cables. They will be losing $,000. Gaining nothing. Maybe not their life savings, but so what?
MRC_Hans
Penultimate Amazing
- Joined
- Aug 28, 2002
- Messages
- 24,961
You mean, based on a graph from some vaguely defined test set-up, with a non-descript reference, published on the vendor's website?
Which well-known scientific equipment? (the scope is just a voltage measurer).
Actually, you cannot infer that a higher voltage necessarily means more energy transmitted. The cable with the lower voltage output may achieve a better impedance match and thus a better energy transfer.
Remember, this is not DC.
It may be science, but the test-setup described in that page ain't.
Hans
1 - claimant sits in a chair in a room with speakers on the wall and a notepad and pencil on his lap, the pad having a column of number 1 - 100
2 - tester in other room (with independant observers to verify the marking procedure if neccessary) with music producing device and 2 sets of cables, one being the set claimed to be 'better' than the other. Both sets are covered in a manner that tester does not know which one is the 'better' set. Cables marked 'A' and 'B'. Tester uses a random method to select a set of cables to connect and once cables are connected, plays a piece of music for a pre-agreed number of seconds/minutes. Tester notes whether selection is 'A' or 'B'
3 - claimant makes a mark beside number 1 on his pad.
4 - tester repeats random choice of cables and playing music for another 99 times.
5 - tester makes either a 1 to signify the sound is no different from the previous sound, or a 0 to indicate that the sound is different from the previous. Repeats 99 times
6 - results are compared. Failure of the claimant to identify the difference between the 2 sets of cables will be verifiable. If claimant fails to identify between the 2 sets of cables 10 times out of 100, he fails the test. If he succeeds in discerning the difference 91 times out of 100, he passes.
Ok Claus, I take back my claim about writing this up in 2 mins. Took me about 6 and a half.
Last edited:
rjh01
Gentleman of leisure
The obvious flaw is what happens if the expensive cables are worse than the cheap cables? The person must say which ones are the better ones. So let us modify the above test. The bold writing are the changes.
1 - claimant sits in a chair in a room with speakers on the wall and a notepad and pencil on his lap, the pad having a column of number 1 - 100
2. - tester in other room (with independent observers to verify the marking procedure if necessary) with music producing device and 2 sets of cables, one being the set claimed to be 'better' than the other. Both sets are covered in a manner that tester does not know which one is the 'better' set. Cables marked 'A' and 'B'. Tester uses a random method to select a set of cables to connect and once cables are connected, plays a piece of music for a pre-agreed number of seconds/minutes. Tester notes whether selection is 'A' or 'B'. Tester then changes the cables and plays the same music again.
3 - claimant makes a mark beside number 1 on his pad to indicate which piece of music sounded better.
4 - tester repeats random choice of cables and playing music for another 99 times.
5 - reserved
6 - results are compared. If claimant fails to identify the more expensive cable 10 times or more out of 100, he fails the test. If he succeeds to identify the more expensive cable 91 times or more out of 100, he passes.
1 - claimant sits in a chair in a room with speakers on the wall and a notepad and pencil on his lap, the pad having a column of number 1 - 100
2. - tester in other room (with independent observers to verify the marking procedure if necessary) with music producing device and 2 sets of cables, one being the set claimed to be 'better' than the other. Both sets are covered in a manner that tester does not know which one is the 'better' set. Cables marked 'A' and 'B'. Tester uses a random method to select a set of cables to connect and once cables are connected, plays a piece of music for a pre-agreed number of seconds/minutes. Tester notes whether selection is 'A' or 'B'. Tester then changes the cables and plays the same music again.
3 - claimant makes a mark beside number 1 on his pad to indicate which piece of music sounded better.
4 - tester repeats random choice of cables and playing music for another 99 times.
5 - reserved
6 - results are compared. If claimant fails to identify the more expensive cable 10 times or more out of 100, he fails the test. If he succeeds to identify the more expensive cable 91 times or more out of 100, he passes.
rsaavedra
Muse
- Joined
- Oct 2, 2007
- Messages
- 650
Once again, making sure the tester won't know which cable is which isn't as trivial as it seems.
Just by looking at the terminations someone will tell which cables are which. Some wires are stranded, some other might be solid. Some might have less strands, or thicker ones. Some might be just copper, some are silver plated, some are silver. Some are ribbon shaped, some have banana plugs, some have spade plugs, and on and on. Even wrapping the cables up to the insulation so that only the conducting part is exposed, tester will still very likely be able to tell the cables apart, violating his part of the double blind test. And the tester needs to see the exposed part when connecting the cables, if he is to connect them properly.
Seeing the cables is not the only telling part. Once again, the weight, girth, and stiffness of the cables might also be easy to tell apart by the tester, unless the cables get wrapped with some fancy thick wrapping that somehow properly masks all of those features.
A cheating-proof DBT for a cable challenge will require a lot of atention to seemingly irrelevant details.
Last edited:
Of course. You had my list to work from. But, let's go through your protocol.
1. What music producing device should be used? Who will provide it? How will it be tested for trickery? What speakers should be used? How should the room be designed? It isn't the same to hear music in a room with concrete walls and a room with wallpaper.
Why a pencil? A pencil mark can be erased later and changed. Huge hole in the protocol!
Will it be recorded? Will recording influence the test? You need to establish this before the test begins, because we have seen before that electronic equipment is blamed for a bad result.
What about a baseline test?
2. Which random method will be used? Dice? Coin? Computer? Who will provide it? Who will operate it?
3. What kind of mark will claimant make beside number 1 on his pad? Should it be if he thinks it is A or B? Or if he thinks it is the better one or the worse one?
4. The claimant can claim that such frequent change of cables will damage them in the long run.
5. This is unclear. The tester can tell which is the better cable, yet in 2., he doesn't know? Also, it looks as if you will get a run of 1 with one set of cables, and 99 runs with another.
6. How did you decide that claimant has to pass 10 out of 100? You don't know what the claimant claims to be able to do.
Also, the wording is unclear. It should be "10 times or more", and "91 times or more".
All these unresolved issues aside, I can score 100% correct with this protocol.
You want to know how?
Indeed.
In reality, you need to shield off the cables so much, that the tester isn't even able to tell them apart.
No, it ain't a piece of cake, made in a couple of minutes.
MRC_Hans
Penultimate Amazing
- Joined
- Aug 28, 2002
- Messages
- 24,961
I fail to see why the tester must not know which is which. Since he is picking the cables from a random cue (dice or prng), he should have no chance of influencing the result. The timing must be fixed, however. That is, the interval allowed for changing the cables must be a fixed interval, so that the changing time does not offer any clues. Likewise, of course, the music must play for a fixed interval.
Hans
Hans
MRC_Hans
Penultimate Amazing
- Joined
- Aug 28, 2002
- Messages
- 24,961
The equipment should be one suggested by the vendor of the cables, and agreed on by the JREF, but supplied from an independent source. SInce the same equipment is used for both cables, it should not influence the result. Same with speakers, and room.
This may be a language problem. I think that for many, "pencil" denotes a generic writing device. However, you have a point: The pen used should be un-erasable.
Of course it will be recorded, that is stipulated in the cahllenge rules. Why should recording influence the result?
That might be a good idea. In such a test, the listener knows which cable he is listening to.
I would suggest computer. You can make an excel spreadsheet that can be validated, and will supply a random number. The quality of the randomness is not critical, as long as it does not show an obvious pattern. An observer can operate it.
That has to be agreed upon, of course. No big deal, though. I would suggest check-boxes instead of marks. That is way some of the possible ambiguities are avoided.
He can, but let us adress that when it comes. If the worst comes to the worst, he must supply 50 sets of cables (remember, he is the vendor).
The exact numbers must be up to the statisticians.
Yes, do tell.
Hans
Last edited:
rsaavedra
Muse
- Joined
- Oct 2, 2007
- Messages
- 650
Well, I was under the assumption we want a double blind test for this.
The protocol could be arranged so that the subjects have absolutely zero interaction with the assistant swapping the cables, and in that case, allowing that assistant to identify the cables wouldn't be a problem. Unless, of course, there is some form of assistant-subject pre-arranged cheating: the assistant manages to communicate with the subject (e.g. text message, or by some other means) indicating which cable is being tested each time.
Gravy
Downsitting Citizen
- Joined
- Mar 27, 2006
- Messages
- 17,078
News!
*Nor do I understand why it is. I thought the point was for the manufacturer to put their reputation on the line. If Mr. Fermer passes the test, good for him and Pear. But if he fails, Pear can just say, "Hey, he wasn't our choice and we can't vouch for his hearing ability."
I didn't realize that this was open to non-PEAR audiophiles.* I'll gladly eat half of my words when an application comes in, and the other half when a protocol is agreed on.
*Nor do I understand why it is. I thought the point was for the manufacturer to put their reputation on the line. If Mr. Fermer passes the test, good for him and Pear. But if he fails, Pear can just say, "Hey, he wasn't our choice and we can't vouch for his hearing ability."
Last edited:
As long as a baseline test is being done, and the claimant clearly states that he can tell the difference when he knows which is which, then this is fine.
Not only that: Each guess must be recorded outside the claimant's room, and later compared to the claimant's own list.
Because the goblins got mad at the extra electronics. Because the ley lines were interrupted. Because there was interference.
It is not just a good idea, it is vital: It establishes that the claimant can perform without the controls.
I would not suggest a computer. You can write all sorts of stealth programs that will override any results that the computer will under normal circumstances produce.
It has to be binary. Either he can decide which is which, or he can't.
Another thing that hasn't been addressed in the protocol: What about the incidents where the claimant can't decide which is which? What happens to those data?
Yeah, which is why the vendor can't be the claimant doing the testing.
Think he can't make a special cable that will alter the sound in a way so only he knows what to listen for?
Absolutely. But, of course, the more, the better.
You gave the answer yourself:
I, in collusion with one or more of the controllers, can simply set up a system where our watches are synchronized. The controllers will stall or hasten when the sample is being played, depending on which cable it is.
Point is, this wasn't in the protocol. It's not a 6 minute walk in the park.
Oh yeah.
Ever changed speaker cables?
Yep: It's a good idea to turn off the system first, because you can often hear the cables being switched through the speakers. If you're unlucky, you might even damage your speakers, if you get too much of a "scratchcracknoise".
All the assistant needs to do when changing cables, is tap once for cable A, twice for cable B. 100% success rate.
No, not a walk in the park.
rsaavedra
Muse
- Joined
- Oct 2, 2007
- Messages
- 650
Certainly, that's why I said quite a while ago, won't be two minutes trivial to establish a satisfactory test for all parties involved.
One suggestion indicated subject and speakers in one room, assistant and cables in another. Well, that won't work just like that. You have to connect the cables to the speaker posts eventually. So at least there should be access from one room the other behind the speakers, and the assistant will have to eventually hold a cable with his hands to connect it to the speakers through that access pathway, giving quite some chances for indicating somehow (noise clicks or what not) what cable is being tested.
Just a simple black curtain behind the speakers might do, then noise control will be of paramount relevance when swapping (or pretending to be swapping cables) to avoid cheating.
Once again, not out-of-this-world difficult, but not trivial, not trivial. Plenty of details to take care and watch out for.
Last edited:
rsaavedra
Muse
- Joined
- Oct 2, 2007
- Messages
- 650
By the way, the room will be very important. It should be an acoustically "healthy" room. Too many early reflections and audiophiles will claim detail and stereo image are deteriorated by the room acoustic issues. Too wild bass cancellations in the room and you will deteriorate the perceived bass frequency response, snappiness, accuracy, and what not. This is all regardless of the cable and/or speaker and or system's frequency response. Inappropriate speaker-listener placement and you won't have a proper stereo image and width-height of "soundstage." In some cases a different room causes larger sound differences than different speakers.
In fact, it is possibly that some audiophiles are truly able to differenciate cables and components in the listening setup they are most accustomed to (their home), and possibly not having the same capabilities in an acoustically different setup. That's why a baseline must be established at the listening room, for instance allowing the subjects to first hear the system knowing what cables are being used, and allowing them to claim that they can certainly tell the cables apart there. Only after that the DBT would begin.
I'm sure some acoustics engineers will make sure the listening setup will be satisfactory, especially with $1 million at stake. But just bringing to the table more issues involved in this testing.
In fact, it is possibly that some audiophiles are truly able to differenciate cables and components in the listening setup they are most accustomed to (their home), and possibly not having the same capabilities in an acoustically different setup. That's why a baseline must be established at the listening room, for instance allowing the subjects to first hear the system knowing what cables are being used, and allowing them to claim that they can certainly tell the cables apart there. Only after that the DBT would begin.
I'm sure some acoustics engineers will make sure the listening setup will be satisfactory, especially with $1 million at stake. But just bringing to the table more issues involved in this testing.
Last edited:
MRC_Hans
Penultimate Amazing
- Joined
- Aug 28, 2002
- Messages
- 24,961
There should be no reason for double blinding. As long as the listener is effectively blinded to which cable is used, that is sufficient. Double blinding is for removing reporting bias, but since the tester cannot improve skew the result by false reporting (and the recording would disclose any attempts of this), there is no reason to blind him.
Hans
alfaniner
Penultimate Amazing
Well, considering the quickly proposed (and thrown out) protocol for the GSIC chip, I think the speculation and concern is warranted.
Where is Wellfed in all this anyway? You'd just have to make sure the test is not on a weekend...
The double-blinding is important, as it must be made impossible for either side to cheat. Perhaps some skeptics don't realize that any appearance or possibility of impropriety on the JREF side must be prevented also. In other words, assume that every person involved in the test wants to cheat -- preventing this is the only way to have a fair and non-judgemental test.