[watchnerd]

  There is for sure a chance that measurements show some kinds of odd behavior but the equipment sound subjectively good.
But in general there can be no doubt that advertising and reviews go hand in hand. There isn't a single product review in the you know what hifi magazine (in any country) of a companie's product and the related full ad page of that manufacturer is either before the review or directly behind. Sometime next issue announcements are followed by back page high gloss ads. So much for objective reviews

 
This is why I adore "No Audiophile" website.  All the gear is either bought by him or lent to him by friends / readers.  He pulls speakers apart, measures them, does crazy DSP experiments. Plus it's hilarious.  His RMAF coverage alone is worth the read.

 

[mmerrill99]

 
they have that kind of unique situation where measurements and subjective opinions can mix and oppose each other. if anything, compared to most other guys, I would say that at least they have some measurements and I greatly appreciate that.

now when the graphs show a problem and the feedback is "it sounds amazing", I have to say that I'm always wondering if it's one of those cases where distortions sound nice, or if it's marketing and mutual interests that are typing on the keyboard at that moment. 

There is another possible explanation, that even what looks like impressive distortion and noise and FR deviations are *sometimes* still below our detection ability in normal listening and that the supposed huge differences between components can sometimes be attributed to the nature of uncontrolled tests where little or no effort has been expended to remove the array of biases and that even without the biases spawned by foreknowledge of what the reviewer is listening to we have a test method so flawed that any number of imagined differences and/or audio characteristics are discovered.

Be that as it may removing the knowledge of what they are listening to would seem to be an utterly logical first step. The argument that these expert listeners have such superior hearing and judgment that they can remember in clear and total detail what something sounded like three weeks ago in a different room with different speakers and different tracks at different volume levels and thus make 100% judgments off differences between A and B and are not in any way (conscious or not) swayed by appearance or price stretches credulity.

The late Tom Nousaine did some interesting tests. he gave listeners a box that had a circuit in it that was either a transparent pass through or introduced 2.5% distortion. Long term listeners scored randomly (50/50) in their ability to determine if the box had distortion or not. Then the distortion was plugged into a signal using a ABX box listeners able to switch between two signals (distorted/undistorted) quickly performed far better

AFAIR, you left out one important aspect of Nousaine's test - prior to ABXing the testers were given training

 

[nick_charles]

AFAIR, you left out one important aspect of Nousaine's test - prior to ABXing the testers were given training

 
 
My apologies I was conflating two studies. The box study was (Greenhill, L. L. and Clark, D. L., "Equipment Profile", Audio, (April 1985))  (2.5% distortion) and the members of two audiophile clubs (no sample size given). The long term test was as long as the subject wanted and the short term switching test was indeed preceded by 45 minutes training. But now subjects could reliably detect distortion levels down to 0.4%
 
The Nousaine follow up was CDs with distorted or undistorted signals (but now 4% distortion)  the result for the long term test was over a period of 13 weeks with 16 audiophile subjects (most subjects kept the CD for between 1 and 4 weeks - average of 2.7 weeks apart from the two who kept the CD for all 13 weeks) the 16 subjects scoring 10/16 overall with no correlation between length of CD loan and accuracy.

 

[mmerrill99]

AFAIR, you left out one important aspect of Nousaine's test - prior to ABXing the testers were given training

My apologies I was conflating two studies. The box study was (Greenhill, L. L. and Clark, D. L., "Equipment Profile", Audio, (April 1985))  (2.5% distortion) and the members of two audiophile clubs (no sample size given). The long term test was as long as the subject wanted and the short term switching test was indeed preceded by 45 minutes training. But now subjects could reliably detect distortion levels down to 0.4%

So it wasn't Nousaine, OK

The Nousaine follow up was CDs with distorted or undistorted signals (but now 4% distortion)  the result for the long term test was over a period of 13 weeks with 16 audiophile subjects (most subjects kept the CD for between 1 and 4 weeks - average of 2.7 weeks apart from the two who kept the CD for all 13 weeks) the 16 subjects scoring 10/16 overall with no correlation between length of CD loan and accuracy.

The way I saw this being reported was that only one CD was sent to each listener, not both CDs for their long-term comparison. So, what if this is correct, would that prove?

"Experiment #2 was conducted by Tom Nousaine in 1996. He prepared two sets of
CD-Rs. One set of CD-Rs was a bit-for-bit copy of a commercially released
song. The second set added 4% harmonic distortion to the song. He mailed the
disks to 16 audiophiles and asked them whether they had received a clean
disk or a distorted disk. Again, results were null. He then administered an
ABX test to one of the subjects who had gotten it wrong. Using a looped
6-second extract of the song, this subject was able to score perfectly."

 

[nick_charles]

So it wasn't Nousaine, OK
The way I saw this being reported was that only one CD was sent to each listener, not both CDs for their long-term comparison. So, what if this is correct, would that prove?

"Experiment #2 was conducted by Tom Nousaine in 1996. He prepared two sets of
CD-Rs. One set of CD-Rs was a bit-for-bit copy of a commercially released
song. The second set added 4% harmonic distortion to the song. He mailed the
disks to 16 audiophiles and asked them whether they had received a clean
disk or a distorted disk. Again, results were null. He then administered an
ABX test to one of the subjects who had gotten it wrong. Using a looped
6-second extract of the song, this subject was able to score perfectly."

 
It would suggest that long term listening is not particularly good for detecting quite high levels of distortion on that specific track on the CD medium across a variety of listeners on their systems - we have no info on the nature of their playback systems***meaningful that is ***
 
 

 
 
I did not bother reporting the Nousaine one subject follow up (I dug out the paper)  as one subject proves very little, he should have used a bigger sample

 

[RRod]

AFAIR, you left out one important aspect of Nousaine's test - prior to ABXing the testers were given training

 
For those of us without the paper, what was the training?

 

[mmerrill99]

So it wasn't Nousaine, OK

The way I saw this being reported was that only one CD was sent to each listener, not both CDs for their long-term comparison. So, what if this is correct, would that prove?


"Experiment #2 was conducted by Tom Nousaine in 1996. He prepared two sets of

CD-Rs. One set of CD-Rs was a bit-for-bit copy of a commercially released

song. The second set added 4% harmonic distortion to the song. He mailed the

disks to 16 audiophiles and asked them whether they had received a clean

disk or a distorted disk. Again, results were null. He then administered an

ABX test to one of the subjects who had gotten it wrong. Using a looped

6-second extract of the song, this subject was able to score perfectly."

Well, first point to make is that your first report of this test led one to believe that they could tell the difference using long-term listening between two CDs that they had been given, one with 4% distortion on it. This is very different to what actually happened in the test, it would seem. So the normal procedure suggested for long term listening is by comparing two devices over a period of time, not listening to something & deciding in isolation if this is distorted - I think there was an example given on a video recently of 4% clipping distortion that isn't so audibly noticeable (I think about 10% was where it became audible) whereas a much lower level of crossover distortion was audible. The reason given for this was that real music spends more time in crossover area this area than in the clipping domain.

It would suggest that long term listening is not particularly good for detecting quite high levels of distortion on that specific track on the CD medium across a variety of listeners on their systems - we have no info on the nature of their playback systems***meaningful that is ***






I did not bother reporting the Nousaine one subject follow up (I dug out the paper)  as one subject proves very little, he should have used a bigger sample

The second point to make/ask is - do we know what distortion was added? Some distortions at 4% won't disturb our normal experience of what we are listening to as I said above

 

[nick_charles]

   
For those of us without the paper, what was the training?

 
 
For the first study subjects were exposed to 13% distortion which is pretty bad , that seems to be all. Then in the test proper they had to detect distortion of 4%, then it was lowered to 2% and then down to 0.4% and these levels were reliably detected by all subjects 

 

[RRod]

   
 
For the first study subjects were exposed to 13% distortion which is pretty bad , that seems to be all. Then in the test proper they had to detect distortion of 4%, then it was lowered to 2% and then down to 0.4% and these levels were reliably detected by all subjects 

 
So start with a really pathological case, then tone it down. Seems like exactly what many of us have done with other audio parameters in various personal ABX tests.

 

[nick_charles]

Well, first point to make is that your first report of this test led one to believe that they could tell the difference using long-term listening between two CDs that they had been given, one with 4% distortion on it. This is very different to what actually happened in the test, it would seem. So the normal procedure suggested for long term listening is by comparing two devices over a period of time, not listening to something & deciding in isolation if this is distorted - I think there was an example given on a video recently of 4% clipping distortion that isn't so audibly noticeable (I think about 10% was where it became audible) whereas a much lower level of crossover distortion was audible. The reason given for this was that real music spends more time in crossover area this area than in the clipping domain.
The second point to make/ask is - do we know what distortion was added? Some distortions at 4% won't disturb our normal experience of what we are listening to as I said above

 
I did not imply that, that is your inference. I did not mention CDs till my second post after I went back to the paper. Initially I only mentioned the infamous box which I clearly stated **either** had distortion or not and when I mentioned the CD test I also said that the listeners got a CD not two CDs you misinterpreted this.
 
Harmonic distortion on both studies. 

 

[mmerrill99]

I feel lots of generalisations & simplifications being made above
- This video is worth looking at - about 25 mins in some audience tests of audibility of various distortions

 

[mmerrill99]

Well, first point to make is that your first report of this test led one to believe that they could tell the difference using long-term listening between two CDs that they had been given, one with 4% distortion on it. This is very different to what actually happened in the test, it would seem. So the normal procedure suggested for long term listening is by comparing two devices over a period of time, not listening to something

I did not imply that, that is your inference. I did not mention CDs till my second post after I went back to the paper. Initially I only mentioned the infamous box which I clearly stated **either** had distortion or not and when I mentioned the CD test I also said that the listeners got a CD not two CDs you misinterpreted this.

Harmonic distortion on both studies. 

Yes, I know you incorrectly stated the first test was from Nousaine but when you went back to the paper & reported the follow-up Nousaine test it was easy to miss the fact that only one CD was given to each listener so they hadn't the 2nd CD for comparison in their long term listening. In their ABX testing they have the 2nd CD to compare. I would call this a lot less of a test than it's purported to be

 

[nick_charles]

   
So start with a really pathological case, then tone it down. Seems like exactly what many of us have done with other audio parameters in various personal ABX tests.

 
Yep, I did the same with some lowpass filter tests. I was trying to emulate the pre-CD JVC studies which had very high range speakers and Synthesizer noodling (above 30 KHz)  and progressively lower cutoffs, in their DBTs they found that none of their 32 subjects (audio engineers, producers and musicians) could reliably detect a cutoff at 16K , many could not detect a 14K lowpass, - they concluded that a cutoff at 20K was probably good to go...with my hearing it was about 13k - 14k (less now I guess, ho hum)

 

[nick_charles]

Yes, I know you incorrectly stated the first test was from Nousaine

and I did apologize for this, I'm not going to apologize again  

 

but when you went back to the paper & reported the follow-up Nousaine test it was easy to miss the fact that only one CD was given to each listener so they hadn't the 2nd CD for comparison in their long term listening. In their ABX testing they have the 2nd CD to compare. I would call this a lot less of a test than it's purported to be

 
It is (was) what it was - make of it what you like.
 
 

   
The Nousaine follow up was CDs with distorted or undistorted signals (but now 4% distortion)  the result for the long term test was over a period of 13 weeks with 16 audiophile subjects (most subjects kept the CD for between 1 and 4 weeks - average of 2.7 weeks apart from the two who kept the CD for all 13 weeks) the 16 subjects scoring 10/16 overall with no correlation between length of CD loan and accuracy.

 

[nick_charles]

I feel lots of generalisations & simplifications being made above
- This video is worth looking at - about 25 mins in some audience tests of audibility of various distortions

 
 
For a presentation by an engineer it does not look promising. First he announces his audiophile credentials (I'm one of you) and then makes a set of assertions "good sounding kit can have crappy specs and vice versa we all know this" I paraphrase, but this is standard audiophile cant and this does not fill me with great confidence, nevertheless I will persevere... tomorrow, I'm gonna finish listening to Lohengrin and watch the snow  stay safe and warm everyone !