1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.

    Dismiss Notice

Testing audiophile claims and myths

Discussion in 'Sound Science' started by prog rock man, May 3, 2010.
837 838 839 840 841 842 843 844 845 846
848 849 850 851 852 853 854 855 856 857
  1. KeithEmo
    You and me both.

    Let's say I ask someone a more or less simple question...
    And they answer in some foreign language or using complicated technical jargon...
    Then, when I ask them to please repeat their answer in plain English, their reply is "You're being lazy; just Google it..."
    My natural reaction is to assume that either they're just plain rude or they really don't want to be helpful.
    (At which point my natural reaction is to go look for someone who actually wants to be helpful.)

    It's also worth noting that, believe it or not, information you get on Google is quite often wrong.
    There was a recent article in The Wall Street Journal about Google (incorrectly) saying that various people were deceased when they were Googled by relatives and business contacts.
    Therefore, in many cases, saying "just Google it" is tantamount to saying "just ask someone on the street and hope they have the right answer".
    (And, yes, in something simple like the meaning of an acronym, you have at least a decent chance of getting the correct answer on Google.)
    However, the idea that "Google is a reliable source of accurate information" is still a major overstatement.

    However, it is worth pointing out that a lot of the people on forums like this one are here in the first place because they're hoping for better information than what they would get "by just Googling it".
    They're looking for a simple answer... and not a lesson in someone else's idea of proper Internet etiquette... or a lesson in how to conduct research in a modern world.
    (And, to be quite blunt, I'm inclined to believe that people in that position are entitled to the courtesy of a direct answer... in English... if that's what they're hoping for... and it isn't too much trouble to provide one.)

    Last edited: Jun 6, 2019
    TheSonicTruth likes this.
  2. KeithEmo
    I would go quite a bit further there...

    This idea also points out a serious MISUNDERSTANDING that many people have about the meaning of statistics and probability.

    If you graph a truly random distribution of something it will often produce a bell curve with a characteristic shape.
    HOWEVER that does NOT mean that the only way to end up with that shape is to start with a random distribution.
    It could be the result of a random distribution...
    Or it could be the result of something else entirely...

    If you fire a shotgun at a target the result will be a 2d version of a bell curve... with the most hits towards the center.
    However, it's also possible for an artist to produce the exact same pattern by carefully drawing in the dots one at a time.

    See if you can find the time results of some local marathon type race.
    I'll bet the finish times follow some sort of bell type curve...
    Do you think that means: "Everyone runs at about the same speed and it's just a random distribution"?
    Or do you think it might mean that some people really DO run faster than others?
    And, "at the next level", do you think it means that we humans just have a random variation in how fast we can run?
    Or could there be some other non-random factor - like, perhaps, people who practice become able to run faster, or run fast longer?

    To put that another way.....
    Show a bunch of grade schoolers a $1000 department store violin and a $2 million Stradavarius sitting next to each other in a display case...
    I'll bet that MOST of them can't tell which is which...
    And, if you ask them to pick the "better violin", the results will turn out to be "more or less random"....
    But a few probably will recognize the Strad (maybe)...
    Would you say that proves that "there's not really a significant difference"....?
    Or would you say it proves that "only a few grade schoolers know much about expensive musical instruments"....?

    How you interpret statistical results depends quite heavily on context.

    All you can say about the shape of a statistical response curve is that it "appears to fit the characteristic you would expect from a random bell curve".
    However, you CANNOT say with certainty whether it is actually the result of a random distribution... or not.

    Want to have a try at a better guess...
    Run ten more races...
    And see if a different 5% of the runners come in first in each one...
    Or if it's always the SAME 5% who come in first...
    (Then you can calculate the odds of the same 5% "randomly finishing first in ten races".)

    Or do a dozen more studies about whether the differences between those two amplifiers are audible or not.
    And see if the same one always wins by the same 5% - or whether the OVERALL distribution of the results is random.
    (And, if the result is the first one, you've just proven that the difference is almost surely actually real, but only 5% of the participants can hear it.)
    (And, if the result is the second one, then your claim that "it probably really is just random" has become somewhat more likely to be correct.)

    I would also point out that, if you read a lot of recent Stereophile equipment ratings...
    You'll quite often find Atkinson expressing doubt when equipment that measures poorly is rated to "sound good" in the "subjective review"...

    There's also one more thing worth pointing out....

    I always find it quite entertaining when results show that "audiophiles often score worse than skeptics"... because it tells us a lot... about human psychology... and specifically the phschology of audiophiles However, while entertaining, as a metric concerning test accuracy, that is quite meaningless. I suspect that "the fastest 100 human beings on Earth" probably have a pretty good idea who they are - based on actual race results. However, I have no reason whatsoever to suspect that "the 100 most dedicated audiophiles", or "the 100 people with the most expensive audio systems", are "the 100 humans with the most accurate hearing". I'm pretty sure nobody has actually ever tested that. In fact, I'm not even sure whether "100 highly rated professional musicians" actually have better than average hearing, or simply benefit from a lot more training and practice, or where that balance lies either. I'd like to think that a professional musician is more capable of recognizing a note that's off key than I am... because it's essential to his or her job performance... but I don't really know whether that translates to being able to recognize a lower level of THD or IMD than I can or not.

    Last edited: Jun 6, 2019
  3. bigshot
    ...the evidence would seem to suggest that that distinctive amplifier sounds, if they exist at all, are so minute that they form a poor basis for choosing one amplifier over another. Certainly there are differences between amps, but we are unlikely to hear them. -Stereo Review January 1987

    Nice quote.
  4. KeithEmo
    I agree... they phrased their conclusions in a reasonable manner... which is refreshing.

    Rather than attempt to make a blanket claim that "differences don't exist"...
    It properly characterizes the differences that exist as mostly minor - followed by an opinion that they may not be significant, or even audible, to any given individual listener.
    I'm personally inclined to agree with their opinion.
    Obviously many audiophiles are quite convinced that any difference that is audible, no matter how small, is a matter of life and death... which is an opinion that they are also entitled to.
    (And it is that opinion that keeps a significant portion of the audio equipment community in business - so it's sort of obvious why so many companies are inclined to encourage it.)

  5. bigshot
    Amps don’t sound different unless there’s something wrong with them. Audibly transparent is as good as any human being needs. Beyond that is overkill.
  6. analogsurviver
    The mantra that all the amps sound the same or at least differ so little to be safely ignored has to stop.

    Stereo Review had a very manufacturer pleasing policy whenever making any references to the listening tests. As good as the objective measurements from Hirsch - Houck Lab were, they were not really comprehensive, nor did focus on the specific problems - which they even admitted sometimes have been commented upon by the listening test panel.

    I did go trough most phono cartridge reviews in Stereo Review - since there were cases no other objective measurements have been available even back in the day - let alone surviving online today.
    And have heard or even own quite a few of them that Stereo Review claimed to reach such a level of quality for the differences to be essentialy meaningless, if not downright inaudible.

    NOTHING could be farther from the truth.

    Let's face reality : audio journalists and reviewers LIVE FROM their work; and that goes for both objective and subjective camp.

    They can't afford to say Component X from 4 decades ago has been the best - everything else available today regardless of price is a pale approximation at best. They can't say that even for one month ago... - if they want to stay in the bussiness.

    Subjective camp tends to over exaggerate the audible differences, the objective usually does exactly the opposite - NEITHER of which is good.

    Objective camp has it , at the same time, both infinitely easier and more difficult task - standard measurements usually are not capable of revealing the differences reported by discerning listeners, and the willingness to devise better measurements that would correlate with listening better is not there or costs too much to implemented.

    Subjective camp has an even harder task - as finer points of reproduction are sometimes consciously limited ( MP3 being a good example ) and, to compound the problem further, usually requires to cater to both better dynamic range ( particularly in soft part of the range ) and frequency extension well past 20 kHz. That requires recordings to be made with at least similar high calibre of the recording equipment - anything that went trough normal equipment in pro studios is inherently of lower resolution than required.

    I try, best I can, to learn positive ( as in bringing the reproduced sound closer to live ) from both camps. And experience clearly shows there were, are and will be - from unwilling errors to purposedly spread misinformation - from both sides.

    Stereo Review bore the banner of "everything sounds the same", "that is exactly how a quality X is supposed to work/sound and is indistiguishable from its peers in similar price bracket", etc, etc. Even if and when their own objective measurements did hint on a problem, they tried to downplay it in the listening comments as much as possible.

    The Absolute Sound did - on purpose - postpone a review of an audio component long enough for the company making it announced its discontinuation. For a VERY real fear ... that of the report of a component at a fraction of the competitors' price running circles around its pricier competition reaching the actual consumer. The objective camp did cover the said component - in the usual "nothing to complain about, everything ship shape" fashion - failing, as by default, to recognize the exceptional performance - not only at the price, but in absolute terms.

    NEITHER of the camps would publish anything of substance on exceptional overachievers - because it is bad for the bussiness. Here, they could not have been more in bed with each other... - no matter how on opposite banks they appear to be otherwise. Deafening silence.

    Objective audio press has a hidden "review entrance fee" - advertising. Open the small print in any magazine to find out how much advertising space costs. No advertising - no review. Plain and simple.

    The days subjective audio press has been reporting serial numbers of the components tested are largely gone - and review samples are offered to reviewers at trade cost - if not "better". Plain and simple.

    Draw your own conclusions from the above.
  7. KeithEmo
    I absolutely agree....

    If two amplifiers were absolutely perfect then they would sound exactly the same.
    And anyone would be foolish to choose one of anything over another, with the expectation that it would sound better, if both sounded "audibly identical" to them.
    (I consider these both to be "logical identities" more or less equivalent to "1=1".)

    The obverse is also clearly true...
    If two amplifiers sound audibly different then at least one, if not both, must not be perfect.
    (This negates the statement above. If both were perfect then they would sound exactly the same.)

    However, since nobody has so far produced a perfect amplifier, we're still stuck with the experience of the listener about "what is perfect enough".
    In other words, we cannot measure "audibly transparent"..... it is a statement of experience.
    "Audibly transparent" is simply a fancy way of stating that some particular listener doesn't hear any difference between two things.
    I'm pretty sure I haven't seen a graph anywhere on my AP analyzer with a mark labelled "audibly transparent" or even "inaudible" on it.

    I think that would be really great.
    If we had that, I could simply send all of our customers a nice report, with "Audibly Perfect = PASS" circled and highlighted...
    And we could stop with all those other annoying measurements...
    And nobody would have to wonder if any other product, at any price, could even possibly be "better".

    Now, as for overkill, I agree there too.
    However, I'm also pretty sure my car can go a bit faster than I've ever driven it.
    But I don't necessarily see that ability as a problem... per-se... either.

  8. bigshot
    "However, since nobody has so far produced a perfect amplifier..."

    WHOOPS! Huge assumption there...

    How are you determining that an audibly transparent amplifier has never been produced? (and you guys accuse me of making absolute statements!!) Yes, thresholds of perception are documented. Everyone is a little different, but that difference is the degree of hearing damage due to wear and tear, not superman hearing. Hearing is finite. Beyond a certain point, you are going to have to test dogs and bats, not people.

    Just look at the specs. On paper they are audibly transparent. Look at the listening tests. Yup! They're transparent! Compare a Futterman as big as a washing machine to a $200 Pioneer receiver... uh... they sound the same. What makes you think there is no such thing as an audibly transparent amplifier? Are you assuming that all amps are flawed in the exact same way? That is just silly.

    You guys are grasping at straws now. You either are incapable of thinking in a straight line, or you are being deliberately absurd just to get attention. Or maybe you just don't like this thread because it puts the lie to things that make snake oil salesmen a lot of money.
    Last edited: Jun 6, 2019
  9. TheSonicTruth
    He's got a point, RE 'perfect' amplifiers.

    I do know someone who could have built a perfect amp, but He lived over 2,000 years ago. He made ends meet in the woodworking industry.
  10. bigshot
    Audibly transparent is perfect to human ears. You can improve specs but it doesn't make a lick of difference to how the amp sounds. The limiting factor is your ears, not the transistors. If you want some sort of theoretical perfection, OK. But that has nothing to do with how an amp sounds.

    Do all amps sound the same? This test indicates that they do. If someone can suggest an amp that does sound different, have at it. But show a controlled listening test to prove it. Or give our Sound Science brain trust a crack at it to verify the audible difference and figure out why.
    Last edited: Jun 7, 2019
  11. analogsurviver
    The limiting factor in amps are most usually not the active components, like transistors - but the passive. And the difference is perfectly audible. Trouble is, he whole chain has got to be up to certain standard - one can not hear the difference of say one single superior component in an otherwise setup of lower quality. So, unless using a really good DAC ( there is amplifier inside...), one can ABX various digital files on foobar ad nuseaum - and, quite correctly - determine there is no audible differences. Even if all the other amplifiers in the chain are "perfect".

    Reminds me of a woman, who has been constantly bitching about the neighbor's laundry being less than perfectly clean. And asking her husband to "finally say something to the other husband". For quite some time...

    The next day, she was astounded - the neighbor's laundry was finally - perfect. She asked her man : " What did you tell to her man ? "

    " Nothing. I cleaned our window..."
  12. gregorio
    1. Obviously not, because "some sort" of bell curve is obviously not the same as the precise bell curve indicating randomness.
    2. If a few recognised the Strad then you wouldn't get the random distribution bell curve and it wouldn't be centred on 50%!
    3. Absolutely. For example: Are there only two possible responses/results (such as an ABX test) or are there multiple (such as target shooting or marathon times), sample size is another obvious example and there are various others.
    4. And yet in the referenced article he does the exact opposite!
    5. It doesn't entertain me or "tell me a lot" because I've never seen a scientific study which separates the test subjects into groups of self identified audiophiles or sceptics. However, I've seen quite a few which separates the test subjects into audiophiles and professionals (professional musicians and/or audio engineers).
    5a. Firstly, those are two very different groups, the "100 most dedicated audiophiles" is a different group to the "100 people with the most expensive audio systems", the latter group being comprised exclusively of very skilled/experienced sound engineers (who constantly use the systems) although they're mostly actually owned by share holders who've probably never heard those systems. Secondly, it has been tested! Many/Most sound engineers do test their hearing and so do musicians often/sometimes. In general, they have pretty much average hearing or lower than average, the latter typically being due to many years exposure to high SPLs. However, you seem to be confusing "hearing" with "listening skills/abilities", two very different things which are largely unrelated.
    5b. That would depend on the musician. A violinist probably wouldn't but a pro electric guitarist probably would, as distortion and IMD are important and adjustable aspects of the sound/s they are trying to produce. Of course, it also depends on how well trained you are in identifying THD or IMD and how much you exercise that training.

    1. I would agree, if this sub-forum were a audiophile marketing forum then it would have to stop. However, despite several years posting here you still haven't realised that this is the sound science sub-forum or what that means, which is bizarre considering how many times it's been explained to you!!

    2. That's partially true! For example, we have no "standard [audio] measurements" for the placebo effect, expectation or other biases which are often responsible for the differences reported by audiophiles, because these are obviously not audio properties and therefore cannot be measured with standard audio measurements. It would therefore be stupid to even try to devise a better audio measurement that would correlate with such differences! Any audible differences which actually exist in the audio itself will be revealed by the existing standard measurements. How do you not know this?

    1. True, no amplifier or other analogue audio component can ever be "absolutely perfect", due to the laws of physics. It's also true that in such a case "audibly transparent" is a statement of experience. However, you then fallaciously state that "we cannot measure audibly transparent"! Firstly, in the audiophile world we are not only dealing with such a case. We can for example run a null test and if the two signals/outputs null, we have (without question) measured "audibly transparent"! Secondly, even in the case of analogue components, where the result of a null test will never be a perfect null, if the difference file is down below -120dB then many decades of scientific "experience" tells us unequivocally that we have measured "audibly transparent". Or, if two cables' outputs differ by say 0.01dB, then scientific experience tells us this measured difference is "audibly transparent" because in many decades of countless controlled tests, not a single human being has ever demonstrated the ability to identify differences even a hundred times greater. ...
    1a. Clearly then, this statement is false! "Audibly transparent" is NOT "simply a fancy way of stating that some particular listener doesn't hear any difference", it can be a fancy way of ACCURATELY stating that no listener could hear a difference!

  13. TheSonicTruth
    What I meant by no such thing as perfect amplifier is that sooner or later parts will fail. Caps will leak, thermal circuits will fail to protect when the amp is driven hard, etc. Sound-wise, I agree: Any amplifier from 1980 to present should be sonically transparent, whether it costs $200 or $2,000.
  14. KeithEmo
    Actually, if you read the words carefully, you'll see that I made no such assumptions...

    My phrase, which you quoted, said nothing about audibility.
    I simply pointed out that a technical description of "a perfect amplifier" does exist...
    But that there is no existing piece of equipment that fully fits that description...

    The reality. which you prefer to ignore, is that there is no "blanket definition" of the word "audible" either.....
    Whether it is explicitly stated or not, that claim always includes an implied condition, or a context.....
    Whether you spell it out or not, the most you can claim is that something is or is not audible TO A CERTAIN OBSERVER UNDER CERTAIN CIRCUMSTANCES.

    For example, we can measure "the smallest sound audible at the bottom of the Howe Caverns"...
    And we can measure "the smallest sound audible 42 seconds into "The Revenge of Vera Gemini" when it's played at a typical Blue Oyster Cult concert"....
    I think we both agree that those two measurements will be quite different (so you need to specify which one you're talking about).

    An amplifier could be "audibly transparent" TO YOU.
    An amplifier could be "audibly transparent" TO ME.
    And an amplifier can be "audibly transparent" to a particular group of 42 people - based on your conclusions about a particular test in which they participated.
    And, in fact, it could be "audibly transparent" to me one day after listening to a Blue Oyster Cult concert....
    But that same amplifier just might not be "audibly transparent" to me after I spend a week at a quiet cabin in the woods (and my ears start to respond slightly differently).

    You continually repeat "on paper they are audibly transparent" as if it were a mantra... or even a widely accepted standard...
    Yet the reality is that no such standard exists...
    All we have are a bunch of opinions, some based on the results of very specific tests, and some based merely on "basic assumptions".

    At best, if you were to say that, "two amplifiers were audibly identical, within the limits of statistical error, to a certain group of 92 observers"...
    A lot of people would find that both useful and informative.
    (But you STILL can't treat it as if it were a universal fact, applicable everywhere, to everyone.)

    Incidentally.... I am not at all assuming that "all amplifiers are flawed in the same way".
    Quite the opposite.... Based on facts, I am claiming that all amplifiers are less than perfect, so all are flawed TO SOME DEGREE IN SOME WAY.
    Considering the measurements I've seen I would say that virtually every amplifier is flawed in a slightly different way.
    (And, yes, to many of us, under many conditions, many of those flaws are inaudible.)

  15. bigshot
    Ah. OK. That is true. But build quality doesn't enter into sound quality until many years later. And everything will fail eventually. The question is, if a $200 amp sounds the same as a $2000 amp, then will a $2000 amp last ten times as long as the cheap one? Odds are, the answer to that is no.
837 838 839 840 841 842 843 844 845 846
848 849 850 851 852 853 854 855 856 857

Share This Page