Why 24 bit audio and anything over 48k is not only worthless, but bad for music.

Discussion in 'Sound Science' started by keanex, Apr 30, 2014.
  1. pinnahertz
    Interestingn conclusions.

    So, your "test" was at a company event, the comparison was fully sighted, and therefore the results fully biased. Your results weren't 100%, but were they actually tallied? And from THAT you get the differences are either not being measured or that small measurable differences are audible?

    Could your methods BE any less scientific?
     
  2. 71 dB
    Cars are different in many ways while listening tests show no reliable difference between 16/44.1 and high-res beyond placebo effect. That's why Lexus makes sense.
     
  3. bigshot
    A Lexus has better features. If you are buying a more expensive piece of equipment because it has additional features that you like, that is a good reason to spend more. I have an Oppo BDP103D player, and it has the exact same video and sound quality as my $100 Sony blu-ray player, but it plays DVD-A, SACD, MKV on thumb drives and it has Darbee video noise reduction. It has a whole slew of features that justify the price. When I bought it, they also made a BDP105D, which was the exact same player, except the specs were even further into audio overkill. I passed on that one. No point to throwing good money at sound I can't hear.

    You might want to look into Handbrake. If you plan to save that video you downloaded, you can compress it to MP4 format with no real quality loss. You just need to use the proper compression scheme. In fact, if you really understand the settings in Handbrake, you can end up with a MP4 version that is *better* than the uncompressed blu-ray rip.

    Too bad the testing procedures weren't!
     
    Last edited: Nov 9, 2017
  4. KeithEmo
    This was not designed nor run as a scientific test (although I sort of wish we had taken the time to do so).
    Therefore, this is just a bit of anecdotal evidence.
    Note that everyone who posts "I don't hear any difference" is also simply adding one more piece of anecdotal evidence as well.

    I should also note that this discussion, and every test which I've seen cited (as flawed as they are), has related to the audibility of differences between different sample rates.
    Does anyone here know of any actual test to determine whether there were audible differences between different brands or models of DACs?
    If not, then we're ALL just relating various pieces of anecdotal evidence.

    However, if you're curious, my "test" was "partially sighted".........
    The switch box was in plain sight, but isn't designed to be easily read (the LEDs are relatively small and close together - and were not labelled).
    The switch also involved a key-press on the associated computer (so it wasn't as simple as "watching the hand that flips the switch").
    In some instances I was narrating the switches - "here's the DC-1...... now here's the DC-2....... now here's the DC-2 playing DSD".
    At other times it was simply - "OK.... here's one.... now here's the other... do you notice any difference?"
    This all also occurred in what I would call "a presentation situation"....
    Both DACs were set up.... and, whenever a significant number of audience members accumulated, I played some sample files for them.
    However, at least to me, it does seem significant that the vast majority of listeners claimed to hear a difference, and described the difference they heard in similar terms.
    (Nobody was really attempting to TEST for an audible difference because we all found the difference to be relatively obvious - with a wide variety of program material.)

    And, yes, in a QUALITATIVE test such as this, if any number of participants can reliably hear a difference, then the existence of an audible difference has been established.
    (They do NOT have to be able to reliably identify which is which - simply that they can reliably distinguish a DIFFERENCE when switching back and forth.
    A proper A/B/X test is testing a more complex question - and providing more information in its results - than we are talking about here.
    This is simply a matter of "I'm playing two runs - are they the same or are they different?")

     
  5. bigshot
    My evidence may be anecdotal but at least my test was controlled and wasn’t conducted by a company advertising their own products!
     
    JaeYoon likes this.
  6. pinnahertz
    1. Yes, of course.
    2. That's "sighted", not partially sighted.
    3. No, not at all. What you have is called noise in your statistics.
    4. A proper ABX test would provide you with actual scientific data. What you have is more anecdotal and statistically noisy data.
     
    JaeYoon likes this.
  7. KeithEmo
    re #3.

    Err...... no....... because it's not a statistical test; it is a qualitative test.

    In a qualitative test, if ANY SINGLE TEST SUBJECT can reliably distinguish a difference, then we have proven the existence of a distinguishable difference. This is an "all or nothing" result. If a single human, on the entire planet, can reliably tell the difference, then we have answered the question "can ANYBODY tell the difference" - in the affirmative. It makes no difference whether we find a billion other subjects who cannot - or a billion other subjects who can - because we have determined that at least one human can. Since the result we're seeking is not statistical in nature, the term "noise" is meaningless. (Of course, we might well choose to expand the test, and so to determine WHAT PERCENTAGE of humans notice the difference... but that is a separate question.)

    The only purpose of statistical analysis would be, at the individual level, to determine if our test subject was actually able to distinguish a difference or was simply random guessing.

    If I were doing this properly, I would test a whole bunch of subjects - individually. And, if a single one of them could reliably notice a difference, then I would stop the test - because I would have proven the positive assertion by producing an example of someone who could determine the difference. (Even an outlier is "part of the human population" - and so "proves the case".)

    re #4.

    An ABX test would provide lots of information.... including whether people could distinguish which SPECIFIC sample they were listening to and which they preferred. However, if you're asking a simple question, such as "is there a detectable difference or not", then you don't need such a complex test to do so. (If you're looking for a single result then you're always better off using the simplest protocol that provides it.)

    For example, all you need to do is to play a series of PAIRS of test samples. Assuming we're trying to determine whether "there are audible differences between A and B" we would play a random mix of pairs (A+A, B+B, and A+B). The test subjects would then be asked simply to state whether the two samples were the same or different. There is no need to determine if they can tell which is which - as long as they can detect whether they are the same or different.
    Likewise, there is no need to know what criteria they are using to determine similarity.
    If they can reliably identify which pairs were "mixed" and which are "the same" then we have established that the two different samples can be reliably distinguished. (Although, assuming that the subjects can reliably determine that the samples are different, FURTHER TESTS to determine how they're distinguishing them would be useful. And I'm sure the Marketing Department would love to know if they consider the difference worth paying extra for - or can even tell which is which.) Note that our initial test has NOT determined "which is better" nor "how big the difference is" - merely that a recognizable difference does in fact exist.

    So, for example, if we're testing to see if different brands of DACs sound identical or not, all we need to ask is: "I've just played two samples for you; were they played on the same DAC or different DACs?" If the subject answers correctly a statistically significant percentage of the time - beyond random chance - then we've proven that there are audible differences. The test protocol is much simpler, and answers the actual question we're asking, with no extra complexity.

    In this case "are two items identical or not" is a very simple criterion to test for.... The only real pitfall is that you might end up with false-null results if your samples or test protocol are badly chosen. For example, the difference may be undetectable using certain speakers, or a certain amplifier, or certain test samples, or even a certain sample population. (For example, since we all agree that the ability to hear high frequencies decreases with age, it seems to make sense to include at least some children in tests for the audibility of high frequencies; using 60 year old audiophiles is surely a badly chosen test population.) HOWEVER, any legitimate positive is definitive..... because, if a legitimate difference exists, which can be demonstrated under ANY circumstances, then the two UUTs are "different". (So, for our purposes, if one ten year old child reliably hears a difference, it doesn't matter how many 50 year old audiophiles fail to do so.)

    The real purpose of a simple test like that would be to serve as "a qualifying round". If we were TOTALLY UNABLE TO FIND A SINGLE SUBJECT WHO COULD RELIABLY NOTICE A DIFFERENCE, then we would have a null-result. And that null-result would preclude future more extensive testing as long as it remains authoritative. It means that you have not successfully completed the qualifying round. (Once you prove there are differences, there are all sorts of additional things you can test for. Until then, you have nothing to test... beyond perhaps looking for more sensitive tests that might detect the differences that have so far eluded you.)

    All we have at the moment is a positive but unreliable result.
    However, that is in fact rather different than a reliable negative result.

     
    Last edited: Nov 10, 2017
  8. castleofargh Contributor
    you're overreaching with your conclusions. maybe this specific test was very clean in some instances, and maybe people weren't all pre-fed with marketing expectations? but the way you describe it, it's just a nice sighted test while presenting a product. did I misunderstand the protocol involved?
    if I did not, then no the existence of an audible difference hasn't been established. people "hear" differences when there is none to hear all the time when asked if they do. that's why we need a control to take those false positive out. maybe it's there and maybe it's enough for people to reach that conclusion based on sound alone. but as long as the test isn't about sound alone, it isn't conclusive about sound.

    because we seem to get into war those days and it's not my aim at all, I want to add that I'm not trying to suggest your DACs don't sound different. you know better than I do if that's the case or not. my remark is only about the protocol of the experience and the range of conclusions we can reach. and contrary to what it may start to look like(maybe it's irrelevant, maybe you care), I'm not on a crusade against you. I actually enjoy having you posting here, and I'm proud to say that I've learned several things from previous posts of yours. but I'm skeptical first, and fanboy second. ^_^
    the wish to disprove everything is not personal, is what I'm trying to say poorly here. my default position, maybe I'm wrong about that and need a more open approach, is to go for the null hypothesis and move from there. so this always makes me look like I'm thinking everything is the same. I'm not. it's just my starting point and I wait for evidence to resist my hypothesis.
     
    JaeYoon likes this.
  9. pinnahertz
    Qualitative testing and statistical analysis are not mutually exclusive, but rather, if any qualitative test is to have any meaning other than biased opinion, it must be statistically significant. One result is statistically too noisy, and in fact completely meaningless.
     
  10. KeithEmo
    Hmmmmmm..... maybe because the methodology involved in doing so is flawed.......

    We have two "widely accepted claims" about the thresholds of human perception.
    Both are based on very specific test conditions.
    Yet you want to generalize them to ALL conditions - without offering any sort of proof that such a generalization is valid.

    Personally, I would NOT be willing to bet $1 million that someone cannot produce a single living human being who can hear 22 kHz.
    (And, no, the fact that it hasn't happened yet does not PROVE that it cannot or will not happen.)

    At best you can state that nobody has demonstrated that any of the claimed benefits have been shown to be audible in a properly designed and executed test.
    (And I'll cheerfully agree with you there.)

    To answer your question.... I don't specifically expect to hear anything... but I can't entirely rule it out based on the currently available evidence either.
    (And, even if I can't hear it, I can't rule out the possibility that someone else will. After all, I don't have "perfect pitch" either, yet some humans do.)

     
  11. bigshot
    I read somewhere that the record for highest frequency heard was 23kHz. But that really isn't all that much. 20kHz to 40kHz is one octave... 7 musical notes. 20kHz to 23kHz doesn't even get from "doe" to "rey". It's a drop in the bucket. That's the thing. You need to keep in mind relative importance. Frequencies above 17kHz are largely irrelevant when it comes to recorded music. A large proportion of the population can't hear them at all, the majority of musical instruments can't produce those frequencies, sound that high doesn't propagate through a room well at all, it's far beyond the point where we can perceive a musical note in sound, and even if those frequencies exist in recorded music they aren't audible because they're masked by lower frequencies. Super high frequencies just don't matter. If you focus on them, you'll be neglecting the things that really do matter.

    Frequencies up that high are as useful as teats on a bull hog.
     
    Last edited: Nov 10, 2017
    JaeYoon likes this.
  12. KeithEmo
    I would like to be able to disagree with your initial statement - because I have tried to avoid claiming any conclusions whatsoever.
    However, I agree entirely with your statement that "I didn't prove anything".
    My only claim is that I failed to rule out the need for further testing... as has everyone else.
    Actually, since people claimed to hear differences under certain conditions, it seems reasonable to suggest using those conditions as a starting point for a real test.

    My only intent was to offer a counterbalance to various statements from people stating "I didn't hear any difference - so they sound the same".
    I simply offered a significant quantity of anecdotal evidence that "a lot of people do seem to believe they hear a difference".
    (Mostly to offset the perception that "nobody hears any differences".)
    In terms of preference and bias, as with any situation, this one was quite complicated.
    Some people would undoubtedly prefer to believe that the new product sounds better; while some, who own the current product, would no doubt prefer to believe they have no reason to upgrade.
    I also note that, up until that point, we had been promoting the new product based on added features rather than improved sound.....
    (Of course, we have now adjusted our marketing strategy, based on those results..... we'd be dumb not to mention positive testimonials.)

    In fact, I'm not claiming to have presented any evidence at all.... beyond some anecdotal evidence that suggests that actual testing MIGHT turn up something interesting.
    (Incidentally, I'm also not at all convinced personally that high-res files do or do not sound audibly different... and neither I nor anyone else has done what I consider to be sufficient testing to prove the point either way.)

    However, when the discussion shifts to the question of whether all DACs sound audibly identical, the whole discussion devolves into fantasy.
    Clearly all DACs do NOT sound the same, and all DACs do not measure the same, or even close.
    (For example, one DAC I owned had a very dull sounding high-end, almost certainly related to the fact that its frequency response was -3 dB at 20 kHz.)
    So the real question is whether DACs sound different IN WAYS NOT OBVIOUSLY RELATED TO DIFFERENCES IN MEASURED PERFORMANCE.
    And, in order to test that question, we must predict what specific measured differences are audible.

    We can't simply say "all DACs sound the same" - because obviously they don't.
    We need to make a specific claim which can be tested - for example "all DACs which are flat within 0.2 dB from 20-20 kHz, and have a THD below 0.05%, and a S/N ratio above 95 dB, sound the same".
    Once we have that specific claim, we can choose a bunch of DACs that meet the required criteria, and then we can see if people can tell them apart.

    My experience, and that of many people I know, is that the vague claims being made by a lot of people don't seem to agree with our experience.
    I've owned many DACs which had "very flat frequency response; very low noise; and very low distortion" and which still sounded distinctly different to me.
    Therefore, either:
    1) I (and they) must have been imagining the differences
    2) the claimed specs were wrong, and real major measurable differences existed - even though the published specs said otherwise (unless you measure them yourself it's not really a good idea to accept the manufacturer's specs)
    3) the claimed specs were right, but the differences were audible, and the assumption that the differences were "below the threshold of audibility" is wrong
    4) there were easily measurable differences that simply don't fall under the specs we're currently measuring

    I'm also going to offer an interesting example... in the form of a "thought experiment".
    As with many thought experiments, this takes the idea to the absurd - but not the impossible.
    I'm going to start with a perfectly clean tone burst - 200 milliseconds of a 440 Hz sine wave tone - sampled at 44.1 kHz.
    I'm now going to create a really silly filter - a 440 Hz bandpass filter one billionth of a cycle wide, with really sharp skirts, centered at 440 Hz.
    If I pass my my 200 millisecond tone through this filter, it will experience some (an absurd amount) of "pre and post ringing".
    In fact, I may add several seconds of ringing to my original relatively short burst.
    However, assuming I don't screw up, I will have altered the time/energy envelope, but the frequency, THD, and noise floor, will remain unchanged.
    (I will have changed my 200 msec burst into a twenty second tone that gradually increases to full amplitude, then gradually declines to zero asymptotically, without adding THD or noise or shifting the frequency.)
    I'm betting that the difference will be audible.
    (Again, note that my filter has NOT altered the frequency response, THD, or noise level......)

    To be totally honest here, I feel bad arguing against skepticism, when I personally believe that too many audiophiles are far too credulous.
    Skepticism is always a good thing.
    (However, that does extend to skepticism against ALL CLAIMS... especially against claims of negative proof... because negatives are damn near impossible to prove.)
    I also agree that, in practical situations, absolutes are often not the best way to discuss things.
    It's probably not a good idea for everyone to pay extra for something that 99% of the people can't hear..... unless they happen to be one of the 1%, or just prefer to buy "insurance" against the possibility.....

     
  13. bigshot
    And a lot of people believe that they can communicate through telepathy with life forms in an alternate universe.

    You have generally accepted science. It is what it is. Then you have claims that contradict that. That's when you require a controlled test to prove that the generally accepted science is wrong. Those of us who have been in this forum for a while, have seen people come in here claiming all sorts of things that contradict established scientific principles. They'll talk and they'll talk, but they never go beyond the anecdotal. Most of them just end up getting mad and stomp off when one of the Sound Science regulars give them a "put up or shut up" challenge. A very tiny percentage are disingenuous and try to trick us. Very few go to the trouble to set up a test for themselves and try to prove something. At best they make a half hearted attempt and then gleefully indulge their bias, making deliberate compromises to undermine any value of the results. Why would they do that? Ego. They don't want to be wrong. Unless they can get past their own ego, they will always have problems with bias.

    The truth is, if some generally accepted scientific principle of audibility is wrong, there are a lot of scientists and sound engineers who would like to know that, and they would be eager to hear what the truth is. Prove Nyquist wrong and you'll be famous. Your ego will be fed. But you better do it fair and square. It's easy to pump out a lot of smoke and raise a bunch of questions and blather on and on about things that may or not be true. High end audio salesmen do that all the time. They train audiophools to march into the Sound Science forum and do a parrot paraphrase version of their sales tear sheets. But that doesn't move actual knowledge an inch forward.

    My approach is simple. I define what it is that I want... a sound system that plays music in my home and sounds great. Then I focus on understanding what makes a sound system in a home sound great. I don't worry about theoretical sound. I don't try to feed my ego. I only care about results. I focus on what I can hear. I start at the top of the priority list and work my way down. I set up a simple system for doing controlled listening tests and I verify that what I'm doing is working and nothing is sabotaging my efforts that I'm not aware of. When I reach audio nirvana, I'm happy and I listen to a whole lot of music. Sometimes I think I'm an anomaly. There aren't a lot of people out there approaching the problem of recorded sound in a practical and organized way like this. But I'm a producer and that's how my brain works. I have a goal, I break down the steps to reach that goal, I move forward efficiently, I check my work. When I reach the goal, I relax and enjoy my accomplishment. If everyone did that, we would all have great sounding systems, we wouldn't be churning through upgrade after meaningless upgrade, and we wouldn't be wasting a whole lot of money on transistors and circuits that would be better spent on music.

    And every DAC and player I’ve ever come across all sound the same. I imagine there are some out there that might sound different, but if I bought one, I’d pack it back up and return it as defective. I require a digital player to be audibly transparent.
     
    Last edited: Nov 10, 2017
  14. KeithEmo
    There actually was a time when 19 kHz was a problem.

    The way FM stereo works - and maintains compatibility with FM mono - is interesting. FM stereo is actually delivered as two matrixed channels; a main channel which is mono L+R, and a difference channel L-R.
    A monaural tuner is able to simply play the main L+R channel without doing anything special, while a stereo FM tuner extracts the L-R channel, then does the math to derive separate L and R channels.
    This way the same signal is compatible with both.
    However, in the actual broadcast, the L-R channel is delivered as a signal modulated on a 19 kHz subcarrier, which is part of the main audio signal.
    The tuner must demodulate and extract the L-R channel, use the result to recreate the stereo audio signal, and then filter the 19 kHz subcarrier out of the main channel.
    However, FM mono tuners, built before the stereo standard existed, failed to filter out the 19 kHz subcarrier at all, and many early stereo tuners didn't do so especially well.
    As a result, many vintage FM tuners included a significant amount of 19 kHz noise mixed in with the audio.
    The 19 kHz noise wasn't very audible as noise..... but could often be noticed as a sort of pressure in the ear (if you walked too close to the tweeter it almost felt like a jet of air blowing in your ear).
    Many tweeters in those days simply didn't reproduce it at all, while others delivered it in a very narrow beam, directly in front of the speaker, due to narrow dispersion at that frequency.....

    I would agree that the value of such high frequencies as "musical content" is rather dubious.
    However, good high frequency response does contribute to fidelity in some situations: for example, enabling the sound of recorded cymbals to sound "properly metallic", and helping the recorded sound of breaking glass to sound natural.
    (A well recorded wire-brush cymbal should be audibly different than the hiss of steam escaping from a relief valve.)
    So, if the high frequencies roll off too soon, or too sharply, it can adversely affect how natural a recording sounds.

     
  15. KeithEmo
    I don't disagree at all...... however we do need to distinguish exactly what we mean by "generally accepted science" and "absolutely undisputed facts".
    For example, several quite popular, and largely informative, videos would have us believe that "there are only three significant types of distortion: THD, IMD, and noise".
    While this was true a few decades ago, when both amplifiers and recordings were analog, it is no longer true - because digital technology has introduced several new ways in which the audio signal can be unintentionally altered.
    (For example, a faulty surround sound decoder could actually route the sound of a certain instrument in a recording to the wrong speaker; it's clearly wrong, but which category would you place the error in?)

    Generally accepted science does sometimes turn out to be wrong.... or incorrect in a new context.... or sometimes simply insufficient to fully understand a new context.
    I've personally been around the audio industry for a long time..... and I've heard plenty of whoppers in both directions.
    Therefore, I'm inclined to do my best to avoid saying things like: "Don't bother to test it; it can't possibly be true."
    (But, yes, being human, I do have certain areas where I allow my opinion to overrule that guideline.)
    However, I would always agree that you shouldn't take anything "on faith" - especially if ti seems to conflict with established science.

    I would also note (I wish I could claim credit for this one):
    Some audiophiles enjoy listening to music, while others seem to listen mostly to their equipment, and only some few seem to do both.
    (And I guess that omits the ones who prefer to DISCUSS the subject rather than listen to it :) )


     

Share This Page