[Phronesis]

something we'd be able to notice when it goes away, we'd notice with a switch. I get what you're suggesting, but what are the odds for something creating an impact to be at the same unnoticeable with rapid switching? it would need to have a really small variation compared to the other signal, and then I doubt the consequences would turn out to be meaningful anyway. or it would need to be something with fairly high energy but outside our hearing range(think the radiations of sound, strong but we just happen not to have sensors readily available for it). ultrasounds could probably be that if they happened to have some impact on our body in the long run while we're unaware of their presence. but then high energy inaudible stuff would most likely turn out to be bad for us.

I don't think it's that easy to manufacture a situation where sound variations we don't notice with rapid switching turn out to have some significant impact anyway. at least I don't expect that to happen in the audible range.

Maybe the best way to test this is to manipulate files in various ways to various degrees, and then test with listeners using different time frames. It’s more a test of perception than gear. My hypothesis is still that being insignificant in the short term generally means insignificant in the long term, but that’s just a hypothesis. I don’t know of any good testing that’s been done for this.

 

[KeithEmo]

You clearly didn't actually READ what I wrote.

I really try to be civil - but I give up.

When you analyze a group of samples, looking for something of statistcial significance, you can either find it - or not. YOU CANNOT STATISTICALLY PROVE THE NEGATIVE. All you can prove is that you made a reasonable attempt to find something and failed to do so. From this you may INFER that what you were looking for wasn't there BUT YOU HAVE NOT PROVEN IT. If you examine a thousand samples of dirt from my back yard, looking for diamonds, and find none, YOU HAVE NOT PROVEN THAT THERE ARE NO DIAMONDS IN MY BACK YARD. You have simpy failed to find any proof that there are.

The M&M test failed to show statistically that any significant number of test subjects could reliably hear a difference beyond what would be expected from random chance. However, several subjects as individuals scored very wel, AND NOBODY BOTHERED TO FOLLOW UP TO DETERMINE IF THEIR RESULTS WERE DUE TO RANDOM CHANCE OR NOT. (From their results there is no way to even guess whether the guy who got 8/10 right could do so every time, or only that once, because they failed to follow up on his results.)

I simply stated that "many people do not find CDs to sound audibly perfect". I SAID NOTHING ABOUT A TEST. If you read a few, or a few thousand, reviews of CDs, you will find that a massive number of people find them to sound "not audibly perfect". In fact, I have yet to find an issue of Stereophile where at least a few people haven't commented about how bad this or that DCD sounds, or that the vinyl version sounds better. Therefore, it would be ridiculous to suggest that "everyone thinks CDs sound perfect".

There you go again, making a claim based on your misuderstanding of statistics and your misinterpretation of results. This appears to be a continuation of your misinterpretation of the M&M study where you claim it proved some could hear a difference in normal listening environments, despite the flaws in your observations being clearly pointed out to you.

I have yet to see one credible test where all other factors apart from 16/44 are controlled that supports your assertion of exceptions or even more absurd, that that they "fail so much of the time".

Still flogging the dead horse? I like Emotiva products but peddling FUD on a sound science forum does you or your company you represent no favours.

 

[Phronesis]

The thread is getting weird and pointlessly contentious lately.

IMO, the bottom line is that if we want to draw general conclusions about what gear sounds different (or not) to what degree, we need an extensive program of good testing. Without such testing, all we have to go by is a lot of non-rigorous controlled testing and a lot of casual sighted listening, and the two categories give conflicting results.

Most people who've done both believe that differences in gear other than transducers are very small, and have firsthand experience with their own expectation bias. Many people who've only done the latter believe that there are significant differences in gear other than transducers, and are largely unaware of their own expectation bias. Two different sets of experiences, and two resulting different opinions.

The matter won't be conclusively settled until we have an extensive program of good testing, and that may never happen. So what else is there to discuss? Are people just arguing because they're bored or like to argue?

 

[KeithEmo]

That's a good idea. The reality is that quite often expensive DACs don't perform any better than low cost ones, and some even perform notably worse, but many people seem not to realize this... and it would make a good point.

I would also suggest specifically comparing a unit that used an ESS Sabre DAC to one that uses another brand - a Texas Instruments Burr Brown or a Wolfson. From my experience, and a huge number of anecdotal reports, Sabre DACs tend to sound different from other brands - and to have a sort of "house sound" - even though their basic measurements are very good.

To get a result that gets people's attention, I suggest comparing a cheap DAC in the $100 range with something like the Chord Dave, which I believe is around $10K.

 

[Killcomic]

The matter won't be conclusively settled until we have an extensive program of good testing, and that may never happen. So what else is there to discuss? Are people just arguing because they're bored or like to argue?

Well, you know, audio equipment for audiophiles is a penis extension, hence why so many middle aged guys here. I've noticed there's this almost desperate need to demonstrate that they have the magical ability to hear sounds beyond normal human hearing. It is what they hope to set them apart from the common rabble. That's why there's so much utterly meaningless language like describing something as "musical" and sounding "organic".
To actually admit that you've been buying very expensive equipment that yields no real benefits, it's a bit embarrassing. It is counterproductive to the myth they would like to surround themselves with, because you know, penis extension.
Could be worse, they could buy a sport car.

 

[gregorio]

[1] The thread is getting weird and pointlessly contentious lately.
[2] IMO, the bottom line is that if we want to draw general conclusions about what gear sounds different (or not) to what degree,
[2a] we need an extensive program of good testing. Without such testing, all we have to go by is a lot of non-rigorous controlled testing and a lot of casual sighted listening, and the two categories give conflicting results.

1. Not just lately but for quite a while!

2. Well that's the issue; drawing a "general conclusion". There is not and cannot be any absolute proof because we cannot test everyone alive, nor everyone who was alive or will be alive. We therefore have to draw a "general conclusion" on the balance of probabilities and this is where it "gets weird" because we have an overwhelming probability on one side, supported with lots of reliable evidence and a hypothetical possibility on the other, supported by no reliable evidence at all.

2a. We've had lots of "good" testing, going back many years. Some of it is published as scientific papers and much of it isn't, such as that carried out by international bodies, audio professionals and university departments/educators. Therefore it's extremely unlikely there will be another program of extensive testing because as far as the professional and scientific communities are concerned the matter is effectively "put to bed". To get it out of bed again would require some serious evidence, but audiophile marketing and audiophile testimonials do not qualify as "serious evidence". So we have the impasse of a segment of the audiophile community vs pretty much everyone else and as that segment appears immune to logical/rational arguments, the scientific and professional communities simply leave them to it.

G

 

[KeithEmo]

That makes sense.....

If you would like to suggest a particular test, I'll be glad to try to point out at least the obvious flaws.
I'm going to start with the Meyers and Moran study.

One of the most common errors I see is the MISAPPLICATION of statistics to situations where they don't apply.

The asserted purpose of the Meyers and Moran test was "to determine whether people could reliably determine an audible difference when a "CD loop" was inserted into the audio signal chain". Most of us here interpret that to specifically mean "whether any human being can reliably determine the difference". Assuming that to be the case, then here is the first (MAJOR) error. Because of the nature of the testing and reporting involved, THE PERFORMANCE OF EACH INDIVIDUAL MUST BE ANALYZED STATISTICALLY. The only way to tell whether an individual can reliably tell the difference is to run a bunch of test trials and see whether their responses score significantly better than would be expected by random chance.

However, if you apply a bit of logic, it will be obvious that there is no reason to apply statistics to the OVERALL results, and doing so would be improper in light of the results desired. If the purpose of our test is to determine whether ANY HUMANS can reliably detect a difference, then, as soon as we find a single human who can do so, we have our answer - which is yes. We have a human who can do so, therefore "some humans can do so". That is a simple YES/NO question and NOT a statistical question. (It doesn't matter how many can do it; once we find one who can , we're done, and the answer is known: yes. We might go ahead and see how many can do it, and probably would do so, but that is outside the scope of the original test.)

If Meyers and Moran were REALLY trying to determine "whether most people would hear a difference" or "how many people would hear a difference", then statistically analyzing the overall results was a perfectly valid way to do so. I Strongly suspect that this was what they were trying to do - since this is what most of the public, as well as most CD vendors, would really be interested in. Therefore, attempting to claim that their test "showed that nobody could do it" is simply a misinterpretation of their results. Their results simply showed that "they were unable to suggest that a statistically significant percentage of the people they tested could reliably determine a difference a statistically significant percentage of the time".

(It is also absolutely true that "a statistically significant percentage of the human population is not able to run a mile in four minutes". However, that is FAR different from "proving that nobody can do it", along several different axes.)

Here is the proper way to run a test "to determine whether ANYONE can reliably detect a difference".

You conduct the first level of the test exactly as they did, using statistical analysis ON EACH INDIVIDUAL'S RESULTS, to determine whether each individual test subject scored above what might reasonably be expected by random chance. However, you treat the first level of the test as a screening round. You assume that all the subjects whose results fell very close to random are "uninteresting" and send then home. HOWEVER, if any subjects produced individual results that appeared significant, you conduct further test runs with them to determine whether their results were random or not. Typically you would set some sort of threshold level for "subjects to be passed on to round two". This would typically be either a direct threshold like "everyone who scores above 7/10" or a fraction like "the best scoring 10%". (And, if nobody scored above that threshold, then you would either declare a null result, or find a new set of test subjects and try again.)

At this point you should also note and report two obvious potential causes of error:
1) Your sample didn't contain all humans - so you could simply have missed one who could do it.
2) It's possible that one of your subjects could do so - and his or her failure this time was the anomaly.

So far that constitutes either an error of "performing a test that is inappropriate to the desired results" or "attempting to mis-interpret their results to prove something the test was never intended to measure".

The test itself also contains numerous errors of methodology and procedure.

We all know that, whether it's audible or not, the CD signal chain fails to exactly reproduce the original audio in many measurable ways. It adds some noise, some distortion, and some errors in frequency response and impulse response, as well as possibly other unknown differences. They should have provided detailed analyses of their test content, both "before and after". They were attempting to determine whether their sample listeners could detect differences - YET THEY FAILED TO PROVIDE DETAILED INFORMATION ABOUT THE DIFFERENCES PRESENT IN SAMPLES IN THE TEST. For all we know they used low quality original material, the CD signal loop made no measurable difference to it, and there were no measurable differences between them. They should have provided both detailed data, like product disc numbers and specifications, as well as test results like spectrum analyses, showing what was present in each sample, and what the measured differences were. In fact, since the "item of interest" was the differences, they probably should have provided an actual difference file containing those differences for easy analysis and confirmation. They should also have included measurements that showed that the differences involved were being accurately presented to their listeners - and not being obscured by any of the components used for the test. Then we would know precisely what differences the subjects were or were not able to hear.

I should note, in their defense, that they DID note the anomalous results obtained from several test subjects.

It was always my impression that Meyers and Moran simply set out to "provide a simple test to demonstrate that, when used to reproduce typical consumer-quality content, most consumers didn't notice any audible degradation caused by CDs".

I’ve seen repeated commentary here suggesting the tests documented in the first post and others may be flawed. That may or may not be true, but if that assertion is goin
g to be made, then it’s encumbent upon the person suggesting the tests were flawed to be specific about those flaws, not just throw a turd in the punch bowl.

General statements that “there might be flaws” aren’t helpful and could be construed as deflection, particularly from those averse to participating in reasonable testing. While it may not be conclusive, an ABX via Foobar is quite simple to construct and enough data gathered may be inidicative. Particularly if statistics aren’t abused and single run results aren’t stated as being significant. If someone can score 8/10 or better on 20 test runs, then we have something to discuss.

 

[KeithEmo]

Quite true.

However, in this case, we are assuming that all of the products we're discussing already have exceptionally low THD, exceptionally flat frequency response, and very low levels of noise - by "commonly accepted standards". The specs on most DACs, even most cheap ones, are quite good... although we may always question whether they actually meet those specs when tested. If there was a simple and audible flaw, like too much bass or treble, that would be obvious on a basic frequency response measurement. Likewise, excessive THD would show up on a distortion analyzer.

If I may drag things down to my level, I notice if there is too much bass or too much treble or uneven bass and that is fatiguing or tiring or whatever and that's easy to fix with EQ. If I close my eyes and switch between two settings it can often be clear in my lay-opinion that part of the spectrum is over or under emphasized and that can be fixed with EQ. An extra subwoofer has helped keep the bass even in my room (a nightmare of a room, by the way, L-shaped with an opening to other parts of the house). You can walk around the room and hear that it is much more even with the two subwoofers. I can make more minor adjustments and close my eyes (a sort of self-imposed and very vulnerable and flawed blind test) and switch in and out and have a very hard time deciding which is "better." That's my happy place. Once I settle down and relax I enjoy it.

When it comes down to having to do a a blind A/B or A/B/X test to hear very minor differences, I've done that, and it's tedious and difficult and reassuring to know everything is cool, at least to my ears, but I'm just confirming that my equipment and lossy encoding are working right. I'm not really making any progress in my set-up.

 

[KeithEmo]

Excellent point....

Which is why, if you're choosing equipment to purchase for yourself, then what YOU are able to discern is the most important factor. The one exception is that I would always "aim a little higher" as a sort of safety margin.

For one thing, you don't want to purchase a whole bunch of lossy files, based on the fact that you can't hear the difference with the headphones you are using today, only to find that they sound obviously worse on the higher quality headphones or other equipment you purchase tomorrow.... (It's true that you would be wasting money to purchase lossless files if you don't ever hear a difference. However, it's also true that some people who purchased 128k AAC files on iTunes, which sounded "just fine" on their cheap headphones, ended up buying them over again as lossless files when their flaws became obvious on better equipment. It's a bit of a slippery slope - in both directions.)

I should also point out that both our hearing and our perception change with time. Physically, our hearing becomes less acute as we get older. However, many of us "learn to listen more critically" over time, and actually do notice things that we may have ignored or failed to notice previously.

Harman International has an excellent little (free) app for "training your hearing":

http://harmanhowtolisten.blogspot.com/

Oh for the love of... You'll first need to tell the difference between a high quality lossy vs lossless, only then do you even consider going higher.
Everyone here thinks they got dog hearing.

 

[KeithEmo]

The two "most highly regarded" commercial programs for converting between DSD and PCM are probably
Weiss Saracon and Korg Audiogate.

Saracon has consistently remained very expensive...
However Audiogate has, at various times, been available at low cost, or even free...

(For a while Korg was offering it for free - but only if you would allow it to Tweet "I'm using Korg Audiogate" whenever you started it up... yeah... really. I haven't checked the situation lately...

For starters, I would like - LOVE is the more appropriate word - DSD to PCM conversion software would be better in the first place.

The annoying phase differences between the two channels that some ( most ?) conversions from DSD to PCM produce are audible in the extreme ... - and there ARE combinations of software/hardware that would do that with PCM, too.

 

[Phronesis]

2a. We've had lots of "good" testing, going back many years. Some of it is published as scientific papers and much of it isn't, such as that carried out by international bodies, audio professionals and university departments/educators. Therefore it's extremely unlikely there will be another program of extensive testing because as far as the professional and scientific communities are concerned the matter is effectively "put to bed".

If people could provide links to info on good testing available to the public, that would helpful. If there's better testing out there than what's in the first post of this thread, I'd certainly like to read about it.

 

[Phronesis]

Well, you know, audio equipment for audiophiles is a penis extension, hence why so many middle aged guys here. I've noticed there's this almost desperate need to demonstrate that they have the magical ability to hear sounds beyond normal human hearing. It is what they hope to set them apart from the common rabble. That's why there's so much utterly meaningless language like describing something as "musical" and sounding "organic".
To actually admit that you've been buying very expensive equipment that yields no real benefits, it's a bit embarrassing. It is counterproductive to the myth they would like to surround themselves with, because you know, penis extension.
Could be worse, they could buy a sport car.

Probably all true to some extent, but I wouldn't want to generalize it too much.

I see a lot of people admitting that they're saving up to buy gear, so clearly not trying to project an image of affluence. And audio expenditures are peanuts for anyone with a decent amount of money, so even a $10K DAC isn't really a big deal (though perhaps a waste of money) in the scheme of things, still a lot cheaper than a car or college education. Personally, I don't mind saying that maybe it turns out that my Hugo 2 doesn't sound any better than my Mojo despite costing 5x more and my buying it because I thought it sounded better (I do like the looks, amp power, and portability), though I might feel peeved if I had bought a Dave and had the same result. Generally, I try to find truth, even if I don't like the truth.

As far as magical hearing, I would again go back to the point that people perceive clear differences (even if they're not really there), so when other people say they don't hear them, it's natural to conclude that their own hearing ability is better (even if it's not). And of course, there is some variation in hearing ability, and I'm pretty sure that my own hearing ability has improved with effort and experience.

Regarding sports cars, a lot of us buy them because we like driving them (I've actually tracked every car in my garage, including the SUV and sedan). Status display can be a factor, but a lot of us don't like the attention that sports cars garner.

 

[bigshot]

Well, you know, audio equipment for audiophiles is a penis extension, hence why so many middle aged guys here. I've noticed there's this almost desperate need to demonstrate that they have the magical ability to hear sounds beyond normal human hearing. It is what they hope to set them apart from the common rabble.

There's a psychology to internet forums that is stacked up on top of that too... There are people who feel inadequate in real life who try to construct online identities that are designed to make people think they are experts. It's a lot easier to be a "bigshot" in an internet forum than it is to be one in real life. Often these people will park themselves in a single thread or forum and comment on each and every post that comes by either talking about themselves and their personal thoughts and opinions, or cut and paste information from google searches. They rarely actually interact with anyone, choosing instead to look at other people's posts as springboards for launching into their own extended rambling monologues. These types feel threatened by people who actually do have experience and knowledge on the subject and attack them to avoid being shown up as not being what they're pretending to be. We've seen a particularly extreme example of that in this forum where multiple armchair forum "experts" gang up on Gregorio because he clearly is better informed and has more experience on the subject than most of us here.

When you add the "internet expert" persona to the "it's either your cheap stereo or you're deaf" one, you end up with a particularly obnoxious combination.

Well, repeatable results are a requirement. So far no evidence has been presented.

I and others here are offering to provide independent verification of a clear audible difference in a DAC. All we need is a clearly different sounding DAC to check.... crickets. It's one thing to spend hours and days and weeks and months spewing out paragraph after paragraph of unsubstantiated opinions, but backing them up seems to be too much effort for some people. All they want to do is tell other people how they should back up their opinions!

 

[analogsurviver]

The two "most highly regarded" commercial programs for converting between DSD and PCM are probably
Weiss Saracon and Korg Audiogate.

Saracon has consistently remained very expensive...
However Audiogate has, at various times, been available at low cost, or even free...

(For a while Korg was offering it for free - but only if you would allow it to Tweet "I'm using Korg Audiogate" whenever you started it up... yeah... really. I haven't checked the situation lately...

I am well aware of these two - and it is precisely because of this I wrote what I have written.

Since I own 5 Korg DSD recorders ( 3x MR-1 & 2x MR-1000 ), I have never had any problems with obtaining and authorizing ( the High Quality version with all the functionality gets authorized to the computer to which you want to install Audiogate by connecting any of the Korg DSD capable recorders or DACs - free is only Lite Load version, which has MUCH worse sound quality/conversion and next to none but the most basic functionality ) of the latest Korg Audiogate software.

Saracon has consistently remained very expensive. For me, it does THE capital sin - its low pass is set way too low, it filters out everything by ( 30 ? 40 ? kHz ) . I would have to re-check the comparisons with spectrum analysis etc , published a few years ago on https://www.whatsbestforum.com/ - when it was being decided which hardware and software should be used for DSD by now commercially available transfers of the better>best analogue recordings, etc. That excessive filtering above 20 kHz plus $$$ has been more than deterrant enough not to mess with Saracon.

Korg Audiogate went from Version 2.0 trough many updates to Ver.3.0 ( here somewhere it caught up in SQ with the then latest jRiver ) and finished off by Version 4.0 and very few updates, which basically added recording both in DSD and PCM, limited to DSD128 and PCM192.

It has one very hard to swallow limitation - it works in DSD - but ONLY using Korg devices. Everything else, even if native DSD capable, will work only by conversion to PCM.

As none of the other DSD player soft can "see" markers made during recording with Korg recorders, I did consider adding a Korg DSD DAC just because of this - sound quality of the DSD file over the PCM takes a MAJOR hit in this case ... - insisting on native DSD playback using non Korg DAC and finding a particular spot in a "take" of one song ( actually, MANY takes of the same song in one single file ) with just a slider in style of F2K or jRiver is, for all practical purposes, hopeless.

As I write this, I am fiddling with one of the latest additions to the DSD players/editors - TASCAM Hi Res Editor that I just downloaded and installed. It allows for processing/conversion DSD up to DSD256 and PCM up to 384kHz.
It is FAR less robust than either Korg Audiogate, F2K , jRiver - it will stutter where all of the three mentioned will work flawlessly. Will have to see if it can be adjusted etc a bit better - but, very few controls over anything ... - BUT IT DOES warn you, at each launch, to set your laptop to High Performance ( high battery consumprtion )...

THE problem I am having with Korg Audiogate ( any version... ) is the fact that it introduces phase lag between the channels - which I have already described some half or so year ago. Square wave recorded to DSD ( which, by default, can not introduce such an error ... ) that gets perfectly recporded and reproduced on Korg recorder(s) , with total synchronicity between the two channels, gets lost when converted using Audiogate. It is ALWAYS the precise delay between the two channels equal to the rise time in any given sampling frequency of PCM - from 44.1 to 192 kHz. In other words - one channel is still silent when the other has reached its full output - or vice versa... I forgot which channel is leading and which lagging - but it always remains the same.

I will convert the approx 3 kHz square wave test files using TASCAM Hi Res Editor - AGAIN - and, hopefully, that unwanted delay between the channels will go away....
Will report what happened tomorrow.

 

[Killcomic]

Probably all true to some extent, but I wouldn't want to generalize it too much.

I see a lot of people admitting that they're saving up to buy gear, so clearly not trying to project an image of affluence. And audio expenditures are peanuts for anyone with a decent amount of money, so even a $10K DAC isn't really a big deal (though perhaps a waste of money) in the scheme of things, still a lot cheaper than a car or college education. Personally, I don't mind saying that maybe it turns out that my Hugo 2 doesn't sound any better than my Mojo despite costing 5x more and my buying it because I thought it sounded better (I do like the looks, amp power, and portability), though I might feel peeved if I had bought a Dave and had the same result. Generally, I try to find truth, even if I don't like the truth.

As far as magical hearing, I would again go back to the point that people perceive clear differences (even if they're not really there), so when other people say they don't hear them, it's natural to conclude that their own hearing ability is better (even if it's not). And of course, there is some variation in hearing ability, and I'm pretty sure that my own hearing ability has improved with effort and experience.

Regarding sports cars, a lot of us buy them because we like driving them (I've actually tracked every car in my garage, including the SUV and sedan). Status display can be a factor, but a lot of us don't like the attention that sports cars garner.

Maybe it's a cultural thing, but I never intended to indicate that people here were trying to project an image of affluence. That's not much of a thing where I live (there are always the isolated exceptions though). However, I am utterly aghast at the level of superstition and elitism found in what is, let's face it, a rather trivial hobby.
Don't get me wrong, I've been subject to delusion and expectation bias myself, but to attach a sense of self importance to it is utterly baffling. If anything, I questioned what the value of what I was getting. When a lot of the "evidence" of the benefits you'd be getting from an expensive piece of equipment is based on marketing and pseudoscience, you really have to question what's going on in there.
And as far as sports car go, sure, you do have car lovers, that's true and all respect to them because they appreciate the beauty of design and engineering. But when someone puts on racing gloves, aviator glasses and a polo shirt with a popped collar and a sweater tied around the neck, there's not mistaking that person is a wanker.