Testing claims about the sound of different DACs
Jun 12, 2015 at 12:53 AM Post #2 of 300
none that I would accept - not without controls, blinding - the 1st rule of sighted listening is that you will convince yourself you hear differences - simply the way human brains are wired
 
false correlations are generally a cheap price to pay for the true inferences from experience that helps us survive long enough to breed
 
as to the dominant "audiophile culture" of "subjective impressions" its not even accepted that you have to have the DACs in the same room or system at the same time in order to make comments here - many reports here are "by memory" "comparisons"
 
such "soft" reportage may be a place to get ideas for controlled tests - but are not much more to me until you get positive results in controlled tests
 
you all should know rather sophisticated versions of "the objectivist" position by now regardless of your own interpretation of your personal listening "story" - your experience are your own
 
most "objectivists" are quite happy to learn of new human listening results, modify positions based on scientifically acceptable evidence
 
personally, here in "Sound Science", I badgered Ethan Winer who writes on Audio to try listening for phase shift/"polarity" in test waveforms - and he changed his former "denial" position when he could get positive ABX results
 
 
why is this a bigger deal for the forums than accepting people do differ on when/if they say a piece of equipment has or hasn't got digititus, glare, detail, prat, plankton (I mean really guys: prat, plankton??)
 
 
as an engineer I can appreciate, probably more deeply than many here, the "conceptual art" involved in something like the Yggy, making the AD5791 work as a audio DAC
 
I doubt I will ever compare many of these "high end" pieces, certainly wouldn't put any of my personal uncontrolled personal listening experience out there one way or the other expecting any to believe
 
but I am quite interested in both the tech and psychoacoustics of music reproduction
 
I'm quite willing to accept that there are genetic, developmentally and training gifted people who do hear things I never will - when they show up for and pass well designed, well executed, controlled listening tests with positive results
 
Jun 12, 2015 at 12:56 AM Post #3 of 300
  none that I would accept - not without controls, blinding - the 1st rule of sighted listening is that you will convince yourself you hear differences - simply the way human brains are wired
 

 
To me this says you're claiming all the listening reports of differences are the result of placebo. That claim itself, if its to be within science needs to be falsifiable. How are you going to test the claim?
 
Jun 12, 2015 at 1:10 AM Post #4 of 300
you can argue it in the Sound Science forum - I'm not doing that here
 
I'm trying to contribute my "EE impressions" where I feel it is responsive to the thread's topics flow and development
 
 
and the big ah, guy (yes that 's the 1st word following big that entered my head) at Schiit:
  2015, Chapter 10:
Knowing Our Place?
 
...
  • Meets that challenge you. You know, I’ve been thinking a lot about the whole objective/subjective dichotomy we have in audio lately, and I think it may be time to create a new kind of test. Not ABX with unfamiliar music, but something much more interesting—letting people choose their own music and switch between two different signal chains when they wanted (both level-matched and with distortion lower than audibility, you know, typical “good test results.” The goal would be to see if (a) a consistent majority preferred one signal chain over another, and (b) to see how many people could consistently tell the difference. Yes, I know, this kind of test will never satisfy the ABX folks, but I think it might get us a little closer to the truth about whether or not there are some people who consistently hear audible differences in systems that should sound the same. I suspect the answer is neither 100% subjective nor 100% objective, but a continuum…some people can hear the difference, some can’t, some care, some don’t…and all of that is fine. Of course, this is a big undertaking, but…hmmm…I need to think about this some more, I kinda just write stuff as it comes to me.
 
So, am I crazy or what? (Yes, I know, you’re probably laughing and nodding your head.) But I will say one thing: the past years I’ve spent in headphone audio has been more fun than all the years of two-channel, home theater, and video servers put together. There’s still a fresh, raw edge to the headphone side that I really don’t want to see plowed under in the ongoing rush to unobtanium pricing.
 
I’m hoping we can help keep the headphone side on its toes.
 
(I’m sure you’ll let me know if we don’t.)
 
And with that, I’ll sign off…with one last reminder…that as of June 15, we’ll be celebrating exactly 5 years of being “shock over substance,” “a flash in the pan” and “flavor of the month.”
 
Here’s to the next 5 years!

 
Jun 12, 2015 at 1:17 AM Post #5 of 300
Having been teleported - as if by magic - over to Sound Science, I'll ask again. How are you going to test your claim that all listening reports are not admissable as evidence due to placebo effects?
 
Jun 12, 2015 at 1:26 AM Post #6 of 300
^ I would certainly be interested to hear this question taken seriously and answered seriously. It merits careful thought I think...
 
Jun 12, 2015 at 1:32 AM Post #7 of 300
 ...your claim that all listening reports are not admissable as evidence due to placebo effects?

I didn't make that claim - there are in fact many reasons uncontrolled listening "impressions" by the self identifying "audiophile" "subjectivists" are not actionable evidence to me
 
people really do hear differences when level, frequency response aren't matched - and do so with positive ABX results
 
hearing has multiple internal "inputs"
 
focus - listening for one thing can make it harder to hear another
 
training and accommodation - repeating a test, relistening you can hear different features, can learn fairly quickly to make new discriminations, learn to ignore or compensate for say moderate smooth frequency response differences
 
longer term accommodation, physiologic changes with weather, health, ect.,  the few bits of information that get encoded in longer term memory, the demonstrated plasticity of memory lead me to even more heavily discount "I heard xx at a meet.." "comparisons" to something heard elsewhere, elsewhen
 
 
"expectation" is more accurate than "placebo" anyway - and its only one part of the well explored effects in Psychoacoustics, Perceptual Psychology, sensory testing Scientific fields
 
Jun 12, 2015 at 1:40 AM Post #8 of 300
So it seems you've not answered. You'd like it rephrased with 'expectation' (or even 'confirmation') instead? I'm quite aware of confounders but I'm not interested in listening tests. What I'd like to know is how you're going to test your claims that in every case of a listening report, the description of the qualities of the sound can be discounted due to the confounders you've mentioned.
 
Jun 12, 2015 at 1:48 AM Post #9 of 300
I don't need to disprove anything, particularly not "your claims that in every case..." - when again that's your strawman, not my argument
 
your hobby of philosophy may deepened, improved by understanding the differences between good rhetoric and your version
 
I'm quite impressed if  English is a 2nd language for you - but not by the logic
 
Jun 12, 2015 at 1:51 AM Post #10 of 300
I infer your claim from your behaviour - you reject evidence that does not conform to your particular standards. So on what grounds please? Do please explain - class actions don't count in science as far as I'm aware.
 
Ad hominems are pointless, suggest you cease to engage in them.
 
Jun 12, 2015 at 2:03 AM Post #11 of 300
will you offer the return courtesy of quit proposing I must debunk your strawman constructions
 
my claiming your rhetoric is bad is something that is in fact scored in formal debates, methods of argument, rhetoric can be used to seek understanding, or as you are using
 
 
I did just give a short list that could be mined for search terms to see what I am alluding to as reasons why I don't think many "testimonials" here are objectively reliable, transferable to others, repeatable experiences
 
Jun 12, 2015 at 2:09 AM Post #13 of 300
Since when did "description of the qualities of the sound" constitute evidence in any sense of the word, let alone science. You'd have more chance based on results from an ABX test with multiple applicants but that's it for use of subjectivity.
 
Jun 12, 2015 at 5:55 AM Post #14 of 300
  Having been teleported - as if by magic - over to Sound Science, I'll ask again. How are you going to test your claim that all listening reports are not admissable as evidence due to placebo effects?

 
No need to test accepted scientific findings, eh?
 
These are the top irrefutable scientific reasons why listening reports are not admissible as evidence due to their inherent serious flaws:
 
(1) Listening reports are not admissible because they are not tests. That is, they do not involve comparison to a fixed, reliable standard.
 
(2) Listening reports are not admissible because they involve excessively long switchover times, which makes them highly susceptible to false negatives because they desensitize the listeners.
 
(3) Listening reports are not admissible because the do not involve proper level matching, which makes them highly susceptible to false positives because people report the level mismatches as sonic differences.
 
(4) Listening reports are not admissible because they do not involve listening to the identical same piece of music or drama within a few milliseconds, creating false positives because people report the mismatched music as sonic differences in the equipment.
 
(5) Listening reports are not admissible because they constantly reveal the true identity of the UUTs to the listener, creating false positives because people report their prejudices and preconceived notions as sonic properties of the equipment.
 
Jun 12, 2015 at 6:44 AM Post #15 of 300
... I'm quite willing to accept that there are genetic, developmentally and training gifted people who do hear things I never will - when they show up for and pass well designed, well executed, controlled listening tests with positive results

 
 
 
  none that I would accept - not without controls, blinding - the 1st rule of sighted listening is that you will convince yourself you hear differences - simply the way human brains are wired
 

 
To me this says you're claiming all the listening reports of differences are the result of placebo. That claim itself, if its to be within science needs to be falsifiable. How are you going to test the claim?

you are indeed making a strawman argument.  look at what I quoted from him, then look at what you yourself quoted, you're the one making it a "all" the listening reports are result of placebo. he didn't say that. he said that he didn't put his faith in uncontrolled tests, not that they were all false. that's a pretty clear difference to me.
please argue about what he said, not about what you feel like he meant deep down secretly.
 
now about DACs, most measure amazing, and most other parts of the audio chain have lower measurement fidelity. so if you believe you own a transparent amp or a transparent headphone/pair of speakers, then it kind of implies that most DACs are also transparent as they measure better for most specs. it's not a proof, just a rational comparison.if you don't feel like any amp or headphone are ever transparent, then maybe it's worth looking into DAC audible differences too.
 
-if the source makes a difference, then it's a problem with the source, so I wouldn't put it on the DAC sounding different. and because of how DAC and amps are linked(impedance bridging) the analog section of a DAC shouldn't have much trouble delivering a very clean signal.
-then if the amp itself behaves differently, it might be because of the voltage output of the DAC being different or stuff like that, but then on a good amp you would expect to find a gain setting to adapt optimally to the DAC's voltage, if that isn't on the amp, then isn't the amp the one to blame for sound difference?
 
 
once that is put out of the way, we still need to make sure whatever we test is volume matched. else the test itself is a waste of time as we can't trust the results. you may of may not believe how important volume matching is, but I feel there is a consensus on the subject, we do perceive louder music as better music(until it's just annoyingly loud music). I think starting 0.2db variation will make a difference. when you read about DACs, they go from 2V to slightly above 3V, and portable stuff can go as low as 0.4V on my sony. if we just take a DAC with 2V and one with 3V, that's more than 3db louder on the 3V one. so denying the need to volume match and testing without doing it precisely, that is enough to make a test worthless.
so, if only because all DAC won't have the same impedance and same voltage output, that would make it a necessity to control the test at least for loudness. and that makes me agree entirely with all of jcx's first post. it doesn't mean that some DAC will not happen to have the same maximum output and impedance and whatever, and that the guy doing an uncontrolled test can't be right about his impressions. but it does mean that we shouldn't trust his results unless he makes sure there was no need to volume match. and then sighted evaluation is another problem, so yeah, uncontrolled tests, that's not something I will ever trust.
 

Users who are viewing this thread

Back
Top