Like the wise man once said - "discuss the opinion, not the person." I've owned, heard, used, demo'd, been loaned, worked on, played with, etc my fair share of gear at all ends of the price and performance spectrum (from cheap as chips to $texas and back again), both studio and consumer, and stand by what I said. In my experience (that's what "IME" means if it wasn't clear - I'm learning there's apparently a whole new generation of "Internet slang" these days, and apparently I've become "an old fart" without even noticing), there is not a *significant* difference between DACs, if one is even noticeable at all. This does not mean "they all sound the same" but I have never (and this point expect to never) heard the "Jesus singing" kind of massive night and day differences that some audio rags laud every latest DAC (or "digital device" - I'm *really* done with the "end of digititis" being proclaimed every 6 months) with bringing about, and in some cases there's nothing at all to get up in arms about. Beyond that, IME, if the two DACs (or CD players, or whatever) aren't being directly A/B'd in a sighted and non-matched environment, I frankly don't find the differences worth getting super excited about in most cases.* Again I'm not saying "its all exactly the same 100%" but I am saying "if there are differences, they're generally very slight, and if I'm not critically A/B comparing two devices, those slight differences can easily fade into the background/be of no consequence." And I think that absolutely gets to this thread's main question: so what's good material to even uncover those differences, if they are even there? And I responded to that back a few posts, and will repeat that here too: music that you're familiar with, that's high quality (as in, high bitrate lossy or lossless files; not something that's been mangled by 64k ATRAC just because it lets you get about 200 tracks on a CD), and for my personal taste, something that is heavily vocal focused (both because I tend to listen to that kind of music more often and because I believe, factually or not, that our ability to easily identify and process human voices makes it easier to hear defects in playback or recording of voices as a result). I would absolutely go as far as saying the source material itself probably matters more than the DAC or transport (or whatever) because the audible differences between, for example, the 64k ATRAC and a lossless flac rip of the same track will be significantly greater than the differences between two CD players or DACs or whatever else, all else being equal.
* Free example: I own what is, to my understanding, the most flagshippy flagship HD-DVD player ever produced, the Integra DHS-8.8, which is a rebadged (and if you believe some of what you read online, slightly souped up) Toshiba XA2 (and it could be a straight rebadge for all I care - it plays my HD-DVD collection and that makes me a happy camper), and it is frankly a fine sounding CD player in its own right. I also own a similar age, and similarly priced (when new) Blu-ray player (a Yamaha BD-S1900), which is also frankly a fine sounding CD player in *its* own right. They have different DACs from different manufacturers (Wolfson and AKM, if I'm not mistaken), different transport mechanisms, and don't even play all the same kinds of discs - point is they're pretty different machines inside and out that happen to have some format commonality. I can happily enjoy CDs equally on both of them, and think both of them sound pretty darned good (and at roughly US $1000/ea when new, they better), to the point that I could (and have) happily live with either of them as a CD player in a system. Out of curiosity, because they happen to be in the same system/rack now, I decided to hook both of them up with a toggle, grab a Sinatra album, and see what they're like "side by side" - and wouldn't you know, I'd swear there are very slight differences between them. Specifically the Integra sounds a bit colder and a bit dryer. But in general they both very much conform to the "low noise, low distortion, high fidelity" archetype (and again, at roughly US $1000/ea when new, they better), and if they aren't locked side-by-side with quick switching between, those slight differences really melt away to nothing pretty quickly, and its really just "wow, this sounds pretty good - this machine is doing a nice job with this CD." I'm not sitting there going "gah, I wonder what this record sounds like on the Yamaha - would it sound more engaging and warm?" or "wow the Integra would be a better match with these cans, it'd really balance them out!" These are not "life changing differences" in my book - the difference between that Sinatra album from the CD (or lossless rip thereof) vs compressing it down to 128k would be genuinely a bigger difference; I say this to help provide further context. It is really that simple in my view - they're both quality machines and both do a good job at what they do. And that's how I've felt about most DACs and ADCs I've ever heard, at basically any price point (which goes into the five figure range and probably beyond), both pro and consumer. I would agree with the idea that extremely one-sided comparisons, like lets take either of my movie players and put it up against an el-cheapo portable CD player, the movie player should do a better job (and just from having owned my share of el-cheapo portable CD players over the years, they generally aren't great - but for $29.95 what do you really expect?). That said, I'd say with DACs more than any other device (except maybe cables or power products) its really easy to hit the point of diminishing returns, especially with modern gear. And it makes sense - most DAC designs target basically the same performance standards: low noise, low distortion, flat frequency response, and filtering that eliminates artefacts rather than cause them. As far as "pure transports" go, I'm even more skeptical, simply because there's not much for a transport to screw up - especially if it isn't a disc spinner and/or trying to run some DSP/EQ/whatever (which in a lot of cases, especially historically, usually caused more problems than it fixed). Feeding garbage into it is going to be a significantly bigger problem than anything it could likely influence itself, which again goes back to "you really should pick quality stuff that you're familiar with" if you're going to attempt a comparison. But again, these are (IME) small differences that don't make much difference outside of very nitpicky comparisons or audio magazines. The best analogy I can give here is that of a water hose - the "overall system quality" is like the water flow, and if you kink it off way up at the beginning by the spigot, it doesn't matter if it opens up to a bigger diameter with fancy teflon coating for decreased resistance later on - you've already choked it out before that stuff can make any difference.
I'll add that I'm not really trying to "convert" anyone to my way (or anyone else's way) of thinking - I didn't arrive at this conclusion by "theorizing" about anything. Sure, I've read books and studied a lot of stuff that relates to the above, but frankly I've arrived at this conclusion through experience more than anything else (although there are plenty of books and articles that support the same conclusion, and have good reasons for doing so), and that took years. And before anyone screams "expectation bias!" - I "went into it" absolutely expecting that the fancier and better the gear got, the better it not only could, but should, sound, and was instead somewhat disappointed once the genie was out of the bottle and the realization that frankly significantly diminishing returns kick in way before most folks are priced out of the market.
There are, however, other good reasons to own "nicer" gear (which have nothing to do with sonics) and frankly I wish "audiophiles" could just honestly accept them, things like:
- Better build quality.
- Better warranties.
- Better end-user support.
- More/better non-music functionality/user experience (things like backlit remotes, quality displays, non-retina-burning LEDs, smartphone apps that work, network functionality that works, good user manuals, good ergonomics, etc)
- Pride of ownership.
It's like, nobody who buys a Rolls-Royce goes and screams at Honda owners how much of a better/more serious driver they are for owning a Rolls-Royce based on about how much more efficient, safe, versatile, high performance, whatever their Rolls-Royce is, nor do we expect the Honda owners to go out and gang up on a Rolls-Royce owner and tell him/her what a rube they are for having bought the Rolls-Royce; but we see that exact back-and-forth play out in the world of "audiophilia" on a regular basis. I do truly wish it would stop.