Relying solely on anecdotal evidence is silly as well, otherwise we'd be content with the geocentric model of the world to this day. As they say - don't throw the baby out with the bathwater.
Nah, it's silly to claim that any kind of "scientific" evidence is necessary before choosing one DAC over another (or amp, or what have you). And comparing this to the geocentric model is way off: science has zero interest in things like "is there an objectively perceivable and scientifically significant difference between ODAC and Yggdrasil"; if you think otherwise, it's possible you're confused what science really is about. This question is of interest only within the context of the wild "objectivist/subjectivist" debates, not otherwise.
And relying "solely on anecdotal evidence" is what we do every day for survival, e.g. when crossing a street (have you ever DBT'ed yourself that you can reliably tell red from green lights, in different cities/countries?) or when having a stroke (do you dismiss strange, wild symptoms as placebo and imaginary
because they're subjective and anecdotal, or do you reach for your phone to dial 911?). Again, anecdotal evidence is integral to the evolutionary process and thus for something as critical as survival, so I can't see any reason why anecdotal evidence wouldn't be sufficient for something as non-critical as choosing audio gear and listening to it.
And yeah, there is a lot of science involved into competitive gaming and gardening. If you're serious about it.
Of course not. When comparing DACs we're really talking about subjective perceptions and enjoyment of one specific gear over another, when looking for a pastime.
So, in
gardening, how often have you DBT'ed two different plants, to see if there is a reliably perceptible and scientific difference between the two when you tend over them or when you watch over your garden? Or in
stamp collecting, is there really any evidence out there, and I mean published peer-reviewed scientific evidence, that two different stamps will generate different perceptions in a philatelist? Or is it all philatephoolery? And in
gaming, have you ever DBT'ed Halo vs Sims, to see if you can perceive ANY difference between the two? Or in
programming languages --- surely there are differences between Python and C, but have there ever been formal DBT that show that programmers can perceive any differences between them? Or when eating
ice-cream, do you doubt your perceptions of a difference between two flavors until you're presented with formal proof from a carefully set up DBT that controls for all factors, including the level-matched temperature of the said ice-creams, on an infinite sample size, published in Nature; and will you just assume that all ice-creams taste the same until scientific evidence emerges from carefully controlled DBT'ed studies? But there are many hobbies out there involving gear, so have you ever gone about DBT'ing ski gear, tennis rackets, foot balls, golf balls, diving gear, TV sets, digital cameras, cars, planes, parachutes, keyboards, game consoles?
But let's not stray from audio and let's do
reductio ad absurdum right here. Is there really ANY 'audible' difference between an Elac UB5, an iPhone ear bud and a Stax 009? I mean, objective, scientific proof from carefully set-up
double-blind tested studies with matched levels, not subject to the much maligned "expectation bias"? And I insist, for any evidence to have any validity both the researchers and the test subjects must NOT know, at any point during the testing, which piece of equipment they're listening to. (Good luck with that, of course, but it's a fun thought experiment.) Of course, you can point to all the measurements you'd like, but "is it audible?!" Until any such peer-reviewed, scientific evidence emerges, I say we can safely assume that there is no difference between these three pieces of audio gear and all those who claim to hear a difference are hapless audiophools gullible to their expectation biases and being abused by mean and greedy audio corporations.
The thing is, this obsession with "scientific proof" in some circles is quite unique to high-end audio, and the real question of interest here is why? I suspect part of the answer is that in audio there are things that we
can DBT, whereas in many other hobbies DBT'ing would be a futile endeavour at best.