[RRod]

 
in wine tasting a certified Sommelier actually has to pass blind tests - no pass = no certification

 
When I saw that movie (Somm), that was the actually impressive part.

 

[landroni]

  In the two cases you mentioned it all comes down to a matter of taste, There is no absolute in Paintings or Wine, there are very many different styles or vintages that can be appreciated for themselves.
In audio we have a goal, an absolute, the recreation of the sound of real instruments. Its called High Fidelity for a reason and taste should not be considered when judging.
Of course it's your money and if you like pounding bass or screaming highly detailed and etched treble, that's cool too.  Just don't set it as the goalpost of achievement. To get High Fidelity we have to try and remove taste and bring scientific procedures to bear on the judging.

All nice and well. But what does science tell us? Measure the audio. Fine. Look at THD and flat FR. Check. Yet as Jason Stoddard tactfully puts it, this is little more than dickwaving.
 
For a crash course on how many headline specs advertised by manufacturers are irrelevant to actual audio fidelity, and how specs cannot be the ultimate judge of audio quality see this RMAF presentation:
https://www.youtube.com/watch?v=2V6YN-mshmY

 

[RRod]

 
All nice and well. But what does science tell us? Measure the audio. Fine. Look at THD and flat FR. Check. Yet as Jason Stoddard tactfully puts it, this is little more than dickwaving.
 
For a crash course on how many headline specs advertised by manufacturers are irrelevant to actual audio fidelity, and how specs cannot be the ultimate judge of audio quality see this RMAF presentation:
https://www.youtube.com/watch?v=2V6YN-mshmY

 
Science tells us to observe and then verify with experimentation. It's that last step that seems to be so hard for people to accept.
 
*edit:
Also note that science doesn't say "you can't know everything so throw your hands up in the air," which seems to be a common attitude around here.

 

[landroni]

 
 
Science tells us to observe and then verify with experimentation. It's that last step that seems to be so hard for people to accept.

I'm all for experimental verification done right.

 

[RRod]

 
I'm all for experimental verification done right.

 
We of course agree. But the point of recent comments has been that ABX does perfectly well for one whole big class of audio comparison (lossy vs. lossless), so we have no reason to think it has deficiencies for testing hi-res vs. Redbook simply because people can't pass tests. If people want a longer-termed test then fine, but as I stated previously it would be perfectly possible for people to answer an ABX trial at the end of multi-hour listening sessions. So it seems the issue isn't the protocol, it's just willingness.

 

[sonitus mirus]

 
I'm all for experimental verification done right.

 
In this situation, we can't be expected to prove that no differences exist, as you put it.  Hopefully that criteria is not part of the experimental verification.   If anyone truly cared enough, they might be hell-bent to prove there IS a difference.  Is there any better method to test for an audible difference than by isolating only our hearing to best of our ability?  I'll assume, for now, that nobody has been able to hear any differences, otherwise they would embrace the methodology and let the world see the results.  That does not mean that I am certain that there is no difference.  Again, I'm waiting.  I consider myself to be impartial and would like to follow the evidence to wherever it leads me.

 

[Sal1950]

 
All nice and well. But what does science tell us? Measure the audio. Fine. Look at THD and flat FR. Check. Yet as Jason Stoddard tactfully puts it, this is little more than dickwaving.
 
For a crash course on how many headline specs advertised by manufacturers are irrelevant to actual audio fidelity, and how specs cannot be the ultimate judge of audio quality see this RMAF presentation:
https://www.youtube.com/watch?v=2V6YN-mshmY

Who ever said they did?  NO ONE
You just made up a ridiculous statement to try a prove a nonsensical point.
But without measurement, science,  and the improvements they lead to, you'd still be listening to a Edison wax cylinder

 

[goodyfresh]

 
All nice and well. But what does science tell us? Measure the audio. Fine. Look at THD and flat FR. Check. Yet as Jason Stoddard tactfully puts it, this is little more than dickwaving.
 
For a crash course on how many headline specs advertised by manufacturers are irrelevant to actual audio fidelity, and how specs cannot be the ultimate judge of audio quality see this RMAF presentation:
https://www.youtube.com/watch?v=2V6YN-mshmY

Yes sure, if you're ONLY talking about THD and FR.  But any differences in fidelity that ARE audible HAVE to be measurable somehow, otherwise they are simply in our heads.  That should be obvious to anyone.  So if THD and FR alone can't cut it, we then need to also look at IMD.  If that too still doesn't cut it to account for an audible difference, we need to look at things like impulse-response and CSD waterfall plots.  And so-on.

 

[goodyfresh]

  the general opinion against short blind tests is that they stress people, that some stuff like fatigue aren't noticeable on short tests, and that it tends to lead to more null than other tests.
 
all seemingly valid points. but just seemingly:
 
-a blind test put stress on people so they don't hear what they would normally hear:
well that is true, and like anything else, stress will go down the more tests you do. first time I was on the driver's sit, I took the car into a wall right next to where I started(well I was around 12). I just pushed on the gas like my dad told, and panicked. but now time has passed I've done it soooo many times, and I don't piss myself anytime I drive a car. let's be reasonable here, I can accept people who would rather not know than bother. but those who do want to know, they just have to train a little and stop looking for false excuses.
I've done so many abx that now passing a blind test is like doing laundry. most certainly not fun, but nervous is not what I would call myself when I do it. bored might fit better.
 
 
-short stuff aren't good to test everything:
again it's true. so why not test what can with rapid switching, and then test the rest differently with longer periods of time? the answer is "because it's demanding", and usually needs someone doing the switching for us. but technically, there is really nothing stopping anybody, certainly not the blind test advocates.
have both DACs turned ON for a few days, and someone coming once or twice a day to change the output on foobar and make sure to match the loudness somewhere else if needed. there you go. don't go near the DACs(hide them), don't cheat looking at foobar settings and only use one resolution in case sample rate changing noises occurs on one DAC.  and you have your test. not impossible.
 
 
-blind tests lead to more null:
in practice we could say it like that. but it leads to more null than what? ^_^ the biased sighted evaluation? lol that's not even a test at all. the only ting being tested is if we are a little biased or very much biased. the gear we test doesn't not matter one bit.
 
people letting time pass to get a full idea of the gear? sure they find more differences, they would find more differences if the device hadn't change. there are endless experiments to do to demonstrate that for a fact.
that's because memories aren't accurate long enough to test audio in a very precise way. and when we're talking about something that failed to be noticed in a blind rapid switching test(the most effective audible method to test differences known to this day), we're obviously talking very very small differences.
listen to music for a few weeks on a DAC, science says that we have high accuracy for the last 3 to 10seconds. of course we remember a lot of the rest, but we remember it with errors. the more we will recall a particular detail, the more it will get the flanderization treatment by our brain. because it's our brain best known trick to remember important stuff. make them important!
so obviously we find less differences in short blind tests, because we don't give enough time to the brain to store the memory for good adding it's special sauce to it. we're asking to remember something still very fresh from 0.1second ago.
so yes, blind test leads to more null, simply because things really are less different than we imagine.
 
 
 
like most people here, I'm ok with any reasonable audibility testing method to replace blind testing. it's just that I'm still waiting for any alternative that isn't ludicrous. like "sit in a chair and just listen". that's fine to listen to music, but as a an effective testing method....

 
 
 
on many tests, there is a clear audibility threshold. when you can change bit depth, sample rate, loudness imbalance... there is a consistent area were things go from "I can't pass a test", to "I can pass a test". I'm tempted to think that it demonstrates the test works just fine.
my opinion is that people who think they heard a difference but fail a blind test for those differences, either imagined it, or used a wrong way to "evaluate" the differences in the first place. I am yet to be put in presence of someone that wasn't clearly fitting into one of those 2 profiles. all the others passed a blind test for the thing they claimed they heard.
can you hear 96kps mp3 vs flac, sure I can most of the time.
can you notice the noise floor on the sony A15? yes I can as long as I have my sensitive IEMs and I'm not in the subway.
all those claims of audibility have one thing in common, I can pass a blind test. that's what justifies me into making a claim. the darn test!
 
 
to me, the rest are excuses born from failure to admit to being wrong. they're not claims of audibility, they're claims of self righteousness as they have absolutely nothing to show.
 
 
 
if I believed in magic, I would cast creepy spells onto all people making empty claims. empty claim is the lie of the ignorant. I hate it.

Yes, yes YES about the stuff about human memory.  One thing that REALLYYYYYYY pisses me off in this forum is people CONSTANTLY making comparisons between stuff purely from memory alone, without adding the disclaimer that it is purely from memory and thus subject to bias.  Human memory has been shown time and time again to be as unreliable as anything can ever get.

   
 
it seems clear the desired implication is that 44k isn't enough and that this anecdote is intended to lead you to that conclusion
 

Bias, bias, bias.  People have already paid a bunch of money for 24/96 tracks from HD Tracks or wherever, so they are already mentally invested in the idea that Hi-Res MUST be better!
 

  I have spent a whole lot of time trying to blind A/B identify closely volume matched Bifrost Multibit & my ODAC (many hours per day for days) with my HD800. I couldn't ever get better than random. It could also mean I just have really bad ears. Either way it put an end to my DAC upgrade craving.
 

Neat!  I wonder if other people would be able to identify a difference or not.  It certainly seems plausible that people with Golden Ears could pass such a test, given that we're talking about R2R vs. DS that is not even close to "summit-fi" level or anything like that, but rather square in mid-fi territory.

 

[castleofargh]

 

  Quick, back-and-forth switching works when testing for differences between 96kbps mp3 and a lossless version, but not between a 320kbps mp3 and a lossless version?  Is that what I am supposed to believe?  If identifying difference is too tiring and difficult, at what point do we concede that any differences are not significant enough to be concerned about?

If a test is flawed for failing to take into account an important factor, it is flawed. Period. Such issues can invalidate decades of scientific research, which is fine as this means scientific progress. I am not saying that it is the case here, but it could be. If audio scientists are hell bent to prove that there is "no difference" (which, for the record, they can't), and previous testing yielding a null result failed to take into account ALL potential confounding factors (including newly raised questions, that weren't originally considered), then the burden is on them to redo the work to retrieve yet again a null result when controlling for ALL factors.
 
And how can a test concede that a difference is "not significant enough to be concerned about" for a given individual? Always remember that tests and scientists tend to be concerned with averages.
 
Now take a deep breath and consider two similar instances:
- hi-end paintings. For a regular Joe the difference between a Rembrandt and a Van Gogh is not existent. They can look at one, look at the other, and concede they don't care. An art "subjectivist" (i.e. an acclaimed art critic) would BEG TO DISAGREE. Surely, it may take hours and weeks to appreciate a painting, and decide what in particular you like about it, and how particularly it is different from the other painting. It doesn't mean there is no difference. It simply means that it takes time, will, training, attention to appreciate artistic rendering, of which musical reproduction is. In this sense, 10 sec ABX testing are laughable --- may as well use 0.5 sec and be done with it, and prove whichever null you're seeking. Imagine if you did ABX testing on Rembrandt paintings using a 1% sample area of the paintings... And do consider that art critics are being paid handsomely and admired for being able to notice what others can't, instead of being rudely dismissed as ignorant art fools.
 
- hi-end wines. For regular Joe the difference between two fine wines going in the thousands would be impossible to assess. Both wines would be very good. Yet for a wine "subjectivist" (i.e. a wine taster with a rarefied pallet), the difference between two wines could be LIGHT AND DAY. One would be undrinkable, and make you physically cringe. The other would be heaven on earth. For that person. The difference is. Huge. It doesn't matter if others can't sense it, or if scientists can't prove it in a blind test. There, too, those with a fine taste are appreciated instead of being ridiculed.
 
So what is it in audio that generates such boatloads of scorn and angst? Subtle differences can be difficult to detect, and not by everyone, but just because it's subtle it doesn't mean it doesn't exist, or that it's emphatically prescribed as "doesn't matter" or "not huge". With humans, something that might be altogether irrelevant for average Joe might be a handicapping or life-or-death factor for a given individual. (I speak from non-audio experience.)

but how do you decide a test is flawed when you actually have no other test to validate that claim?
you pretend to be on the rational side of things, but you really show all the bad stuff. I have a feeling that I heard a difference, I go with the hypothesis that it is the DAC (as this topic supposedly deals with that). so what will I do, I will test both DACs with the same tracks time aligned volume matched and a switch. if I still feel that I heard a difference, then I will ask someone to help me do a blind test. if I pass that test, I will conclude that I indeed heard a difference. 
now another controlled test showing otherwise will ask the question "which test was flawed? and why?". but in this case you carefully avoid mentioning what is the other test, and we all know why. because claims of audible differences that fail a blind test come 100% of the time from uncontrolled sighted evaluation. which is biased and has been proved to be so many times for many reasons.
so you want to justify the flaws of bind test by using the results you get from the worst subjective test method known to man after guessing. I fail to see the science behind this.
 
so again, give us an audible test that has meaning and controls. and if the results contradict actual blind tests, then and only then, we'll look into using both tests depending on what we look for.
blind test isn't perfect, but I won't accept crap methods as arguments against a method that works way better and has been demonstrated to do so thousands of times.
give me an actual testing method for audible differences that outperforms rapid switching in a blind test. then we'll talk.
 
 
 
 
the painting and wine are pure and simple fallacies. I would love to see less of those in your future posts. in both cases, almost anybody is able to tell the 2 paintings are different even if they never heard of the guys, and that 2 wines taste different even if they never had wine before. you're sliding out of your honesty chair, careful not to fall.
 
 
 
 

  All nice and well. But what does science tell us? Measure the audio. Fine. Look at THD and flat FR. Check. Yet as Jason Stoddard tactfully puts it, this is little more than dickwaving.
 
For a crash course on how many headline specs advertised by manufacturers are irrelevant to actual audio fidelity, and how specs cannot be the ultimate judge of audio quality see this RMAF presentation:
https://www.youtube.com/watch?v=2V6YN-mshmY

what does that have to do with anything? I thought we were talking about audibility?

 

[watchnerd]

My latest studio monitors have an A/D converter (sigma-delta based) and a DSP-based crossover.  This is becoming pretty common amongst the newest active studio monitors.
 
What's the point of fetishizing over R2R DACs if it's going to get converted to via sigma-delta in the crossover, anyway?

 

[Osirison]

Hello Head-Fi-rs,
 
I'm new but have been reading the Head-Fi forums for some time now.
 
Since a year I have moved from transistor to tube amplifier's first balanced EL34 then single ended KT88 but listening on it there was some digitized sound and overly sharp S tones that was bothering me that I did not notice before on the Classe transistor amp and came to the conclusion it must be the DAC which was a Musical Fidelity m1dac.
Before just buying a new DAC I did some research on the basics of conversion and came to the conclusion that binary-weighted or R-2R are actually a true way of decoding digits instead the alghortimic delta-sigma type and bought a PCM56 based R-2R dac which does not oversample or utilize a digital filter and it has a tube output stage.
With this DAC in place I now have not a single transistor in the audiopath from the output of the PCM56 chip.
 
Sounds very natural but not its not a magic device of course, the problem is that since about '85 recording studios began implementing more digital equipment and also sigma-delta/delta-sigma converters.
This also means that your current vinyl production is probably not true analog anymore.
 
Right now I enjoy listening to older Decca recordings from the Analogue Years box these old tape recordings where digitized around 1982 and are "captured" with R-2R AD converters.
Instruments sound more real and not harsh compared to similar music from the digital Deutsche Grammophon recordings which are good recordings.
Luckily labels like Tacet are into recording amazing sound without the use of sigma-delta, their "tube only" recordings sound beautiful.
 
To me delta-sigma is only an era in time just like late 90's 2000's where people started to move to digital cameras using not so great image sensors, low resolution, artificial color saturation, but it was the best we could make at the time, just like delta-sigma is now.

 

[spruce music]

  Hello Head-Fi-rs,
 
I'm new but have been reading the Head-Fi forums for some time now.
 
Since a year I have moved from transistor to tube amplifier's first balanced EL34 then single ended KT88 but listening on it there was some digitized sound and overly sharp S tones that was bothering me that I did not notice before on the Classe transistor amp and came to the conclusion it must be the DAC which was a Musical Fidelity m1dac.
Before just buying a new DAC I did some research on the basics of conversion and came to the conclusion that binary-weighted or R-2R are actually a true way of decoding digits instead the alghortimic delta-sigma type and bought a PCM56 based R-2R dac which does not oversample or utilize a digital filter and it has a tube output stage.
With this DAC in place I now have not a single transistor in the audiopath from the output of the PCM56 chip.
 
Sounds very natural but not its not a magic device of course, the problem is that since about '85 recording studios began implementing more digital equipment and also sigma-delta/delta-sigma converters.
This also means that your current vinyl production is probably not true analog anymore.
 
Right now I enjoy listening to older Decca recordings from the Analogue Years box these old tape recordings where digitized around 1982 and are "captured" with R-2R AD converters.
Instruments sound more real and not harsh compared to similar music from the digital Deutsche Grammophon recordings which are good recordings.
Luckily labels like Tacet are into recording amazing sound without the use of sigma-delta, their "tube only" recordings sound beautiful.
 
To me delta-sigma is only an era in time just like late 90's 2000's where people started to move to digital cameras using not so great image sensors, low resolution, artificial color saturation, but it was the best we could make at the time, just like delta-sigma is now.

 
 
Well I remember all the big news when people went to delta-sigma DACs.  How they sounded so much more refined, smoother and had much better measurable performance than the old R2R gear.  There are some big problems in making accurate R2R gear that delta-sigma makes a non-issue.  So saying we once had good gear and now all we can get is sigma-delta is something of a fairy tale.  Sigma Delta outperforms R2R DACs in a measurable way.  Low level linearity is better, noise levels are better, 24 bit performance is closer to being realized, aliasing is better controlled, sigma delta is simply a better performing method.  R2R DACs can only come close to matching that performance at great expense, and don't do anything better. 
 
Not trying to rain on your parade, but it is so funny.  All the same exact platitudes about how this is a more analog sounding, smoother, more correct way to do things has been used twice.  Once in the switch away from problematic R2R and now whats old is new again as the same spiel only in reverse is used to tell you sigma-delta isn't pure and you need to go R2R. 

 

[castleofargh]

pulse modulated stuff has a lower theoretical resolution limit than R2R. but pulse modulated DAC achieves pretty much in reality what it can do on paper.
R2R on paper is king of the hill. in practice I think it usually has more distortions than most delta sigma because, go get X resistors that are exactly the same value with the same behavior... not that easy to do in the real world.
 
I believe Osi(welcome to headfi) is dismissing a great many changes that also occurred as the years passed that have nothing to do with delta sigma. recording processes changes, techs changed, mastering engineers had to get used to different medias needing different ways to master, and of course computers brought a all new list of digital DSPs to fool around with(just like almost everybody has some matter of vocoder applied to the voice...). and last but not least, the stupid loudness war that ruined thousands of albums so that they would be loud on the radio. so it's pretty hard to blame it on pulse modulation IMO. could be one of the cause, just like both DAC could actually sound just the same given how well they can measure compared to everything else in the audio system.

 

[Max Choiral]

Guys, you're going over and over the same thing for decades. Just make some comparisons with measurements (made by yourself) and explanation already, otherwise this discussion leads nowhere