[analogsurviver]

Back or not, I'm ignoring him. I have zero interest of what he says, because he has zero interest of learning anything.

I only chimed in to see if the most/all of the gang is alive and well - in these crazy times.

There is at least one Finn that could have learned a thing or two by not ignoring me - the music, sound, recorded sound and the life in general did not start in digital - which is only a sampling of.

Digital does have its merits, but it is not "be all, end all" by any stretch of imagination.

 

[bigshot]

I don't think it's possible to scrape off the barnacles. When they get the attention they crave, they'll go away again. I avoid replying with a quote because it's just "invoking the demon". And responding to points only encourages them. Best to just speak past them. I need to take my own advice more often.

 

[dogrelata]

I don't think it's possible to scrape off the barnacles. When they get the attention they crave, they'll go away again. I avoid replying with a quote because it's just "invoking the demon". And responding to points only encourages them. Best to just speak past them. I need to take my own advice more often.

I have to start by confessing I’ve only read the last 25 pages or so, plus parts of the OP, so apologies if what I’m about to ask has already been covered.

The question is about ABX blind testing. I’m curious if control tests are ever carried out, i.e. tests with comparable pieces of equipment that have measurably different outputs to determine how participants fare in being able to differentiate between said pieces of equipment?

 

[bigshot]

dogrelata, See the link in my sig file labelled... Stereo Review January 1987 Pg 78 You will need to open the PDF to page 78.

There are established thresholds of audibility, known as the JDD or "just detectable difference" for the various measurements... signal to noise, response, distortion, timing error, etc. There are variations in how the error is created and measured, but you can look at specs and get a ballpark idea if something is audible or not. Most of the things that audiophiles point to (HD Audio, jitter, etc.) are more than an order of magnitude below the JDD.

 

[danadam]

There are established thresholds of audibility, known as the JDD or "just detectable difference" for the various measurements

Google claims it is "just noticeable difference", JND

 

[castleofargh]

I have to start by confessing I’ve only read the last 25 pages or so, plus parts of the OP, so apologies if what I’m about to ask has already been covered.

The question is about ABX blind testing. I’m curious if control tests are ever carried out, i.e. tests with comparable pieces of equipment that have measurably different outputs to determine how participants fare in being able to differentiate between said pieces of equipment?

ABX is just one method(favored not because it's great, but because it's relatively simple and available to many). When actual researchers are concerned with what you're asking, they always go for different tests. Like maybe tests with known audible samples added to the series. Those will serve to remove participants who are trying to fail on purpose, or just those who have real hearing issues, making them unfit for that experiment.

Don't hold you breath for big serious scientific research about DACs sounding different, or similar gear questions. The guys who can do it, don't give a crap about such questions. And the average audiophile won't bother with anything even slightly inconvenient. Not the facts I like, but the facts I get anyway.

 

[bigshot]

Just do the tests yourself. It isn’t hard to do a simple blind listening test. People make it out to be a lot more complicated than it is.

 

[bfreedma]

Just do the tests yourself. It isn’t hard to do a simple blind listening test. People make it out to be a lot more complicated than it is.

Carrying out a valid DBT isn't really as simple as you're making it out to be.

At minimum, you need measuring devices necessary to properly level match and the skill to do that properly. And, assuming this is at home and not in a lab with automated switch gear, at least one more person willing to control the DBT. That person needs to be willing to execute the switching for a period of time long enough for the test subject to execute enough test cycles to be statistically significant.

Certainly not an impossibly high bar, but also not something one person is going to properly undertake in an hour.

 

[bigshot]

A DBT for peer reviewed publication is the same thing as a DBT for personal research, just to a different degree of precision. I keep explaining this over and over, but everyone keeps making it an "all or nothing" thing. Blind testing is a useful tool for eliminating bias whether you have a lab full of equipment or if you are just a person trying to optimize the stereo in your living room. It's still a double blind test, and it still helps eliminate bias.

Level matching can be done simply to a degree that can discern if a difference is big enough to worry about. Just run a tone through both sources and switch back and forth and adjust until you can't hear a difference any more. That works fine. If you can't hear a difference with a steady tone, there is no way it's going to make a difference when you play music. And a friend will be patient enough to help you do enough tests to know whether you should worry about it or not. You don't need to calculate pi to the 100th decimal point. There is such a thing as "good enough for government work". If you can't do it in an hour, take two hours.

Test for yourself. Don't throw up your hands and give up because you don't have a degree in engineering. Critical thinking and testing is for EVERYONE, not just people in white lab coats.

 

[dogrelata]

dogrelata, See the link in my sig file labelled... Stereo Review January 1987 Pg 78 You will need to open the PDF to page 78.

There are established thresholds of audibility, known as the JDD or "just detectable difference" for the various measurements... signal to noise, response, distortion, timing error, etc. There are variations in how the error is created and measured, but you can look at specs and get a ballpark idea if something is audible or not. Most of the things that audiophiles point to (HD Audio, jitter, etc.) are more than an order of magnitude below the JDD.

Thanks.

It was quite ironic that the sceptics 'out-scored' the believers by 2%, albeit statistically insignificant. I also noted the use of "very heavy duty speciality speaker cable". I wonder if it was to placate the believers or an indication of subconcious bias on the behalf of David Clark.

 

[bfreedma]

A DBT for peer reviewed publication is the same thing as a DBT for personal research, just to a different degree of precision. I keep explaining this over and over, but everyone keeps making it an "all or nothing" thing. Blind testing is a useful tool for eliminating bias whether you have a lab full of equipment or if you are just a person trying to optimize the stereo in your living room. It's still a double blind test, and it still helps eliminate bias.

Level matching can be done simply to a degree that can discern if a difference is big enough to worry about. Just run a tone through both sources and switch back and forth and adjust until you can't hear a difference any more. That works fine. If you can't hear a difference with a steady tone, there is no way it's going to make a difference when you play music. And a friend will be patient enough to help you do enough tests to know whether you should worry about it or not. You don't need to calculate pi to the 100th decimal point. There is such a thing as "good enough for government work". If you can't do it in an hour, take two hours.

Test for yourself. Don't throw up your hands and give up because you don't have a degree in engineering. Critical thinking and testing is for EVERYONE, not just people in white lab coats.

You are reducing the controls to a level that the DBT has no value. Instead, you have some personal "good enough" criteria that, frankly, is well short of achieving statistical or test validity.

Level matching by ear isn't "close enough" for a proper DBT. I wouldn't accept that answer from a "cable guy" either. It should really be done at the electrical level and if not available, with an SPL meter. We don't accept level matched claims from the cable guys and we shouldn't be moving the goalposts for ourselves.

You're also dramatically underestimating the time and test cycles needed to achieve statistically significant results. Perhaps you have a friend who will conduct 10 test cycles of 10 tests per cycle, but most folks won't have a test partner willing to invest days into proper testing. This is certainly going to take more than the hour or two state in your post

We've done this before Bigshot, and while we probably land at the same conclusions, I can't support your position that limited test cycles without proper controls, particularly accurate volume matching creates acceptable testing. If one of our cable friends executed a "DBT" without proper level matching or enough test cycles, would you consider it proof that cables sound different? I know I wouldn't...



PS - Please quote the posts you respond to. It's both the internet norm and provides context.

 

[redrol]

Yeah I have a very hard time volume matching. These days I do a 1k tone sine wave and try and get that the same volume in both ears with 2 different iems.

 

[analogsurviver]

I don't think it's possible to scrape off the barnacles. When they get the attention they crave, they'll go away again. I avoid replying with a quote because it's just "invoking the demon". And responding to points only encourages them. Best to just speak past them. I need to take my own advice more often.

It is interesting I have to thank you for teaching me strictly non audio related words in English - like "barnacles" .

We ( barnacles ) also serve a purpose ( or two, or more ) - like preserving the past :
https://www.livescience.com/barnacle-encrusted-crusader-sword-israel?utm_campaign=socialflow

And, no, double blind ABX testing is nowhere near as simple as you are constantly trying to convey.
There are already answers by others to have pointed it out, but things really begin to get problematic when dealing with analog sources, where a real hardware ABX box that does not degrade the signal more than DUT is required.
Foobar2000 is only usable with PCM digital and unusable with anything else. As that "anything else" by default tends to be better than any PCM, it does present a real problem.
There is only one commercially available real hardware ABX box :
https://www.audiosciencereview.com/...o-you-know-of-any-serious-abx-switcher.14788/

 

[bigshot]

You are reducing the controls to a level that the DBT has no value. Instead, you have some personal "good enough" criteria that, frankly, is well short of achieving statistical or test validity.

Do I have to repeat myself again? I am not aiming at validity for a formal test. I am figuring out for myself if it is worth worrying about. I keep repeating that and it doesn't get through. If something is so small that I have to use test equipment to calibrate my system to properly discern it, it flat out doesn't matter. I use my home stereo to listen to music sitting on the couch. I'm looking for audio fidelity that matters in that kind of a setting, not a testing lab. I understand that there are very minute differences. I'm not trying to discern that. That stuff can be measured better than it can be heard. I'm looking for things that matter to my ears. Of course for that purpose the comparison has a value.

I think everyone should make an effort to minimize bias and perceptual error when they are comparing different components for their system so they can make informed decisions. I don't think that requires splitting the atom. And I'm not going to discourage them from even trying because their degree of controls wouldn't please the AES.

For our resident "barnacle" the absolutism of how a "proper" listening test is conducted is an excuse to gleefully abandon controls entirely and embrace pure subjectivism, which is so much easier to do. "You can't know everything so you can't know anything." That is much more counter productive to what all of us home stereo fans here in sound science are trying to accomplish than being off on level matching by a quarter or a half a dB.

It was quite ironic that the sceptics 'out-scored' the believers by 2%, albeit statistically insignificant. I also noted the use of "very heavy duty speciality speaker cable". I wonder if it was to placate the believers or an indication of subconcious bias on the behalf of David Clark.

It was probably because the believers thought it would be easy and didn't put as much effort into listening. The skeptics knew it was going to be tough, and might have tried a bit harder... or it is just normal error.

 

[analogsurviver]

I have never said to abandon any control whatsoever and rely on pure subjectivism. I did say - and stand behind - that after being level matched to within 0.2 dB or less, I will always give preference to whatever that can please me over longer period of time - like at least the time for a normal concert ( roughly two hours ) than anything coming on top in DBT ABX. It is nice if anything audio to come on top in both categories, although it is exceedingly rare - but giving greater pleasure over longer period of time is what people generally seek in listening to any kind of music, be it live or reproduced.

I wish I could afford Van Alstine ABX comparator - but even then, there definitely are things to improve in my audio chain(s) that would make a far bigger actual difference - not only prove that difference(s) exist - for the same outlay.