Testing audiophile claims and myths

[bigshot]

People in these threads take things to ridiculous extremes. It's good to remember that we're talking about home audio and what is necessary to fulfill that purpose. We aren't sending rockets to the moon.

In an ABX test, you can switch between samples fast or slow. You can listen to each sample for a short period or a long period. But not taking auditory memory into account isn't going to get you very far.

 

[old tech]

I thought we are talking about biases, when someone is testing audio equipment, not deliberately killing people.

Obviously that analogy has gone over your head.

 

[Conan1974]

It did not looked like analogy to me. Analogy to what?
I just did not expected something like this, in audio forum. It was a bit far off.
But lets leave it hier. It is time to go to bed, at least where i live

 

[kandamrgam]

Sorry but I don’t understand that. Firstly, an ABX test is NOT a preference test.

I didn't mean ABX to be preference testing. I am proposing all blind tests better be a preference testing unlike ABX testing.

When you sit to evaluate the difference between two things in a typical sighted test

Btw, I didnt advocate for any sighted tests. Simple A/B preference testing should also be blind.

When you sit to evaluate the difference between two things in a typical sighted test, then don’t you actually have to evaluate that difference, do you just sit there and never make any attempt at evaluation? How’s that any different to an ABX?

The auditory evaluation happens either way, but your mental state is different. There is no realworld benefit for an end-user to do an ABX test (unless to win competitions, which is stressful anyway). He isn't gaining anything. The only practical reason why a music lover would attempt an ABX testing is in arguments like these.

A preference testing is much more beneficial for users. He gets to choose and own the best sounding device. It's also stressful but to a much lesser degree. He enjoys the process a lot more than ABX. Long listening sessions should make him familiar with the sounds. But like I said this has to be done at his own leisure not canjams or shops, and not many has this kind of luxury.

 

[bigshot]

Preference is only valid for the one individual. A test for discerning a difference can be useful for anyone with normal human ears. Ultimately, it all comes down to preference. But you need to discern objective fidelity as a baseline first.

 

[gregorio]

I thought we are talking about biases, when someone is testing audio equipment, not deliberately killing people.

We are talking about biases. Do you think biases only have an effect when “testing audio equipment” but not other far more critical situations? If, as you assert, it’s possible to “just focus” and “switch the bias out” then all the pilots who’ve killed themselves and their passengers due to the effect of biases must have done so because they did not “just focus” and “switch the bias out”. Therefore, you’re claiming at the very least that they’ve been grossly negligent and at worst that they’ve deliberately killed thousands of people. Or, are you claiming that only audiophiles can consciously “switch the bias out”, another supernatural audiophile ability along with hearing what’s inaudible?

The auditory evaluation happens either way, but your mental state is different.

Yes it is. I find ABXs to be tedious/boring these days but evaluating audio equipment for potential purchase to be more stressful, just determining if I can sense a difference is a far simpler and less stressful process than mentally analysing and then evaluating differences. As stress can affect the outcome, your assertion would be an argument in favour of ABX rather than against, at least for some.

There is no realworld benefit for an end-user to do an ABX test (unless to win competitions, which is stressful anyway). He isn't gaining anything.

Actually, the exact opposite is true! There is a great deal of “real world benefit” and the end-user has a great deal to gain. In fact, I highly recommend it to all consumers serious about their listening experience and it is/should be mandatory for student music/sound engineers. The obvious benefit is that it teaches a listener what they’re actually hearing as opposed to what they believe they’re hearing (due to biases), which is surprisingly different in many instances. For this reason, one could argue that anyone claiming to be a serious listener/audiophile is not actually very serious if they don’t perform a significant number of ABXs/DBTs!

Long listening sessions should make him familiar with the sounds.

While you state your belief that blind testing is better, you keep presenting exactly the same false arguments invented by audiophile reviewers in the 1980’s in order to discredit DBTs, it’s these arguments I’m refuting: The argument is false because familiarisation is a requirement of DBTs/ABXs, in fact there is a section dedicated to it, section 4.1 “Familiarisation and Training Phase” states: “Prior to formal grading, the test subjects must be allowed to become thoroughly familiar with the test facilities, the test environment, … Subjects should also become thoroughly familiar with the artefacts under study. For the most sensitive tests they should be exposed to all the material they will be grading later …” - emphasis mine.

G

 

[danadam]

A preference testing is much more beneficial for users.

Testing for preference before testing if there is any actually audible difference seems backwards to me, but that's just me.

You sound quite self-assured when you say that this is "more beneficial to users" but is it, like, just your opinion or is there something more substantive behind that claim?

 

[Conan1974]

@gregorio
I am on the audio forum, in the thread about - testing audiofiles claims and myths.
Can You just stick to that subject, and stop bringing examples of how pilots, can not controll the biases and are killing people?
Like i said, i can focus on the sound, and even if i want to have the equipment, when i do not hear a difference, i do not not buy it. Simple.
You bring the killing people thing again, man it is light years away from what we are talking about here.

 

[gregorio]

Can You just stick to that subject, and stop bringing examples of how pilots, can not controll the biases and are killing people?
…
You bring the killing people thing again, man it is light years away from what we are talking about here.

I am sticking to the subject, the subject currently being that biases affect what we believe we’re experiencing. And, pilots are relevant because aviation is an area where the effects of bias have been particularly well researched. It is absolutely NOT light years away from what we’re talking about here, it is not only about biases in general but some of the exact same biases (confirmation bias for example). Therefore, it could hardly be any closer to “what we’re talking are talking about here”!! Unless you think audiophiles aren’t humans, like pilots?

G

 

[71 dB]

The auditory evaluation happens either way, but your mental state is different.

Ah, the mental state! That's what audiophile marketing is about. If it was honest, snake oil cables would be marketed by saying "This cable doesn't sound any different from other cables, but because it looks fancy and costs an arm and a leg, using it changes the mental state of the (scientifically ignorant) listener so that the sounds is perceived being better."

There is no realworld benefit for an end-user to do an ABX test (unless to win competitions, which is stressful anyway). He isn't gaining anything. The only practical reason why a music lover would attempt an ABX testing is in arguments like these.

There is the benefit of finding out the lies of audiophile marketing and becoming a customer who knows better how money should be spend to have real "bang-for-the-buck."

A preference testing is much more beneficial for users.

Why do we have preferences? Why does Alice like A while Bob likes B? Yes, we have our preferences but they are not written in stones. We can learn away from one preference for another if we wanted, but of course we instead insist OUR preferencies are holy and to be protected!

He gets to choose and own the best sounding device.

I certainly can't own the best sounding speakers or a room with very good acoustic treatment. Too expensive for me. I have to be content with what I can afford. The ability to be content is very important. Without it only billionaires could be happy which would be crazy. A lot of billionaires haven't even worked that hard for their money. Many of them have been born in a very rich family and becoming richer and richer has been relatively easy at least in oligarchic countries. Just bribe a politician giving him/her a million and in return the politician does something (tax cut/government contract/etc.) that gives you ten million back! Easy money! It is us normal people who actually need to work our ass off to pay the bills! Instead of the best sounding device, we should settle for devices that are good enough. Fortunately digital technology has made it so that the difference between best and good enough is extremely small if not non-existing. Only room acoustics, speakers, headphones etc. suffer from the fact that the best and good enough are different things. My living room certainly doesn't have the best acoustics in the World, but it is "good enough." My speakers are not the best in the World, but they are "good enough." My headphones aren't the best in the World, but they are "good enough." I choose my demands to match the reality.

It's also stressful but to a much lesser degree. He enjoys the process a lot more than ABX. Long listening sessions should make him familiar with the sounds. But like I said this has to be done at his own leisure not canjams or shops, and not many has this kind of luxury.

Yeah, it is comfortable to do things this way, but the results are not very reliable (because of the mental states!). For reliable results you need to do the tests in a less comfortable way, unfortunately. Going to have your lungs X-rayed isn't comfortable either, but doing so gives more accurate knowledge about what is going on in your lungs.

 

[Conan1974]

@gregorio
Well, for me it is not a good example.
Audiphiles are humans, but cd player, amplifire or speakers are not plane with hundreds of people.
On top of that, that pilot is not mentally stable, like most of us, because normal, mentaly healthy person would not do it.
So what You are doing here, is comparing us, to sick person.
If You want, compare us to someone like we, mentally stable person.
That pilot is an exception. Not good comparison.

 

[FunkyBassMan]

Here we go again, how many times have you posted that I don’t know what I’m talking about and then WITHOUT EXCEPTION it’s been demonstrated that it’s actually you who have the fundamental lack of understanding? It got to be at least a dozen times you’ve made a fool of yourself and yet you just keep repeating that same error time and time again. Why, you must enjoy making a fool of yourself?!

Always your arguments are based on strawman arguments, outright lies and misquotes but this time, we’ve got a new one, actually citing scientific papers. It’s just a shame they’re irrelevant papers! How long would it have taken you to look up “echoic memory” and actually educate yourself? Probably less time than you wasted trying to find papers to prove me wrong, that actually don’t prove me wrong because they’re investigating something entirely different/irrelevant! The “Auditory Stream Segregation” paper is unsurprisingly about segregating different simultaneous audio streams and also unsurprisingly one obviously has to take the time to actually listen to and mentally analyse the whole streams. An ABX test has nothing to do with segregating and identifying different sequences of notes occurring simultaneously, an ABX test requires absolutely no identification of anything, it ONLY tests if one can reliably sense a difference. You cite Toole in reference to preferences but an ABX test does not test preferences, neither is it an evaluation, which invalidates more of your citations! I did not state that stress doesn’t affect the results and I did NOT “portray” that longer ABX tests were invalid, that is a lie and in fact I actually stated that is an option! I also didn’t state that one can’t train to mitigate to a degree some expectation bias. ABX does not attempt to mitigate some expectation bias to a degree, it attempts to eliminate all cognitive biases.

How much effort have you wasted because you haven’t even bothered to find out what an ABX actually requires or what echoic memory is? It’s just ridiculous, as it always is with you!

As usual, your last response totally twists what I said and completely misses the basic sceince about how people actually hear..

First , I didn't randomly criticize rapid ABX tests—there’s legit research behind it. McAdams and Bregman - (1979) clearly show that our brains need TIME to properly hear and process subtle audio differnces. Quick switching screws this up and makes noticing important differences way harder, not easier. Sure, echoic memory lets you briefly hold sounds for a few seconds, but it's not useful for accurately comparing subtle, detailed audio differences—those need deeper cognitive processing.

Also, you're brushing off the stress and forced-choice problems like they're nothing. They're not. Research by Blauert and Jekosch - 1997- proves repeatedly that listeners under pressure or stress mess up their judgements pretty badly. Ignoring this doesn't make sense when we're seriously tlaking audio testing.

and you're misrepresenting what I said about long-term blind A/B tests— I NEVER said rapid ABX was totally invalid. My actual point is that longer tests match real-world listening WAY better, and there's solid proof (like Berg and Rumsey, 2000). Floyd Toole’s famous Harman research (1992) backs this up even more, showing preferences actually evlove over longer listens.

About bias— you’re oversimplifying. Bias isn’t some unstoppable force wrecking every test. Studies (Olive et al., 2013) clearly show trained listeners significantly cut down bias through careful practcie and experience. It’s NOT always some huge issue ruining results.

Bottom line your insistence on rapid ABX ignores plenty of solid science. Sure, ABX has its uses, but long-term testing is scientifically better for evaluating actual listener preferences and real-world audio perception. Recognize these points if you actually want a productive dicsussion.

 

[danadam]

Audiphiles are humans, but cd player, amplifire or speakers are not plane with hundreds of people.

Are you saying that bias can be turned off when the stakes are low (e.g. in audio) but not when the stakes are high?

On top of that, that pilot is not mentally stable, like most of us, because normal, mentaly healthy person would not do it.
So what You are doing here, is comparing us, to sick person.
If You want, compare us to someone like we, mentally stable person.
That pilot is an exception. Not good comparison.

Huh? What pilot? The talk is about pilots, plural. Accidents/crashes are often caused by biases that affect pilots (just like they affect any human and even audiophiles ). Here's an article with some examples:
https://www.linkedin.com/pulse/cognitive-biases-hidden-threat-pilots-flight-safety-awdhesh-bl5dc
Surely, if turning the bias off was possible, those pilots would have done it. Or are you saying that they were all mentally unstable, sick people?

 

[gregorio]

Well, for me it is not a good example.

This is the Sound Science discussion forum, not what for you “is not a good example”. And, I’m not using aviation as “an example”, I’m using aviation as a rich source of scientific study into biases and their effects.

Audiphiles are humans, but cd player, amplifire or speakers are not plane with hundreds of people.

CD Players, amplifiers, speaker and planes do not have any cognitive biases, it’s the people that operate them who do!

On top of that, that pilot is not mentally stable, like most of us, because normal, mentaly healthy person would not do it.

That’s a lie, you’re now just making up BS to support your false assertions! There are numerous cases of CFIT due to biases by pilots who were entirely mentally stable and provably so, because ATPL holders are required to have regular physical and mental checkups. Clearly you know nothing about the subject and are “arguing from ignorance”, despite the fact you’ve already been pointed to the relevant information (on biases and heuristics). So at this point it’s deliberate/wilful ignorance!

If You want, compare us to someone like we, mentally stable person.
That pilot is an exception. Not good comparison.

I’m not talking about one pilot, an airliner requires at least two pilots and there have been numerous instances. So, if it is “not a good comparison” on the basis of different mental stability as you claim, you must be admitting that you are mentally unstable?

G

 

[Conan1974]

Whatever You going to say, person that is killing other person, is not mentaly healthy. Point. On audio forum we should stay in the listening room, where all equipment is place and all comparisons take place, and all biases can influence the testing process.
You can call me ignorant, but it will not change anything.
Let's stop here.