Audiophile objections to blind testing - an attempt from a layman

[2leftears]

That’s impossible. Sound is a varying pressure wave traveling through a medium (air in our case). Sound waves do not magically change the chemical composition of the air they travel through, so there is literally no difference to smell or taste. Synaesthetes therefore (completely literally!) cannot not sense sound with their nose or tongue (smell or taste sound), they just have the perception/experience of smelling sound when their auditory system is stimulated, without actually/literally smelling it.
“synaesthesia is a perceptual phenomenon in which stimulation of one sensory or cognitive pathway leads to involuntary experiences in a second sensory or cognitive pathway.” (Wikipedia) - Synaesthesia is a perceptual error, specifically NOT literally smelling, tasting or seeing sound.

However, maybe there’s some synaesthete audiophiles out there who can be suckered with a nice $1,000 INM (In Nose Monitor), $2,000 for stereo INMs (one for each nostril, immersive smell!) or $2,500 for the fully balanced, factory burnt-in version? lol

G

Synesthesia is indeed a neurodivergent perception. Some people "see" sound in the sense that sound invokes a perception of e.g. colours, but that doesn't meant they can literally "see" sound.

In order to "see" sound (or rather see shock waves in air) we need help from other equipment, such as Schlieren optics.

One of my favourite musicians has synesthesia since a car accident when she was 16. To her some sounds give a colour perception, and the songs manifest themselves in her head, pretty much involuntary. Her music is unique. In her own words: "I don't write these songs; these songs write me".

 

[Davesrose]

Oh boy, are we going to go down a rabbit hole of semantics over senses vs sensory perception? When someone enjoys music, it may begin with the stimulation of sensory cells...but it still takes the sensory pathways to the central nervous system, and translation/interpretation of the brain to actually listen. A person can lose hearing from trauma/surgery of the ear, cochlear nerves, or thalamus and/or temporal lobe regions of the brain.

 

[2leftears]

Oh boy, are we going to go down a rabbit hole of semantics over senses vs sensory perception? When someone enjoys music, it may begin with the stimulation of sensory cells...but it still takes the sensory pathways to the central nervous system, and translation/interpretation of the brain to actually listen. A person can lose hearing from trauma/surgery of the ear, cochlear nerves, or thalamus and/or temporal lobe regions of the brain.

It is necessary I'm afraid.

I'm not sure what @dstarr3 tried to imply with their comment re. synesthesia, but consider the context of this thread.

This thread (actually most of the Sound Science forum) has a history of visitors failing to grasp the difference between sound and the perception of sound. Emphasising that synesthesia is something that doesn't originate in our sensory cells seems prudent to avoid another 10+ pages of arguing about a red herring, because otherwise for sure someone will e.g. suggest that inaudible frequencies may be visible to someone with synesthesia.

 

[g4ndr1k]

I don’t want to make this hobby too complicated—real life is complicated enough, lol.

I just buy affordable things that sound good to me and are nice to look at, to maximise the enjoyment of my listening experience—after all, I don’t always close my eyes when listening to music.

If two things sound the same in a blind test but one looks better (even if it’s much more expensive), I’ll go for the latter—it simply makes me happier.

 

[2leftears]

I don’t want to make this hobby too complicated—real life is complicated enough, lol.

I just buy affordable things that sound good to me and are nice to look at, to maximise the enjoyment of my listening experience—after all, I don’t always close my eyes when listening to music.

If two things sound the same in a blind test but one looks better (even if it’s much more expensive), I’ll go for the latter—it simply makes me happier.

If you have the funds and it makes you happy, I don't think anyone here objects to that.

Go and enjoy your music!

 

[Joe Bloggs]

I'm not saying its actually like this - I'm just trying to understand if there is any logic in the argument.

My core idea is essentially that we have developed to use our senses in combination. When we consider if something is safe to eat, we might use sight, smell and touch to decide. If we lose one sense, our judgement becomes less precise. If we isolate one sense, we are easily fooled. I'm sure you've seen videos of people putting their hand down in a covered box, and there are a bunch of different objects down there - and the person is freaking out thinking a fuzzy ball and a couple of matches are a giant tarantula. Clearly, if we can't see something our sense of touch doesn't tell us too much.

Obviously our senses are used to perceive music. Thats the entire idea behind much of psychoacoustics, right? Now, these senses and our perception doesn't say a whole lot about the objective reality of the sound coming from the headphones etc. But my point is that while our judgement is clouded a whole lot due to psychoacoustics in sighted testing, is it that far fetched to think that our judgement might be clouded in blind testing, too - just in another way?

The main problem is, if any of this were true, then all is lost anyway, and we must forge on under the extent to which ears can distinguish sounds by themselves anyway.

Why?

Because if you can tell two things apart by your eyes, you don't need to listen to anything at all!

 

[castleofargh]

I don’t want to make this hobby too complicated—real life is complicated enough, lol.

I just buy affordable things that sound good to me and are nice to look at, to maximise the enjoyment of my listening experience—after all, I don’t always close my eyes when listening to music.

If two things sound the same in a blind test but one looks better (even if it’s much more expensive), I’ll go for the latter—it simply makes me happier.

I tend to do the same. I sure have cables I picked only because I thought they were prettier or softer than what I previously had. And I tend to avoid silver or bright colors for the stuff on my desk, so some perfectly legit devices were removed from my purchase list because they only came in a color I didn't want. None of those discussions are saying you can't be you, that only sound matters, or that people must buy what we think is good. Instead, we're trying to rectify all the BS claimed by people, reviewers, marketing, about the sound being almost always night and day different even if it isn't. When it is, it's cool. But claiming sound is this and that based only on sighted impressions, tends to result in too much nonsense and give birth to all the impossible myths we have in audio, where somehow most people are superhumans with no hearing threshold, infinite sensitivity, no psychological bias, and of course they're never wrong about anything they believe in for gut feeling reasons. All the things that research and science can prove to be delusional(yet expected from casual impressions and big ego). Blind tests are kind of the only clear solution to separate sound from the rest and determine if some claims about sound are true or not. That's about it. If only people were a little more cautious about what they claim to be true when actually having no clue, we might never even bring up blind testing. They're complicated, and a bother, just like about any test checking if we can or cannot do something.

 

[71 dB]

If two things sound the same in a blind test but one looks better (even if it’s much more expensive), I’ll go for the latter—it simply makes me happier.

To me, the look of audio devices is only there to sell the product, to make people want it. After you buy something, your eyes get used to the look. Stuff that initially looked ugly becomes more tolerable and stuff that looked really cool loose the effect when you get used to it. Also, I don't "look at" my audio gear any more than needed to use them. I look at/watch things that are made for that: Pictures, photos, art, movies, etc.

Of course it doesn't hurt if an audio device looks really good, but I wouldn't pay much extra just for that...

 

[redrol]

Blind testing is great, I do it all the time. Key thing is though, you gotta do it over a long time, not just seconds. This helps form a better mental picture of what the sound is doing to your brain. Quick switching is also great for finding macro changes in frequency response but the overall sound scape is much harder to gauge in seconds.

 

[castleofargh]

I fully understand the intuitive and gut feeling concept of longer listening giving more knowledge about the sound, I feel that way myself. But feelings and facts aren't always the same(otherwise blind testing probably wouldn't need to exist in the first place). I know how I feel, then I know what experiments and testing recommendations tell me. I trust research more than my own guts, maybe that's how I'm different from some audiophiles and reviewers. Less faith in my ego and more in properly demonstrated results. And data, whatever the main underlying reason can be(memory, loss of focus, gremlins), shows that shorter sound samples give more consistent and more accurate results. It's been that way since someone got the idea to test for that.
There might be exceptions, like the one listed above, or how you might need some serious listening time to develop hearing loss or fatigue from high energy ultrasounds or radioactive crystal grounding boxes. But those exceptions needs to be demonstrated first, otherwise we fall back to trusting some gut feelings instead of evidence.
With that said, selecting the short samples most revealing of sound differences might require signal analysis or long listening sessions. Probably having to listen until we get some feeling of something changing in a given passage of a given track, isolating the alleged difference, doing some blind test to check if it works or not, and maybe rinse and repeat a bunch of times(if there even is something audibly different). So we're not supporting short samples just to be quick and to reject long listening. Just that the long listening should be done beforehand, and then leave the actual test the way that works best.

 

[castleofargh]

MODO MAD
I removed everything regarding "ATMOS blablablah! No you!". I asked to move on, others did, then you guys made almost as many more posts that still had nothing to do with this thread(or IMO any thread, as it's an argument the 2 of you could have had in PM long ago, or not at all).
You know I'm not exactly the active punishing type, but this is ludicrous.

Now, hopefully, back to blind testing and the issues involved with it.

 

[Joe Bloggs]

The main problem is, if any of this were true, then all is lost anyway, and we must forge on under the extent to which ears can distinguish sounds by themselves anyway.

Why?

Because if you can tell two things apart by your eyes, you don't need to listen to anything at all!

Lol this post almost made it back to the bottom! Anyone care to respond to this? Or is this one of those elephants in the room that can't be talked about lest the whole hubbub dies down for good?

 

[2leftears]

The main problem is, if any of this were true, then all is lost anyway, and we must forge on under the extent to which ears can distinguish sounds by themselves anyway.

Why?

Because if you can tell two things apart by your eyes, you don't need to listen to anything at all!

Well, yes, that is the 'problem' with perception bias. The visual clues can be so strong that they bias our sound perception to be aligned with our visual clues, even if there is no difference in sound at all.

The McGurk effect is but one example. But as another example I have had it occur numerous times myself where I extracted the audio stream from a music video, only to find that the audio stream, when listened to in isolation from the video, sounded nothing like I remembered it from when I watched the video. Only an A/B comparison between the extracted audio stream on the one hand, and "watching" the video with my eyes closed on the other, convinced me that the sound was in fact identical. This has been as extreme as "hearing" the bass player playing from the left because the bass player was clearly featured and visible on the left side of the video, whereas on careful examination of the audio track it was obvious the sound had been mixed with the bass playing balanced toward the right channel. If visual clues can even affect the direction sound is perceived to come from, then it is hardly surprising that sighted test of different equipment can make us perceive subtle differences in sound that don't actually exist at all.

Of course in the wider context of audio and "blind testing" we are not only talking about eliminating or at least isolating visual clues, but also perception influenced by price, marketing, social context, etc.

What it all boils down to is that if we want to compare audio performance in listening tests, we need to set up and conduct those tests carefully and appropriately (i.e. using established scientific test methodologies) such that our perceptions are clearly attributable to the factors we are testing, and are not resulting from an unattributable mix of different uncontrolled factors.

Thus, I think some of the endless debate re. blind testing stems from the mixed objectives used by some audiophiles. They do uncontrolled tests to ascertain which equipment sounds "better". That is a perceived "better". And they may well choose to do those test sighted/uncontrolled (or poorly controlled) consciously allowing influence by all sorts of external factors on the basis that our "hearing" should be considered as the complex multi-factor perception it is. But then they subconsciously change the objective retrospectively and claim that their preferred equipment actually produces a different sound. That is a potential (and often actual) erroneous attribution.

It is a matter of ascertaining perceived differences on the one hand, vs. on the other hand attributing the perceived differences to the correct factors. For the latter I'm afraid a scientific approach is necessary and blind testing is a required method in that toolkit, as are acoustic and electronic measurements to help analyse the test results.

 

[AudioThief]

There’s two huge holes in your logic here:
1. Can we smell or taste sound/music? We can see abstract representations of music (a music score for example) and we can see musicians performing music but we can’t actually see sound or music. That leaves two senses to sense sound, our sense of hearing and our sense of touch (which can be triggered by loud low freq components and contribute to our sensing of sound). So, these two senses are what we isolate because our other senses cannot sense sound but might bias our perception from those senses that can.
2. Audiophiles constantly claim they trust their ears and know what they’re hearing, that differences are clearly audible, that others must be deaf if they can’t hear them, they constantly discuss sound quality, etc. So, they themselves are talking in terms of “isolate one sense”, the sense of hearing (and touch to a limited extent)!

No, NOT right! Firstly, “our senses” are not used to perceive music, only our ONE sense of hearing is used to perceive music. As mentioned, our sense of touch can contribute but we can’t sense the music with our sense of touch, only the relatively loud low freq components of it, if/when they exist.
Secondly, the entire idea behind psychoacoustics is NOT “our senses used to perceive music”, it is very clearly defined: “Psychoacoustics is the branch of psychophysics involving the scientific study of the perception of sound by the human auditory system.” (Wikipedia) - Your eyes, nose and taste buds are not part of the human auditory system.

What is “far fetched to think”, in fact way, way beyond “far fetched to think” is that you (probably derived from some audiophile marketer) have thought of a question that not one of the million or so highly educated scientists/researchers in the 165 year history of psychoacoustics has ever thought of. In fact, a more extreme example of the Dunning-Kruger Effect is hard to imagine! Do you even know what blind testing is, why it exists, what the protocols of the various (double) blind tests are and why they exist? Double blind tests and their required protocols exist precisely because scientists/researchers asked themselves; “What other ways might our judgment/perception be “clouded” and what can be done to mitigate them?”!

Sorry if this response comes off as exceptionally harsh, but really? If you’re going to make assertions about psychoacoustics in a science discussion forum shouldn’t you at the very least go and find out what the definition of psychoacoustics actually is?

G

1. I won't get into a semantic argument about the definition of psychoacoustics, but to illustrate:

We can both agree that knowing what certain gear costs, or how it looks impacts how the sound of such equipment is perceived. Whether you define this as being a part of "psychoacoustics" or not isn't important to me. If I misused the term, my apologies. The point I made, however, still stands: Our perception of music is impacted by more than just the physical qualities of the sound interacting with our ears etc.

2. "What is “far fetched to think”, in fact way, way beyond “far fetched to think” is that you (probably derived from some audiophile marketer) have thought of a question that not one of the million or so highly educated scientists/researchers in the 165 year history of psychoacoustics has ever thought of. In fact, a more extreme example of the Dunning-Kruger Effect is hard to imagine!"

I know that you are logically capable and intellectually honest enough to know that this is just a complete non argument. By that same logic, Einstein was showing a truly elite level of Dunning-Kruger effect when he challenged our previous understanding of time and gravity. Yes, I just compared my incoherent ramblings about a topic I don't understand to Einstein.

3. "Do you even know what blind testing is, why it exists, what the protocols of the various (double) blind tests are and why they exist? Double blind tests and their required protocols exist precisely because scientists/researchers asked themselves; “What other ways might our judgment/perception be “clouded” and what can be done to mitigate them?”!"

Yes. I'm not sure where you're going with this.

castleofargh:

"It's also kind of what we're saying. People will easily assume their impression only comes from sound when it doesn't. If I try to put myself in your shoes and consider the issues of blind listening (which can be numerous depending on the method and the cues we're trying to test, nobody is saying it's absolute and perfect), I still see no reason why a casual subjective impression would be more trustworthy."

Well, I am certaintly not making the claim that a casual subjective impression would be more trustworthy than blind testing. Not even close. I am simply considering the possible flaws in how people think about blind tests and their accuracy. It seems quite obvious to me that they are many, many times more reliable than any subjective, sighted impression.

 

[gregorio]

We can both agree that knowing what certain gear costs, or how it looks impacts how the sound of such equipment is perceived.

No, we apparently can’t even agree on that. That knowledge may and often will impact our perception but not necessarily always and not necessarily for everyone.

Whether you define this as being a part of "psychoacoustics" or not isn't important to me.

Your OP was all about the perception of sound and there is a branch of science that deals specifically/exclusively with that topic, “psychoacoustics”. You stated that “I am a former subjectivist”, that “I really want the sound science users take on it” and elsewhere that “I decided to stop rationalizing and accept a more scientific approach” but in the above quote you’re saying that the definition of psychoacoustics, which defines “our take on it” and the “scientific approach”, “isn’t important to me [you]”. So which is it?

The point I made, however, still stands: Our perception of music is impacted by more than just the physical qualities of the sound interacting with our ears etc.

That wasn’t the point you made, or at least not the main point. Your main point was a fallacy, a false equivalence, namely that because you saw a TV show which appeared to demonstrate that the senses of taste and smell were relatively useless without sight, that the perception of sound/music was also largely dependent on sight and therefore, a blind test will limit our ability to perceive music/differences. There are numerous reasons, some of them obvious, why your assumption is false.
Firstly, a blind test does not mean you are literally blind, a blind test refers to knowledge rather than sight.
Secondly, when listening to an audio recording of music we are “blind” anyway! Regardless of whether you’re doing a “blind test” or just casual sighted listening, you cannot see the orchestra/band/musicians you’re listening to.
Thirdly, when you close your eyes and listen, can you no longer tell the difference between Mozart and Motorhead, can you no longer pick out the violins, electric guitar or other instruments? Our perception/hearing can obviously still function well, even when we can’t see what we’re listening to, if it couldn’t there would be no reason or demand for audio only music recordings. We can’t see, smell, taste or touch music, what other senses are we reliant on to interpret sound and what reliable evidence do you have for such an assertion?
Lastly, our perception of music is impacted by a great deal more than just the sound entering our ears, in fact there would be no perception of music if that were not the case, that is why we have “blind testing” in the first place.

I know that you are logically capable and intellectually honest enough to know that this is just a complete non argument. By that same logic, Einstein was showing a truly elite level of Dunning-Kruger effect when he challenged our previous understanding of time and gravity. Yes, I just compared my incoherent ramblings about a topic I don't understand to Einstein.

How can you judge that I am “logically capable and intellectually honest”, when you are not? Because the actual logic is pretty much the opposite to what you claim is “that same logic”. If Einstein had made “incoherent ramblings about a topic he didn’t understand”, AND if he were contradicting very reliable demonstrated evidence, AND if he did all that purely on the basis of assumptions/fallacies, AND without any reliable evidence, then that would be “that same logic” and indeed he too would have been a victim of the Dunning-Kruger effect! As Einstein did none of these things, so how is his theory of relativity an example of “that same logic” and not instead an example of a “false analogy”?

Yes. I'm not sure where you're going with this.

And there’s your problem, or at least one of them. If you don’t know how a blind test is conducted or even what it is, how can you know what it does or doesn’t account for/mitigate and therefore what questions others have asked (and answered)? An obvious example is you claiming hearing shouldn’t be isolated from sight, when blind testing protocol does not require test subjects be deprived of sight and ignores the fact that we can’t see the music/music performance we’re listening to under any conditions, blind or sighted (when listening to an audio recording) anyway.

G