[mike1127]

Quote:

Doing blind tests of your equipment isn't very expensive (actually cheap compared to the typical costs of the components involved), or very difficult.  All you need are some cables, switch boxes, an accurate voltmeter, and a friend.  I don't even do blind tests, because I can't tell a difference between well-made components (regardless of price point) in a sighted test.  Doing blind tests of speakers isn't as critical, because the differences between speakers (and headphones) are usually quite large.  A/B comparisons are still important (and kind of a pain), but they don't need to be blind.

 
I think there are some puzzling inconsistencies here.
 
I agree with many engineers on this forum that people can be misled by expectation. But why would it only make the illusion of differences? Couldn't it give the illusion of sameness? For example, if you are attached to the belief that CD players sound alike, wouldn't your sighted evaluation be unreliable?---because a difference is something you don't want to find.
 
Also, this idea that a test should be run by the following steps:
 

• listen sighted to hone in on the specific difference
• do a blind test

 
It seems to me that you have to acknowledge that you could be "honing in on" an illusory difference even when there is a real difference. Then your blind test would be useless, because you aren't listening for the right thing.
 
EDIT: same problem comes up when you say you don't do blind tests of speakers. Expectation is still at work.

 

[mike1127]

Quote:

If you can't tell the difference between components in a sighted test, it doesn't matter. They're effectively the same. If you enjoy going the extra mile, that's fine. But if you can't hear a difference unless you do the aural equivalent of squinting really really hard, it isn't going to make your music sound any better.
Sometimes folks lose sight of the goal. It isn't to prove that you have exceptional powers of hearing or to justify the large cash outlay on equipment you've made. It's to make your music sound better.

I don't know anyone who buys expensive stuff for these two reasons ("to prove you have exceptional power...", "justify large cash outlay..."). I think you are projecting them on people.

 

[mikeaj]

Quote:

I agree with many engineers on this forum that people can be misled by expectation. But why would it only make the illusion of differences? Couldn't it give the illusion of sameness? For example, if you are attached to the belief that CD players sound alike, wouldn't your sighted evaluation be unreliable?---because a difference is something you don't want to find.

 
Nah, there's no inconsistency.  Most people who point out expectation bias (at least me and most here), will say it goes both ways, as you say.  Somebody not expecting to hear differences will be predisposed not to report differences.  There's a further possibility of potential lying and intentional sandbagging—dishonest reporting—by those who know what's actually being tested (A and B in general, not which is which).
 
 
On a complete side note, I'm fairly sure that "microdynamic resolution" is not part of the vocabulary or jargon of musicians.  I hope you don't further bring up qualifications or experiences into the discussion, because that's not very productive, and you may find that some people have different backgrounds than you might assume.

 

[scud80]

Nobody said this should be the procedure.  Regardless, I had no expectation of sameness.  In fact, I had an expectation of difference after reading so many reviews on here talking about how different their different pieces of gear sounded.  Hearing none was a surprise.
Quote:

Also, this idea that a test should be run by the following steps:

• listen sighted to hone in on the specific difference
• do a blind test

 
And sure, there is bias with speakers, however the difference is that speakers measure very differently from each other whereas amps/dacs/cables/etc do not (well, they do, but differences are usually -70dB or less).  Yes, blind tests would be better, but they're much less practical for speakers.  Other components can just be switched in place, basically instantaneously, but speakers would have to be moved around for each trial, which is why nobody does it.

 

[kiteki]

Quote:

 

I'm sure much less than 26% of people can hear the difference between FLAC and MP3 128kbps so I don't care for the finding.

 
Check this:
http://seanolive.blogspot.com/2012/05/more-evidence-that-kids-even-japanese.html
 
70% of trials were correct in identifying 128 kbps mp3 (LAME 3.97) from CD audio, by high school and college kids.  That's just one study though, and run by somebody with an interest in showing that people appreciate high fidelity.
 
Even if your figure were true, the philistines in the other 74% must have conspired to all pick the wrong answer?  Actually, I think some are assuming that the more expensive system actually has higher fidelity or sounds better to most people, which may not be true.

 
You have raised the point now (thanks) the 'philistines' could have decided to pick the wrong answer, since in the testing scenario they were all in the same room on chairs (see picture), the individual choices were decided by pointing a finger at which system was which.  Clearly someone could have pointed in the opposite direction of someone else in the room they were familiar with, or same direction.  I don't feel like dissecting it anymore because it's a completely useless test, apart from 'proving' that some random Ikea furniture, power cables, power conditioners, and mystery stereo amplifier (with mystery insides) will only be audible to 24% of people at a Spanish audio convention, so the conclusion is -> you better not buy mystery systems if you want to convince more people than 24%.
 
It's not like I'm defending random power conditioners, just pointing out why should this be held up as evidence of anything?  It's only injuring the so called objectivist view and making it look cynical, which in turn, ushers people to subjectivism when the science is so ill.
 

---

 
As for your MP3 link.
 
http://en.wikipedia.org/wiki/MPEG-1#Quality_2
 
"Subjective audio testing by experts, in the most critical conditions ever implemented, has shown MP2 to offer transparent audio compression at 256 kbit/s for 16-bit 44.1 kHz CD audio using the earliest reference implementation (more recent encoders should presumably perform even better).^{[size=x-small][1][/size][size=x-small][48][/size][size=x-small][49][/size][size=x-small][54][/size]"}
 
So if you believe in peer reviewed papers and critical test conditions, shouldn't you be saying MP2 at 256 kbps is transparent and the rest is mythology?  That's what Wikipedia and the ABX tests of expert listeners indicate, yes?

 

[kiteki]

Yes, blind tests would be better, but they're much less practical for speakers.  Other components can just be switched in place, basically instantaneously, but speakers would have to be moved around for each trial, which is why nobody does it.

 
Several types of speakers connected to the same stereo receiver is actually pretty easy.  Blind evaluation of several types of speakers is most likely pretty common, and should be performed more.  You don't "get what you pay for" in audio, so, it would be nice to have a magazine which only blind evaluates, don't you think?
 
Some people have their senses all mixed up, in their neural pathways, they see colours with numbers, the number 5 is orange, and they see sounds with visuals, it's probably fairly common since they don't know any different, they just think everyone sees like that, just like black/white vision people don't know any different either.
 
I think the only variable everyone should at least be able to agree on is blind evaluation, you don't need to know how much something costs if you have any idea what sound performance should sound like.  Plus, I think the vast majority of people when presented with the same sound will say it's the same.  Using someone who has no idea and says two identical sounds are different is not a case study of subjective fantasy, it's just a case study of some random individual who had no idea, so when he tasted Coke versus Coke he said one was Pepsi.
 
Using him as a case study to conquer subjectivity just makes 'objective' audio look more and more ill.
 

---

 
Of course the alternative to objective audio isn't fantastical delusion, it's something called err, reality.
 
Everything in audio and any field of science has a real truth to it, truth is not subjective it only exists, pretending you've found it with a dScope and Wikipedia is h*******t.
 
If anyone has any more useless academic papers to present I'd be happy to refute them with more conflicting studies or statistical error.
 
p.s. Nick_charles never answered on how subliminal advertising was proven (fMRI) and how 24bit /192kHz was 'proven' (fMRI).  So just a side note isn't the scientific method to replicate a test and come to the same or a contrary finding?  If a study hasn't been replicated it still stands, according to the 'scientific method', unless you want to cherry pick papers to suit your own agenda, right?

 

[mikeaj]

Quote:

So if you believe in peer reviewed papers and critical test conditions, shouldn't you be saying MP2 at 256 kbps is transparent and the rest is mythology?  That's what Wikipedia and the ABX tests of expert listeners indicate, yes?

 
Some people overinterpret the data or for simplicity's sake (of wording, not throwing in the usual caveats), maybe make looser claims than appropriate.  Sure.  Anyway, as many have already pointed out, null results don't prove anything.
 
 
I'm saying that peer-reviewed papers are preferable to non-peer-reviewed papers, and experiments with proper controls are better than experiments without proper controls.  It includes blinding, to control for potential biases.  That said, a lot of experiments that do blind testing or even double-blind testing, may be junk in other ways.
 
 
A lot of audiophile claims being made don't really make much sense in the context of the engineering analysis, so current evidence consists of much more than just listening tests, for the record.  
 
I'd just point out that statements like these:
 

 
If two medicines are 100% equal down to the chemical level then they are equal.  If there are differences in composition or chemicals it becomes likely they could have different chemical reactions.  That's why I look at technology and design in audio and for differences in raw data, instead of completely useless academic papers.

 
seem to reflect a popular sentiment—that different designs / whatevers should sound different.  It's not just you.  However, the reality is that very different topologies and circuits can produce very similar results, so your intuition from other fields is hurting you.  That's one danger of running too many analogies without understanding how they all apply.

 

[mike1127]

Quote:

 
 
On a complete side note, I'm fairly sure that "microdynamic resolution" is not part of the vocabulary or jargon of musicians.  I hope you don't further bring up qualifications or experiences into the discussion, because that's not very productive, and you may find that some people have different backgrounds than you might assume.

Musicians are concerned about the same thing. They might call it dynamic shading or shaping. It's commonly discussed. Musicians that do it well are praised or lauded. Etc.
 
Experiences should be part of the discussion, and it should be acknowledged that people tend to hear the things they have practiced hearing, and tend not to hear things they haven't practiced.
 
I assume nothing about anyone's background -- in fact I have said repeatedly that we all hear uniquely, and this is obviously affected by background. I would be glad if anyone here wants to describe what is valuable to them in music. I've mentioned microdynamic resolution, PRaT, and beauty. No one else has mentioned what they enjoy about music since I joined this discussion.

 

[mike1127]

Quote:

Nobody said this should be the procedure.  Regardless, I had no expectation of sameness.  In fact, I had an expectation of difference after reading so many reviews on here talking about how different their different pieces of gear sounded.  Hearing none was a surprise.

 
It has been said many times on this forum that this should be the procedure. Maybe you didn't say it. That's fine. But how can you know you had no expectation of sameness? Those "deluded audiophools" who do sighted listening of cables would say they have no expectation of difference. It's unconscious. It's easy to imagine someone attached to the idea of sameness -- if they like the idea of having a nice, neat understanding of audio, and that understanding predicted two CD players should be the same, then they would have a motivation to hear sameness. Maybe they unconsciously don't try hard enough to hear a difference, giving up too soon.

 

[bigshot]

What I want to explain is the idea that sensory experiences can be abstracted.

The reason we're talking at cross purposes here is that we are talking about two entirely different things... You're talking about listening for the purposes of appreciating music, which is probably everyone's eventual purpose. But I'm talking about an intemediary step. Listening for the purpose of comparing audio equipment for purposes of determining the faithfulness of the sound reproduction.

With face recognition, a stylized and caricatured painting of a face might be a truer likeness as a photograph. But you don't pick a camera by how well it caricatures. You pick it for how faithfully and accurately it reproduces an image. The first step is simple mechanical reproduction, and that is a very cut and dried thing.

Once you've established that a camera provides a clear, undistorted image, you can hand it to an artist and he can create whatever kind of image he wants... Even unclear and distorted ones. But the basic performance of the camera is the benchmark where that creativity starts.

Recording engineers do the same thing. They start with very carefully calibrated equipment that hits an established benchmark... Flat response, broad dynamics, low distortion... Then they add their own aesthetic choices about the equalization, compression and fuzziness of the sound. When we buy their CD, they have made all of those creative choices, and if our system matches the calibration of theirs, we hear the creative statement they're making.

I understand that the reproduction of music involves all sorts of aesthetic choices, but playing back a CD isn't the place for aesthetics. That's where all you want is accuracy. Once a sound engineer has achieved the first step of accuracy, he can create his sound. When he's happy with it, he passes it along to us, hoping we follow his lead in playing it back with accuracy.

The way we achieve that is by carefully listening to our equipment with the goal of matching the benchmark calibration. That's an achievable goal. Once you get to that point and hear what is intended, you can feel free to get creative with the sound yourself. EQ it however you want, run it through a digital delay to simulate an ambience, or remap the sound to multiplespeakers instead of just two. You always have those options. But if you choose equipment that isn't calibrated to be accurate, you've got no clear benchmark to start from.

 

[bigshot]

I don't know anyone who buys expensive stuff for these two reasons ("to prove you have exceptional power...", "justify large cash outlay...").

You should meet more audiophiles!

 

[mike1127]

Quote:

The reason we're talking at cross purposes here is that we are talking about two entirely different things... You're talking about listening for the purposes of appreciating music, which is probably everyone's eventual purpose. But I'm talking about an intemediary step. Listening for the purpose of comparing audio equipment for purposes of determining the faithfulness of the sound reproduction.
 

It would be nice if you made some kind of response to my point about abstracted perception, because you seem to deny it exists.
 
And no, we are talking about the same thing. If I sit in a studio and listen to a musician perform, I might listen for something like the microdynamic resolution. And when I listen to the same performance as a reproduction, I listen to the dynamics and can make a judgment about whether they are faithfully reproduced. It appears that you reject this possibility entirely, which is because, as far as I can tell, you reject the notion of abstracted perception (but it would be nice if you addressed it).

 

[kiteki]

bigshot has been meeting the wrong audiophiles.

 

[mike1127]

Quote:

You should meet more audiophiles!

I probably should.

 

[bigshot]

Most people who point out expectation bias (at least me and most here), will say it goes both ways, as you say.  Somebody not expecting to hear differences will be predisposed not to report differences.

It depends on what you are listening for. Expectation bias has an effect on determining the difference between two very similar things. But if you don't care about very similar things and you're looking for significant differences, simple controlled observation should do the trick. I really don't care about gnat hairs- just things that make a difference.