[bigshot]

The only thing is that we don't use blind tests. If I do a blind test and prove my "audible big differences" don't exist, that would save me a ton of money. So it's a very practical thing.

Do you do any kind of controlled testing at all? Because depending entirely on subjective impressions is only one jot above random chance in effectiveness.

Every time I get a new piece of equipment, I do A/B testing to see what it can do. When I got a CD burner, I compared a CD burned and ripped ten times to the original CD. When I got an equalizer I spent a month testing it and experimenting before I patched it into my system. When I got an iPod I did a line level matched comparison with my CD player. Then I did comparison tests with various codecs and bitrates. When I got an SACD player, I did a line level matched comparison with my CD player.

I know *exactly* what my equipment sounds like and what I can expect from it. I've learned that electronics are damn near perfect. When you plug in the speakers or headphones, *that's* where the trouble starts. I focus on the big things i can clearly hear, not the details that I have to convince myself that *maybe* I can hear if I do the right kind of double blind test.

 

[Tilpo]

I know *exactly* what my equipment sounds like and what I can expect from it. I've learned that electronics are damn near perfect. When you plug in the speakers or headphones, *that's* where the trouble starts. I focus on the big things i can clearly hear, not the details that I have to convince myself that *maybe* I can hear if I do the right kind of double blind test.

I completely agree with this, and to be honest I wish others did.

I prefer to chase giants over chasing fairies.

 

[scud80]

Doing blind tests of your equipment isn't very expensive (actually cheap compared to the typical costs of the components involved), or very difficult.  All you need are some cables, switch boxes, an accurate voltmeter, and a friend.  I don't even do blind tests, because I can't tell a difference between well-made components (regardless of price point) in a sighted test.  Doing blind tests of speakers isn't as critical, because the differences between speakers (and headphones) are usually quite large.  A/B comparisons are still important (and kind of a pain), but they don't need to be blind.
Quote:

Not every audiophile can do blind tests of everything in their system-- and virtually no one can do quick-switch/blind tests of speakers.

 

[bigshot]

If you can't tell the difference between components in a sighted test, it doesn't matter. They're effectively the same. If you enjoy going the extra mile, that's fine. But if you can't hear a difference unless you do the aural equivalent of squinting really really hard, it isn't going to make your music sound any better.

Sometimes folks lose sight of the goal. It isn't to prove that you have exceptional powers of hearing or to justify the large cash outlay on equipment you've made. It's to make your music sound better.

 

[kiteki]

I haven't read the latest posts in detail yet, however it seems like bigshot's black/white vision is so unrelenting he doesn't notice the 'flaming', he thinks he's helping save the audio society.
 
 

depending entirely on subjective impressions is only one jot above random chance in effectiveness.

Every time I get a new piece of equipment, I do A/B testing to see what it can do.
- When I got a CD burner, I compared a CD burned and ripped ten times to the original CD.
- When I got an equalizer I spent a month testing it and experimenting before I patched it into my system.
- When I got an iPod I did a line level matched comparison with my CD player.
- Then I did comparison tests with various codecs and bitrates.W
- When I got an SACD player, I did a line level matched comparison with my CD player.

I've learned that electronics are damn near perfect. [/]

 
None of these components make any difference anyway.  If you hear differences from one CD burner to the next you are hearing voices, so?
 
You experimented with five fake medicines so now you say all medicine is fake.  It doesn't work like that.  You tried to define medicine with four strict parameters and those didn't work at all either, since "distortion" encompassed the entire in sound from laptop speakers to a live concert.

 

[kiteki]

bigshot here is a blind test for you - http://www.matrixhifi.com/ENG_ppec.htm
 
This test compared lots of unnecessary components, most likely very difficult to hear in a system, what we have here is.
 
- A decent CD player
- A decent studio level stereo amplifier
- Decent cables
- Chair
 
versus
 
- Lots of unnecessary high-end components (without documenting exactly what they are, or where the differences should lie)
 
 
So the result of this advanced blind test is all subjectivist audio and marketing is fake... is it?  In my view this test proves nothing at all.  It's only a selection of fake Asian medicine to demonstrate that no Asian medicine works at all. 

Spoiler: result%20spoiler

 
Comically however, the results show that 26% selected the correct system.  They didn't take that 26% to the side to further test them, they just 'assumed' it was pure luck.  In other words a completely nonsense test pretending to be on a snow mountain of science dispelling all the fairy tale voices in audio to back up cynical theories as usual.
 
Linking to flawed data such as in these tests and holding them up as 'evidence' only hurts your position.
 

 

---

 
Next we have the Meyer & Moran test.
 
Bigshot when you said you tested your new CD burner in post #916, you said you ripped and burned a CD 10 times and then compared it to the original CD right?  That's good.  So essentially you performed a D/D/D/D/D/D/D/D/D/D loop to test for transparency and fidelity there.
 
Now let's look at the M&M study. - http://www.aes.org/e-lib/browse.cfm?elib=14195
 
The title says "Audibility of a CD-Standard A/DA/A Loop Inserted into High-Resolution Audio Playback"
 
Clearly they were too lazy to perform an A/D/A/D/A/D/A/D/A/D/A loop, you know, after buying all that multi-thousand dollar equipment and testing so many people over a year or two, they couldn't rip and burn a CD a few times, or it just never struck them to do so.
 
The statistics were completely flawed anyway.
 
As mikeaj mirrored, you can't test for transparency with non-transparent transducers, and they played the original versus the duplicate CD in the same system.  So the duplicate CD is not only as transparent as the transducers yet the system as well.
 

---

 
So, when I look at all the articles on a couple self-labelled objectivist blogs, which link to these two tests as the current standard of scientific endeavour and evidence, and advocators like Maverickronin follow suit, I can't help but question what kind of science they're supporting.  All I know is I'm not supporting it.

 

[liamstrain]

Quote:

Originally Posted by kiteki /img/forum/go_quote.gif
 
Comically however, the results show that 26% selected the correct system.  They didn't take that 26% to the side to further test them, they just 'assumed' it was pure luck.  In other words a completely nonsense test pretending to be on a snow mountain of science dispelling all the fairy tale voices in audio to back up cynical theories as usual.

 
It's also a little comical that you are phrasing it like that, instead of 74% either thought the cheaper system sounded better, or could detect no difference between them. 
 
74% is a much more impressive number, if you are talking about statistical significance.

 

[kiteki]

Yes.
But spacial perception is very much a psychoacoustical phenomenon. Trying to analyze it using measuring equipment is close to impossible.
This is a fact which we have to accept.
Can you other than spacial perception -- which is immensely complicated -- give another example?

 
It's hard but not impossible. It's about how inter-aural time and level differences vary with the position of the source.

 
Why are you two supporting spatial perception in transducers and the smyth realiser etc.?  It's purely subjective and anecdotal so it's supposed to be in polar conflict with your defined viewpoints on the furtherance and acceptance of scientific fact.
 

---

 

 
[/]
 
You could be right kiteki. Any madman with a sandwich board on a street corner proclaiming the end of the world could be right. The question isn't just why should anyone believe you but why anyone should even bother taking you seriously. Would you even bother to thoroughly investigate the ravings of a madman before discounting them as crap? Most people usually require some evidence before even spending a small portion or their mortality on investigating some random assertion and rightly so. Most new ideas turn out to be wrong.
 
That doesn't mean you should never accept new ideas, only that they should be supported by evidence first and that the amount of evidence required should be proportional to the idea's implications. That doesn't mean you shouldn't test new ideas either but there are still standards. Testing every stupid idea that someone comes up with regardless of it's plausibility is not only boring but a massive waste of resources. If you think there's a decent chance of finding something new or if you're just interested in doing it then go ahead and test it yourself. If you can't then convince someone else that it's worth testing. Calling someone out for testing an implausible claim themselves is like calling someone out for ignoring the madman proclaiming the end of the world and not thoroughly investigating his doomsday scenario. In both cases prior experience predicts a general trend. The probability of any individual claim being true along with the general exclusivity of the individual claims mean that due to the miniscule chance of any one being true you're better off treating it as false until someone else demonstrates otherwise.
 
That may sound complicated but that's really how most people deal with such things even if only unconsciously. That's why I used the "madman on the street example". They're mostly ignored and people are correct in doing so. Of course discoveries can be made by madness that pursues avenues everyone else rightly assumes are improbable based on current data as well as genius which sees what no one else saw before but that's hardly an argument for madness as productive tool of discovery given it's hit rate. The bottom line is that if you want to prove that everything we think we know about audio reproduction is wrong then you're going to need some rather interesting evidence just to get most people to take your claim seriously.
 

 
Talking about sea monsters, madmen on street corners and "new ideas" is not helping your case at all.
 
I don't have any "new ideas", whatever I believe in or try to believe in in audio is supported by most likely tens of thousands of enthusiasts and various companies pursuing these paths.
 
If anything you are believing in a sea monster of transparency, strict audio parameters and totality of data representation which are lacking in evidence or listening data to support themselves so it's just as pseudo-science anything else you try to attack so why should I take your ideals seriously, especially when you define epistemology as the facts we have in 2012 and not looking into the future?
 
In what you outlined I also doubt many things would have been discovered in science such as subliminal advertising since you would have said it's a "massive waste of resources" to look for right?
 
 

 

Oh look I just found the differences between AD797 and OPA627

 
Do you even know what those graphs mean kiteki?

 
They represent differences in raw data.  (x was 3ms and y was 1mV)
 
If two medicines are 100% equal down to the chemical level then they are equal.  If there are differences in composition or chemicals it becomes likely they could have different chemical reactions.  That's why I look at technology and design in audio and for differences in raw data, instead of completely useless academic papers.

 

[kiteki]

 
74% is a much more impressive number, if you are talking about statistical significance.

 
I'm sure much less than 26% of people can hear the difference between FLAC and MP3 128kbps so I don't care for the finding.

 

[scud80]

i don't think that's true at all.  my hearing tops out at 17-18khz now and i don't have any trouble discerning those two when paying attention, but i also know what sort of sounds to listen for to decide.  if you took some well-chosen tracks and 15 minutes to educate an average person i think they could differentiate pretty reliably between 128kbps and lossless.  it gets much harder with 160kbps and higher though.
Quote:

 
I'm sure much less than 26% of people can hear the difference between FLAC and MP3 128kbps so I don't care for the finding.

 

[maverickronin]

Quote:

They represent differences in raw data.  (x was 3ms and y was 1mV)
 
If two medicines are 100% equal down to the chemical level then they are equal.  If there are differences in composition or chemicals it becomes likely they could have different chemical reactions.  That's why I look at technology and design in audio and for differences in raw data, instead of completely useless academic papers.

 
In case you missed it that was a graph of the harmonic distortion leftover after subtracting the fundamental.  That's what it means when it says "residual".
 
Looking up the difference between time and frequency domains might be helpful too...

 

[mikeaj]

 

You experimented with five fake medicines so now you say all medicine is fake.  It doesn't work like that.  You tried to define medicine with four strict parameters and those didn't work at all either, since "distortion" encompassed the entire in sound from laptop speakers to a live concert.

 
This isn't addressed to me, but just to be more careful here, you can take "distortion" in this context to mean any change from the original.  So that will encompass any change in sound from laptop speakers compared to a live concert, by definition.  If you're talking about a couple canned THD benchmarks at a given output level and frequency, then obviously not.  However, it should be noted that while some THD benchmarks don't encompass all possible measurements, they do provide a good idea of the amount of nonlinearity in the device tested.  It's a useful measure in of itself, but it's also an indicator of something more significant.  Particularly with some electronics, if the linear distortion is vanishingly low and the nonlinear distortion is vanishingly low with a decent variety of test signals, scenarios, and so on, there's a good chance that any input signal contained within a certain set of parameters will be pretty faithfully reproduced on the output.  That's how it works.
 
 
The Matrix HiFi test is mostly just an example that a lot of audiophiles can be deluded, because the system that should have been obviously better (to some at least), was probably not, in actual listening.  It's not particularly the best-run, most-vigorous experiment.  However, I'm not much seeing some highly-vigorous, extensive studies proving certain controversial aspects of audiophile mythology?  You've got to work with the evidence that's available, whatever quality it is, and wonder why some doesn't exist.
 
 
 

 
I'm sure much less than 26% of people can hear the difference between FLAC and MP3 128kbps so I don't care for the finding.

 
Check this:
http://seanolive.blogspot.com/2012/05/more-evidence-that-kids-even-japanese.html
 
70% of trials were correct in identifying 128 kbps mp3 (LAME 3.97) from CD audio, by high school and college kids.  That's just one study though, and run by somebody with an interest in showing that people appreciate high fidelity.
 
 
Even if your figure were true, the philistines in the other 74% must have conspired to all pick the wrong answer?  Actually, I think some are assuming that the more expensive system actually has higher fidelity or sounds better to most people, which may not be true.

 

[kiteki]

 
In case you missed it that was a graph of the harmonic distortion leftover after subtracting the fundamental.  That's what it means when it says "residual".
 
Looking up the difference between time and frequency domains might be helpful too...

 
So?  Here's plenty more differences for you
 
- http://www.analog.com/static/imported-files/data_sheets/AD797.pdf
- http://www.ti.com/lit/ds/symlink/opa627.pdf

 

[kiteki]

 
[/]
The Matrix HiFi test is mostly just an example that a lot of audiophiles can be deluded, because the system that should have been obviously better (to some at least), was probably not, in actual listening.  It's not particularly the best-run, most-vigorous experiment.  However, I'm not much seeing some highly-vigorous, extensive studies proving certain controversial aspects of audiophile mythology?  You've got to work with the evidence that's available, whatever quality it is, and wonder why some doesn't exist.

 
All evidence that's available are isolated incidents, the matrix study proves exactly what it tested for, there is nothing scientific about extrapolating on the Matrix Hifi test to cover components which were not featured in the test.  That is like saying medicine A, B, C, D, E from China doesn't work so F~Z is unicorns as well.  It doesn't make any sense whatsoever.
 
As for the second test not furthering the A/D/A loop, it's furthering loops to find non-transparency which is the testing scenario in favour for the non-transparency of high-end buffers and op-amps (+A/D, D/A chips too ideally).
 
So I don't see your point.
 
Edit:  I do see your point about wondering why some don't exist.  I think it's usually a case of the time+difficulty in setting up a valid test, and the fact that 99% of individuals decide to believe or not believe, they are pre-convinced and don't see the experimentation or inquisition as necessary.  Maverick's excuse is sea monsters, however that's his very subjective viewpoint.  As an example a few years ago he could have called subliminal advertising a sea monster or madman on the street all just the same.

 

[mike1127]

Quote:

Originally Posted by bigshot /img/forum/go_quote.gif

When I say "generalized unfocused listening" i mean listening without making sure that you've eliminated as many variables that might skew your impression as possible... Specifically, putting two sound samples side by side with balanced levels and switching back and forth to allow you to identify the difference and its size. Just sitting in a chair with a glass of wine listening to Mozart isn't specific focused listening.

 
First, I would never question your love of music--I'm sure there are very meaningful experiences that you get from it. (I would be interested in how you describe these.) Because we love music and use reproduction systems, we have a lot in common. What I will say is that different people take away different experiences from music, so it is entirely likely that you and I listen for different things, or we are moved in different ways. In fact every person in this thread has a unique way of hearing and processing music.
 
What I want to explain is the idea that sensory experiences can be abstracted. In the domain of visual facial recognition, we probably all agree that people's brains are very good at recognizing the face of a familiar person. The face may appear at different angles, different lighting, different emotions on the face... even photographs of the same person at different ages. All of these images are concrete images that are very different, yet there is an abstraction that unites them all -- "this is my brother" -- or whoever. Scientists have explored the problem of facial recognition algorithms and have successfully imitated a lot of the brain's processing.
 
Musicians spend thousands of hours listening to and performing music. The concept of microdynamic resolution is one that many musicians are interested in. (They might call it something else-- the term is more common in audio-- but it's the same thing.) The concept applies both to an evaluation of a live performance (where it is about the hall acoustics, the skill of the player, and so on) and an evaluation of a reproduction. To a musician, this concept is as obvious as facial recognition is to most people. It's something that they practice perceiving (as well as controlling in their own performance).
 
I only know you through your words here, so I would not presume to know for a fact what is going in on. But your words sound like you are someone who was, metaphorically, born without the part of the brain that recognizes faces, and now claiming that facial recognition under varied conditions is nonsense. It gives the impression you haven't studied and practiced music intensively. I get the same impression from someone like Ethan Winer who claims that Pace, Rhythm and Timing (PRaT) is nonsense. All I can say is that he hasn't learned to perceive it yet.
 
 
Quote:

If you want your recorded music to sound exactly like a live performance, you should move into a night club or concert hall. The acoustics of a live venue and PA system is something that recorded music can't duplicate. Recorded music has its own properties that can be exploited, but expecting two speakers to replicate the complex reflections and directionality in sound one experiences in a jazz club or arena rock concert just isn't going to happen. Those differences are to "microdynamics" as the planet Jupiter is to a grain of sand.

 
No, I think that live acoustics vs. reproduction are equivalent to lighting a face differently. It's the same face. Scientists find this equivalent problem interesting and have worked on it. Also recognition of emotion as an abstraction across many faces. It's a problem you can sink your teeth into, not "nonsense" at all. Your general paradigm -- shared by many engineers on this forum -- is bizarre, in a way, to a musician.
 
 
 
Quote:

When you plug in the speakers or headphones, *that's* where the trouble starts. I focus on the big things i can clearly hear, not the details that I have to convince myself that *maybe* I can hear if I do the right kind of double blind test

 
Everything I'm saying applies to all stages in the audio system. Okay, I'll concede for now that CD players are essentially alike. Speakers and microphones and microphone positions, etc., affect things like microdynamics.
 
 
Quote:

I know *exactly* what my equipment sounds like and what I can expect from it

I know that also, but I learn that through listening to a variety of music at varied volume levels.