[royalcrown]

Quote:

Originally Posted by PhilS /img/forum/go_quote.gif Ok, so if one were to download the software, and the RMAA measurements indicate that the differences are supposedly below the "audible limits," but one's ears indicate that one DAC (the one that costs $500 more) sounds better than the other, should one choose the DAC that seems to sound better, or instead pick the cheaper one, because the measurements, in conjunction with what you and others may say about "audible limits," indicate that both should sound the same?

I'd say test blind. However you reached your conclusion that the 500 dollar one sounds better, replicate that except this time control it such that you don't know which one is which. If you cannot identify it blind, but you can identify it not-blind, that therefore means that the difference you hear is not solely due to your ears but due to something else.

 

[ph0rk]

Quote:

Originally Posted by PhilS /img/forum/go_quote.gif I'm not that familiar with RMAA, and (no offense meant) your response really doesn't enable anyone to further help me with my question, unless they know what RMAA does and what measurements they provide, and what measurements are arguably relevant to our issue (and can take your lead and run with it, so to speak). Could you ( or anyone else who thinks one should not trust his or her ears) elaborate as to what measurements are allegedly relevant to comparing two DAC's or CD players? Also, what type of measurements are you referencing that are easy to produce at home?

I provided a link to a website that both explains what RMAA does and how to use it, but also lets you download it for free. I downloaded an earlier version back in 2003 to test the headphone out of my old klipsch promedia 4-2's, and it was relatively straightforward at the time.

Anyway, I had no intention of describing the software's use - you wondered where objective measurements might come from. As for what to look for, I'd start with flat frequency response, very low crosstalk, high dynamic range, and low distortion (there are often peaks at 60hz from wall power). For amps you'll likely need an oscilloscope and some extra required reading.

Quote:

Originally Posted by PhilS /img/forum/go_quote.gif Ok, so if one were to download the software, and the RMAA measurements indicate that the differences are supposedly below the "audible limits," but one's ears indicate that one DAC (the one that costs $500 more) sounds better than the other, should one choose the DAC that seems to sound better, or instead pick the cheaper one, because the measurements, in conjunction with what you and others may say about "audible limits," indicate that both should sound the same?

In this case, there are one of two things happening:

h1: There is a difference but your test software cannot detect it.

h2: There is no (humanly detectable) difference, and your impressions based on other factors (visual, tactile, price, etc) are clouding your judgement.

We don't know how to (objectively and in a replicable way) test h1, but we can test h2, by a form of blind testing.

If after blind testing you can reliably identify a difference, then we'd reject h2 and take a closer look at h1. Test equipment could be faulty or (theoretically) there could be some quality that matters we haven't measured.

 

[PhilS]

Yes, if you blind test, and you can do it a satisfactory and convincing way, and you don't hear a difference, then it would appear you should purchase the cheaper DAC. That makes sense. But what if it is not practical for you to conduct a blind test, or you don't wish to undertake the effort. Some folks have places to go, people to see, things to do. Would you say it is unreasonable for a person to choose the more expensive DAC, based on what he hears? I mean, not everyone wants to conduct a blind test every time they have to choose between component A, component B, and component C, etc.

P.S. And, of course, the magnitude of the perceived difference also plays a role. If one listens to DAC A, and is clearly convinced it sounds better to one's ears than DAC B, does one really need to allocate limited time resources to downloading measurements to see whether they might reveal a difference within "audible limits, and (assuming no difference is revealed within alleged "audible limits") conducting a blind test to confirm what seems obvious?

 

[ph0rk]

Quote:

Originally Posted by PhilS /img/forum/go_quote.gif Yes, if you blind test, and you can do it a satisfactory and convincing way, and you don't hear a difference, then it would appear you should purchase the cheaper DAC. That makes sense. But what if it is not practical for you to conduct a blind test, or you don't wish to undertake the effort. Some folks have places to go, people to see, things to do. Would you say it is unreasonable for a person to choose the more expensive DAC, based on what he hears? I mean, not everyone wants to conduct a blind test every time they have to choose between component A, component B, and component C, etc. P.S. And, of course, the magnitude of the perceived difference also plays a role. If one listens to DAC A, and is clearly convinced it sounds better to one's ears than DAC B, does one really need to allocate limited time resources to downloading measurements to see whether they might reveal a difference within "audible limits, and (assuming no difference is revealed within alleged "audible limits") conducting a blind test to confirm what seems obvious?

If their time is worth more than the money they'd save for an hour or two of testing, then efficiency is fine. I don' get paid hundreds an hour, but some others might.

When they later argue for the superiority of the piece of gear they purchased without doing the testing that we have a reliability problem.

Of course most of this scenario could be avoided if they just compared the gear blind to begin with.

 

[royalcrown]

Quote:

Originally Posted by PhilS /img/forum/go_quote.gif Yes, if you blind test, and you can do it a satisfactory and convincing way, and you don't hear a difference, then it would appear you should purchase the cheaper DAC. That makes sense. But what if it is not practical for you to conduct a blind test, or you don't wish to undertake the effort. Some folks have places to go, people to see, things to do. Would you say it is unreasonable for a person to choose the more expensive DAC, based on what he hears? I mean, not everyone wants to conduct a blind test every time they have to choose between component A, component B, and component C, etc.

I would say that the measurements take priority. Of course you're not going to be able to certain without blind testing, but in terms of picking on probability there's a much higher likelihood of the perceived effects not coming from the actual component, and thus that perception would be subject to change (for example over time) and you'd be sinking money into something that wouldn't actually make a real difference. So many things affect your listening state at a given time: aesthetic appeal, price, your mood, the temperature, etc. that it's much more reliable to go with what's objective.

 

[PhilS]

Quote:

Originally Posted by ph0rk /img/forum/go_quote.gif When they later argue for the superiority of the piece of gear they purchased without doing the testing that we have a reliability problem.

Well, maybe. It may depend on who is making the argument, i.e., there are some folks on this forum whose "ears" have more credibility than others, IMO.

But in any event, the issue is not whether you should trust what others say they heard, but whether you should trust your own ears.

 

[PhilS]

Quote:

Originally Posted by royalcrown /img/forum/go_quote.gif I would say that the measurements take priority. Of course you're not going to be able to certain without blind testing, but in terms of picking on probability there's a much higher likelihood of the perceived effects not coming from the actual component, and thus that perception would be subject to change (for example over time) and you'd be sinking money into something that wouldn't actually make a real difference. So many things affect your listening state at a given time: aesthetic appeal, price, your mood, the temperature, etc. that it's much more reliable to go with what's objective.

If you want to give priority to the measurements, that's fine. I would not say that's an unreasonable choice. But I think others would choose differently.

Also, I think I would question your assumption about the probability of the perceived differences lasting over time. Indeed, there are well-known instances in which objective testing led to the choice of a component that the listener concluded over time was a very unsatisfactory choice. And just as mood, etc. can affect your "listening state," so can similar variables affect your performance on a blind test. (Ever throw waste paper balls absent-mindedly into a waste paper basket, and then perform much worse when you test yourself to see how many you can make out of 10?)

I'm not saying blind tests have no value, or that people should not do them if they wish, but in the end, I don't think it is unreasonable for people to decide that they want to trust their own ears in certain circumstances. Several people on this forum have routinely pushed the notion that all "well-made" DAC's and CD players sound the same, and I think that notion, based on personal experience, is just silly. I'd prefer to trust my own ears on that issue, taking careful account of the potential for the senses to be deceived.

But, again, the issue in some sense is what makes people enjoy the music more. If I hear a difference, and I'm happy, I think I'll always be happy. If I hear a difference, flunk a blind test, and choose the cheaper DAC, I think I'll always be wondering about what I might be missing.

 

[ph0rk]

Quote:

Originally Posted by PhilS /img/forum/go_quote.gif But in any event, the issue is not whether you should trust what others say they heard, but whether you should trust your own ears.

Then, in short: trust no one, not even yourself.

Less short: (IMHO) to tell if something is enjoyable, apply a liberal dash of occam's razor and listen, no need for tests as they are for discerning what you can't here. When comparing equipment for an objective evaluation, don't trust your ears because they're full of it! Anyone is of course free to hold a different opinion, but I prefer the position of assuming everything is a suit of the emperor's new clothes until I can see or hear otherwise for myself. If that is a complicated or expensive venture, then I won't bother.

It is even worse in speaker-land, without extensive room treatments virtually any gear difference is a waste of time (short of new transducers) - you'll run a free app like room eq wizard only to discover 30 db nulls all over the bass region, and evil spikes in the mid/lower treble, just because.

 

[ph0rk]

Quote:

Originally Posted by PhilS /img/forum/go_quote.gif I'm not saying blind tests have no value, or that people should not do them if they wish, but in the end, I don't think it is unreasonable for people to decide that they want to trust their own ears in certain circumstances. Several people on this forum have routinely pushed the notion that all "well-made" DAC's and CD players sound the same, and I think that notion, based on personal experience, is just silly. I'd prefer to trust my own ears on that issue, taking careful account of the potential for the senses to be deceived.

Ah, well for many of us, we know that in any other realm the bolded part is impossible without a rigorous test procedure, so to do without in audio is... unsuitable.

 

[PhilS]

Quote:

Originally Posted by ph0rk /img/forum/go_quote.gif Then, in short: trust no one, not even yourself. Less short: (IMHO) to tell if something is enjoyable, apply a liberal dash of occam's razor and listen, no need for tests as they are for discerning what you can't here. When comparing equipment for an objective evaluation, don't trust your ears because they're full of it! Anyone is of course free to hold a different opinion, but I prefer the position of assuming everything is a suit of the emperor's new clothes until I can see or hear otherwise for myself. If that is a complicated or expensive venture, then I won't bother.

Huh? You lost me.

 

[mike1127]

Quote:

Originally Posted by ph0rk /img/forum/go_quote.gif At the risk of being extremely pedantic: That paragraph has a Flesch-Kincaid grade level of 10.27 and a Flesch reading ease score of 61.96 (60-65 is taken to be plain english, 100 is very easy to read, and 0 is very hard). Where, exactly, did you get lost?

The paragraph was poorly constructed and included a crucial misspelling. Here is what you meant to say:

Quote:

Originally Posted by ph0rk original Less short: (IMHO) to tell if something is enjoyable, apply a liberal dash of occam's razor and listen, no need for tests as they are for discerning what you can't here (sic). When comparing equipment for an objective evaluation, don't trust your ears because they're full of it! Anyone is of course free to hold a different opinion, but I prefer the position of assuming everything is a suit of the emperor's new clothes until I can see or hear otherwise for myself. If that is a complicated or expensive venture, then I won't bother.

Quote:

Originally Posted by ph0rk rewritten Less short: in my opinion, when you want to tell if something is enjoyable, just listen. No need for tests, as they are for discerning whether two pieces of equipment are audibly the same. But if you are comparing equipment to determine in an objective sense what is different about those two devices, don't trust your ears because they are full of it! Anyone is of course free to hold a different opinion, but I prefer the position of assuming everything is a suit of the emperor's new clothes until I can see or hear otherwise for myself (in the sense that a device that claims to improve the sound should be treated with skepticism). If it's complicated or expensive to "see or hear for myself" then I won't bother.

An enormous irony here is that the measurements you used, the "Flesch-Kincaid grade level" and "Flesch reading ease score", told us absolutely nothing about your paragraph! It takes a human to actually read it and realize that it had poor grammar, a crucial misspelling, poorly formulated thoughts, and so on.

In the same way, standard measurements of audio equipment tell me (and a certain class of listeners like me) almost nothing about how a device will sound, and almost nothing about its quality or resolution. People who favor the use of measurements don't seem to realize that they are an extremely tiny peek into a universe of behavior. Evaluating a device through measurements is like trying to authenticate a Van Gogh painting by looking through a narrow cardboard tube.

 

[ph0rk]

Meaning if you enjoy it, you enjoy it. Listen away.

If instead you are trying to make an evaluation (especially for the purpose of recommendations) between two pieces of gear, keep in mind that your ears are suspect and try to take as many steps as possible to eliminate that source of error.

My default position is that most "upgrades" won't offer much benefit anymore, and are at best a different presentation rather than a clear improvement. I am happy to be proven wrong, but I 'm not going to go out of my way to hunt down every possible upgrade, either. It is more about the music than perfection for me, and I quite frankly have other things I need to be doing.

Quote:

Originally Posted by mike1127 /img/forum/go_quote.gif The paragraph was poorly constructed and included a crucial misspelling. Here is what you meant to say:

I was being a bit snarky, yes, but if you figured it out it was close enough for jazz, wasn't it? Anything posted by me after 10pm is pretty much guaranteed to be posted while more than a bit tipsy.

 

[royalcrown]

Quote:

Originally Posted by mike1127 /img/forum/go_quote.gif An enormous irony here is that the measurements you used, the "Flesch-Kincaid grade level" and "Flesch reading ease score", told us absolutely nothing about your paragraph! It takes a human to actually read it and realize that it had poor grammar, a crucial misspelling, poorly formulated thoughts, and so on.

The measurements told you plenty of information: it was written at a specific grade level, and can be comprehended by an approximate percentage of the world. The measurements did plenty, you just used them for the wrong purpose (as you seem to continuously do) - the measurements aren't there to tell if mike1127 can understand ph0rk's paragraph, it tells you whether or not a specific percentage of the population can comprehend that statement. Ph0rk's score indicates that his post was, according to wikipedia, "easily understandable by 13- to 15-year old students" (Flesch-Kincaid readability test - Wikipedia, the free encyclopedia). Perhaps you just fell outside of the demographic, perhaps you're nitpicking (misspelling "hear" for "here" is not a crucial typo - nearly anyone can figure out what he meant given the context of the post in an audio forum), WHO KNOWS. The point I'm trying to make is that the measurements don't provide interpretation, that's the job of the scientist to interpret results properly, and scientific journals exist so that people don't wantonly misinterpret results without validation from the community at large.

Quote:

Originally Posted by mike1127 /img/forum/go_quote.gif In the same way, standard measurements of audio equipment tell me (and a certain class of listeners like me) almost nothing about how a device will sound, and almost nothing about its quality or resolution. People who favor the use of measurements don't seem to realize that they are an extremely tiny peek into a universe of behavior. Evaluating a device through measurements is like trying to authenticate a Van Gogh painting by looking through a narrow cardboard tube.

I've said it before, and I'll say it again: the purpose of audio measurements are not to tell people how something sounds. They exist to determine whether or not a device is beyond audible limits or to determine differences between components (i.e. whether or not a difference exists, not how that difference will perceptually manifest itself).

Besides, your claim that "people who favor the use of measurements don't seem to realize that they are an extremely tiny peek into a universe of behavior" has no substantiation. Where has it ever been proven in the course of history that the audio measurements we have are incomplete? Also, your Van Gogh analogy also falls through because we have spectrometers that will tell you the exact composition of a Van Gogh painting. It won't tell you what it looks like, but it will tell you whether or not you have a fake (based on whether or not the data matches the data of the original).

 

[PhilS]

This thread has taken a hilarious (and sort or sad) turn. phOrk posted something. I didn't understand the point he was making, and my response was not impolite. Furthermore, I'm not an idiot. I have an Economics degree and a Law degree from a top-10 U.S. University, and have been practicing complex litigation (which involves extensive writing on complex subjects) for almost 30 years. So I would suggest that the fact that the "Flesch-Kincaid grade level" and "Flesch reading ease score" indicates phOrk's post was written at a specific grade level tells you nothing.

And mike1127's point is right on. Instead of dealing with the substance of my post, and explaining what was said, the response was to (1) refer to certain measurements, which are even more irrelevant in this context than they are in audio, and (2) suggest I'm an idiot (which is what many of the skeptics do when we debate the audio issues).

 

[ph0rk]

It is important to specify here, before we begin sketching a straw man, that those of us in favor of the use of rigorous measurement don't rely on such measurements for all choices - if you can listen to two pieces of gear and detect no difference with the naked ear, then there is little need to delve further for purposes of personal enjoyment.

When telling someone else there isn't any difference between those pieces they will rightly say "prove it", and that's where the evidence comes in to play.


Phils: I didn't mean to imply you were an idiot, but I didn't think it was that complicated, either. Given that the discussion was about measurement I thought I'd try some complicated snark. Sorry.