[bigshot]

You aren't reading what I am asking. I'll state it clearly. Did you...

1) set up a switcher to directly A/B switch between the two inputs while they were playing the music same file in the same spot
2) level match the output of the two players so one wasn't a tiny bit louder than the other
3) have a friend switch between them and record your impressions of each as he switched back and forth

This is the next step in being a Sound Science person. Just answer yes or no to 1, 2 and 3. If you answer that, I'll tell you why people say that the 105 is more "audiophile" than the 103.

 

[KeithEmo]

You're not taking that nearly far enough.

There is a MAJOR human bias to not admit TO OURSELVES when we make a mistake.
Therefore, we ALL have a bias to continue to believe what we've decided is true (if we change our mind then we must have previously been WRONG).
That bias becomes even stronger when we share our claim in public.
And we each also have a strong incentive to NOT admit to ourself, or anyone else, when we make a foolish decision.

However, note that this bias applies to everyone, and in all directions:
- Someone who bought the expensive one, because he hears a difference, would feel like an idiot if he turned out to be wrong.
- Someone who bought the cheap one, because he's sure there is no difference, would feel like just as much of an idiot if HE turned out to be wrong.

This presents an interesting dilemma....
- There are many ways to eliminate the effects of "positive bias".
- However, it can be difficult to eliminate the effects of "negative bias".
(How do you eliminate a bias that causes someone to ignore small differences that actually exist?)

Not when there's several obvious reasons not to take their word at face value. Most people have the tact to not tell you that they think you wasted your money. There's incentive to tell a white lie if one doesn't actually hear a difference.

 

[Jaywalk3r]

You have maintained an hypothesis can only be rejected or not rejected...while the link clearly states: "Upon analysis of the results, a hypothesis can be rejected or modified, but it can never be proven to be correct 100 percent of the time."

It can be, and often is, modified from one study to the next, but never within a single study. Rarely, outside of medicine, is much confidence placed in any single study, so it is also common for hypotheses to remain unmodified between researchers, so they can independently verify the results.

 

[bigshot]

This presents an interesting dilemma....

You really think it's interesting? If you aren't going to answer my question, please say that so I don't have to keep hounding you.

Jaywalk3r, you realize that they are tag teaming you to pull you off point, don't you?

 

[Davesrose]

Not when there's several obvious reasons not to take their word at face value. Most people have the tact to not tell you that they think you wasted your money. There's incentive to tell a white lie if one doesn't actually hear a difference.

No, my aunt, the opera singer does not mince words. She found my Audeze headphones way too heavy, but stated the system sounded magnificent and more lifelike then anything she's heard. She's not the sort to tell me a white lie about this...

 

[Davesrose]

It can be, and often is, modified from one study to the next, but never within a single study. Rarely, outside of medicine, is much confidence placed in any single study, so it is also common for hypotheses to remain unmodified between researchers, so they can independently verify the results.

My previous post said you publish your results and then can change....that's very different then claiming you change your hypothesis during the same phase of a study.

 

[bfreedma]

To answer the simplest question first...
For measuring basic specifications like S/N, frequency response, and distortion, we use Audio Precision model 585 testers.
(It's a ridiculously expensive, very cool, computer controlled test set that does a whole bunch of standard tests, and can be run off a test script; we have several.)

Many companies like to pretend that they have some esoteric design process... we really don't.

First we decide what the product should do.
If we want to build a product that supports DSDx2 and PCM up to 384k, we'd better choose a DAC chip that can do it.
(We find that there are plenty of commercial DAC chips available that perform very well, and sound very good, so we're going to pick one to use.)
To be completely honest, we often base our operational parameters on the hardware that's available.
(For example, in terms of the market, support for 192k is absolutely required, 384k is strongly desired, and 768k will impress a few people.)
The latest chips by the major vendors all pretty much have the features that the market is looking for this year (they've already sorted that all out).
We may choose which particular chip based on specific features, cost, availability, or even simply which brand the design engineer prefers.

Then, based on that requirement, we have our engineers come up with a prototype that offers the features we want.
For example, our new DC-2 will do PCM and DSD, will have a WiFi input, and will have a subwoofer output.

We run it through the basic scripted test to confirm that the standard measurements are all "really good"...
And, if anything sticks out as bad, we go back to the drawing board, figure out why, and fix it.

Then a bunch of us LISTEN to it, with a variety of material, and see if we all agree that it sounds good.
(If anyone thinks that it sounds "off" - then back to that drawing board to figure out why.)
This step is usually somewhat informal...
We set it up in one of our home theater systems and listen to it... then we pass it around.
I may listen to it in my office, or at home with headphones, or both...
Someone else may listen to it on their office system...
We compare it directly to at least a few of our previous products...
And usually compare it to a few competitors' products...
We always design our products to sound neutral, so we do not compare them to dozens of competitors' products...
Our goal is to design a product that performs well and sounds good... and not to "beat" anyone else.
Depending on the circumstances, we may invite some members of the public, or some of our insiders, to audition them.
(We had the prototype of our DC-2 available for people to listen to, and compare to the old model, at our Emofest public event; most people preferred the new model.)

To be quite honest, all of our DAC products sound more similar than different.
Our more expensive products generally have more impressive specifications than our lower-cost products.
(The DC-1 has much better measurable specs than the Little Ego.)
And, based on our listening tests, we generally find the DC-1 to sound slightly better than the Little Ego (but it also has a lot more features).
(Most people agree that they hear a difference; most prefer the DC-1; a few prefer the Ego; and, yes, they do use different DAC chips.)

If, by "operating parameters", you mean things like THD and S/N, then the design goal is really simple.
Pick a good DAC chip with good specs from the manufacturer...
Build it into a competent design (following the manufacturer's suggestions and certain well known "best practices").
If you haven't screwed something up then it will perform at or near the manufacturer's specifications.
And, if so, then it should sound both "neutral" and "good".

Those "best practices" include
- providing well-regulated and isolated power supplies for certain parts of the circuit
- laying out the circuitry properly, with proper grounds, and proper care in foil layout
The details can be quite complex... but they aren't especially exciting.

And I mentioned, our DACs tend to sound more similar than different...
There are some really subtle differences...
For example, the DC-1 sounded exceptionally clear and detailed (quite possibly because it had a very sophisticated multi-section power supply).
And, if you switch filters on the Ego DACs, most people hear a subtle differencebetween the various settings (but I wouldn't venture to say which one is "better"; I prefer #2).

On some of our higher-end uints, every unit is tested (using one of those AP 585 testers).
On others, after we confirm that the production units consistently pass, we may simply spot check them.
In general, we do not see "variations" in performance, or minor variations in sound; either they perform to spec - or they're broken (and need to be repaired).
The design sets both the measurable performance and the sound.
(There's no point in confirming that a certain filter on a certain chip delivers the exact waveform the manufacturer says it does.... it will.)

Thanks for the detailed answer, though it could have left out the rather condescending description of the AP 585. I’m well aware of what it is.

I’m still a little lost on the testing methodology described. Still sounds highly subjective - that said, based in the description and desire to stay neutral, it appears as though your engineering team beliieves a variety of DAC chips to be audibly transparent and select them to meet the capabilities required in the end product. That makes sense. Where I’m getting lost is how these transparent DAC chips selected based on product requirements and not audible differences somehow produce audibly different output.

I’m inclined to belive that despite the subjective testing results you describe, if the products were tested via a proper blind test/ABX, they would be indistinguishable.

I understand the need to differentiate product for sales purposes, so I’ll stop here.

 

[KeithEmo]

I'm sorry, but I suspect you haven't been much involved in the "audiophile equipment sector"...
People buy products for a wide variety of reasons...
And, when it comes to luxury products, like "audiophile products", and perhaps especially audiophile products, "objective value" is pretty far down the list.
And, of course, "objective value" means very different things to different people.
For example, many people base their choice on "the reviewers liked it" (so, to them, "reviewer preference" is their "objective criterion").
Others buy 'their favorite brand".
And yet others buy it "because their friend liked it".
And, yes, a lot of them have no preference at all, and simply buy the cheapest one, or the first one the search engine pops out.

If people REALLY were objective, nobody would ever buy an amplifier that cost more than $2000....
However, as you'll note, MANY of them do....

Apparently your problem with my logic is that I use it while you don't.

You're not the only person to have experience in the for-profit private sector. Yes, I get that one can sell more snake-oil without credible research. Baffle them with B*ll****. What you're missing is that when a company isn't selling snake-oil, but a legitimate product, it is in their interest to credibly demonstrate the objective value of their product. You keep ignoring that last bit.

 

[Davesrose]

You aren't reading what I am asking. I'll state it clearly. Did you...

1) set up a switcher to directly A/B switch between the two inputs while they were playing the music same file in the same spot
2) level match the output of the two players so one wasn't a tiny bit louder than the other
3) have a friend switch between them and record your impressions of each as he switched back and forth

This is the next step in being a Sound Science person. Just answer yes or no to 1, 2 and 3. If you answer that, I'll tell you why people say that the 105 is more "audiophile" than the 103.

Love how you have a title "Sound Science person" reserved for particular people. Is there an A/B switch for analogue input for my SACD player and digital input for the Oppo (you yourself have made the generalization that digital input is always better then analogue input)? My evaluation has been changing inputs with my receiver.

 

[Jaywalk3r]

However, note that this bias applies to everyone, and in all directions:
- Someone who bought the expensive one, because he hears a difference, would feel like an idiot if he turned out to be wrong.

Yes.

- Someone who bought the cheap one, because he's sure there is no difference, would feel like just as much of an idiot if HE turned out to be wrong.

Might have regrets, but we wouldn't feel like just as much of an idiot. Most people recognize that mistakes have a degree of magnitude.

The guy who buys a $1200 device only to learn that a $200 device would do just as well realizes he wasted $1000.
The guy who learns that he really does need the $1200 device instead of the $200 device he purchased realizes he wasted $200.
My guess is the second guy gets over the mistake faster.

However, it can be difficult to eliminate the effects of "negative bias".
(How do you eliminate a bias that causes someone to ignore small differences that actually exist?)

Increase your listening audience size if you think the effect is only audible to a certain proportion of the population. If you think people are deliberately sabotage your test, increase listening audience size. I can't think of any other case that doesn't equate to a genuine inability to hear a difference.

 

[KeithEmo]

DAC chips are a rather sophisticated circuit element...

Their output is affected by, among other things:
- the stability and noise in their analog supplies
- the source impedance of their analog supplies
- the amount of jitter produced by the main clock (and how much of it reaches the clock input of the DAC chip itself)
- the amount of jitter introduced by the input circuitry (S/PDIF recievers all have an inherent amount of jitter)
- all sorts of factors that affect the analog circuitry in the overall device

These all produce small, but clearly measurable differences, which also seem to be audible.

Feel free to BELIEVE whatever you like.

Thanks for the detailed answer, though it could have left out the rather condescending description of the AP 585. I’m well aware of what it is.

I’m still a little lost on the testing methodology described. Still sounds highly subjective - that said, based in the description and desire to stay neutral, it appears as though your engineering team beliieves a variety of DAC chips to be audibly transparent and select them to meet the capabilities required in the end product. That makes sense. Where I’m getting lost is how these transparent DAC chips selected based on product requirements and not audible differences somehow produce audibly different output.

I’m inclined to belive that despite the subjective testing results you describe, if the products were tested via a proper blind test/ABX, they would be indistinguishable.

I understand the need to differentiate product for sales purposes, so I’ll stop here.

 

[Jaywalk3r]

I'm sorry, but I suspect you haven't been much involved in the "audiophile equipment sector"...

Perhaps, but I've seen people selling other snake-oils.

People buy products for a wide variety of reasons...

Utility chief among them.

 

[KeithEmo]

But that "degree of magnitude" is both relative and subjective.
If the guy is actually enjoying his $1200 whatzit, he probably doesn't really care how it measures.
(He'll probably say something like: "I like the way it sounds and that's all I care about".)

I personally wouldn't pay $500 for a bottle of wine, but many people do, and consider it a good purchase...
And, you'll find that most of those people really aren't interested in your test report.

Yes.

Might have regrets, but we wouldn't feel like just as much of an idiot. Most people recognize that mistakes have a degree of magnitude.

The guy who buys a $1200 device only to learn that a $200 device would do just as well realizes he wasted $1000.
The guy who learns that he really does need the $1200 device instead of the $200 device he purchased realizes he wasted $200 bucks.
My guess is the second guy gets over the mistake faster.



Increase your listening audience size if you think the effect is only audible to a certain proportion of the population. If you think people are deliberately sabotage your test, increase listening audience size. I can't think of any other case that doesn't equate to a genuine inability to hear a difference.

 

[Jaywalk3r]

DAC chips are a rather sophisticated circuit element...

Their output is affected by, among other things:
- the stability and noise in their analog supplies
- the source impedance of their analog supplies
- the amount of jitter produced by the main clock (and how much of it reaches the clock input of the DAC chip itself)
- the amount of jitter introduced by the input circuitry (S/PDIF recievers all have an inherent amount of jitter)
- all sorts of factors that affect the analog circuitry in the overall device

These all produce small, but clearly measurable differences, which also seem to be audible.

You sounded credible until the bold bit. It would have been more accurate as:

These all produce small, but clearly measurable differences, but there's no reason to believe they're audible.​

It's a justified statement. You're version skips over the all-important process of verifying that the differences are audible before presenting that assertion as near-proven fact.

 

[bfreedma]

DAC chips are a rather sophisticated circuit element...

Their output is affected by, among other things:
- the stability and noise in their analog supplies
- the source impedance of their analog supplies
- the amount of jitter produced by the main clock (and how much of it reaches the clock input of the DAC chip itself)
- the amount of jitter introduced by the input circuitry (S/PDIF recievers all have an inherent amount of jitter)
- all sorts of factors that affect the analog circuitry in the overall device

These all produce small, but clearly measurable differences, which also seem to be audible.

Feel free to BELIEVE whatever you like.

Most of that list should be easily addressed through proper implementation.

As to the “small, but clearly measurable differences”, I don’t doubt that either. Can you show any of those measurements so that it can be determine if they are likely to be audible to a human? Since you have an AP 585, there should be plenty of hard evidence regarding the magnitude of the differences. Jitter, for example, is commonly referenced by audiophiles as the cause of audible differences, yet I can’t remember the last time actual measurements of jitter on a modern component were anywhere near the threshold of audibility.