[KeithPhantom]

When researching for measurements I have seen multiple websites and sources that provide these. One of those websites (forum) is Audio Science Review directed by its creator Amir. He is really knowledgeable guy which his posts have taught me a lot of audio processing and measurement, but something I have noticed is how divergent some of his measurements are compared to other websites and/or the manufacturer itself. I know there could be bias or set-up measurements that do not really represent the actual performance of the gear, but there have been some discussion about some equipment that has been measured by Amir and other sources such as Jude and the manufacturers that show significant differences in the measurements that cannot be explained as simple production variations.

Others have explicitly said that Amir hacks the measurements to be worse because he does not like a manufacturer, thing I cannot affirm since I have never seen something that indicates this to be true, but his work can be radical compared to the results others get. I want to express that this is an honest question and I do not have another interest other than knowing what is this about and what could be the reasons for this. I do not want to attack him or anyone involved in this, I just would like to have a better framework of the people doing the measurements. Thanks for your time.

PS: this will be also published in ASR to know their opinion about this issue.

 

[Speedskater]

As the types of tests that Amir uses are the ones available in Audio Precision test equipment and similar to the tests done by others with skill, I would trust his measurements. The 'Audio Science Review' forum is open to discussions about suitable test protocols.

 

[jude]

When researching for measurements I have seen multiple websites and sources that provide these. One of those websites (forum) is Audio Science Review directed by its creator Amir. He is really knowledgeable guy which his posts have taught me a lot of audio processing and measurement, but something I have noticed is how divergent some of his measurements are compared to other websites and/or the manufacturer itself. I know there could be bias or set-up measurements that do not really represent the actual performance of the gear, but there have been some discussion about some equipment that has been measured by Amir and other sources such as Jude and the manufacturers that show significant differences in the measurements that cannot be explained as simple production variations....

You won't get any argument from me that Amir is very knowledgable. We also finally met in person at CanJam SoCal last year, and it was a friendly meeting.

I looked for posts that you may be referring to, and had to search back to just about exactly two years ago to the day.

The discussion centered around measurements of a particular DAC model, the Schiit Audio Yggdrasil 2. I posted measurements of that DAC, and some did look different from Amir's, and there was definitely discussion about that -- and the discussion no doubt got heated at times. I think the biggest debate was about the linearity measurements and methodology. I summarized that back-and-forth discussion about linearity measurements in this post:

https://www.head-fi.org/threads/sch...e-technical-measurements.881867/post-14327606

It's a long read, but on the discussion of linearity in the context of the discussing and arguing then, it's rather thorough.

If you get through that and still feel like reading, here are a couple of other links related to it (and I'm sure you can find more on his forum):

https://www.head-fi.org/threads/sch...e-technical-measurements.881867/post-14332522

https://www.head-fi.org/threads/sch...e-technical-measurements.881867/post-14328947

...Others have explicitly said that Amir hacks the measurements to be worse because he does not like a manufacturer, thing I cannot affirm since I have never seen something that indicates this to be true, but his work can be radical compared to the results others get. I want to express that this is an honest question and I do not have another interest other than knowing what is this about and what could be the reasons for this. I do not want to attack him or anyone involved in this, I just would like to have a better framework of the people doing the measurements. Thanks for your time.

PS: this will be also published in ASR to know their opinion about this issue.

Hacks his measurements? Radical? You trying to stir things up? There was heat under the collars then, no doubt, but that was also two years ago.

I think there's a greater likelihood we were measuring two units that varied. Another gentleman who goes by the username atomicbob installed the Yggdrasil V2 upgrades on his first-generation Yggdrasil himself (as I recall it). (This was two years ago, but I think I'm remembering that right.) I think he and I had measurements that were more similar. Again, perhaps there were unit differences with the unit Amir had.

I do remember questioning the strange appearance of a THD+N measurement that he posted, in which it seemed to me it was likely too bandwidth-limited to show THD+N out to 20 kHz. As it turns out, his bandwidth was indeed set too low for that measure, and I think he corrected it after it was pointed out.

If you're itching to see the heat rekindled on this old issue, you may have more success over in his forum.

 

[bigshot]

The differences were also likely measurable, but far below the threshold of hearing. All equipment has manufacturing tolerances. Numbers on a page can vary, but that doesn't matter if human ears can't hear it.

 

[KeithPhantom]

You won't get any argument from me that Amir is very knowledgable. We also finally met in person at CanJam SoCal last year, and it was a friendly meeting.

I looked for posts that you may be referring to, and had to search back to just about exactly two years ago to the day.

The discussion centered around measurements of a particular DAC model, the Schiit Audio Yggdrasil 2. I posted measurements of that DAC, and some did look different from Amir's, and there was definitely discussion about that -- and the discussion no doubt got heated at times. I think the biggest debate was about the linearity measurements and methodology. I summarized that back-and-forth discussion about linearity measurements in this post:

https://www.head-fi.org/threads/sch...e-technical-measurements.881867/post-14327606

It's a long read, but on the discussion of linearity in the context of the discussing and arguing then, it's rather thorough.

If you get through that and still feel like reading, here are a couple of other links related to it (and I'm sure you can find more on his forum):

https://www.head-fi.org/threads/sch...e-technical-measurements.881867/post-14332522

https://www.head-fi.org/threads/sch...e-technical-measurements.881867/post-14328947



Hacks his measurements? Radical? You trying to stir things up? There was heat under the collars then, no doubt, but that was also two years ago.

I think there's a greater likelihood we were measuring two units that varied. Another gentleman who goes by the username atomicbob installed the Yggdrasil V2 upgrades on his first-generation Yggdrasil himself (as I recall it). (This was two years ago, but I think I'm remembering that right.) I think he and I had measurements that were more similar. Again, perhaps there were unit differences with the unit Amir had.

I do remember questioning the strange appearance of a THD+N measurement that he posted, in which it seemed to me it was likely too bandwidth-limited to show THD+N out to 20 kHz. As it turns out, his bandwidth was indeed set too low for that measure, and I think he corrected it after it was pointed out.

If you're itching to see the heat rekindled on this old issue, you may have more success over in his forum.

Actually it was an honest question. About "hacking" I meant that maybe measurements are set-up to have a certain outcome, but I don't think so in both of your cases (differences can be present just for any change in the measurement methodology, which guys over there just explained me) . I would not want to assume anything without having any evidence to back it up. I respect both of you since you both have taught me stuff I didn't understand, and that's something I'm grateful for.

Jude, I'm a guy that actually wants to learn how his stuff works, from the simplest resistor to the most complex FPGA, and it was itching me the fact that I found differences in the same product not just with the Yggdrasil (I only used it as example since it is the most know case of this), also with other products from different manufacturers. Some differences (I would even say most) are attributed to manufacturing divergences between units, but some were striking. I wondered the reasons, but guys at ASR showed me all the variables that increase the marginal deltas between samples.

I love audio (dreamed to study digital signal processing, but chose finance) and I still have my scientific arm always ready for the flexing (always been good in science). I'm humble, I learn and admit my mistakes if I ever make one. For now, I see that many of my questions have been answered, I can lay my doubts down and move to enjoy the music. By the way, thanks for answering, it is flattering to have the owner of a large forum to actually answer questions and inquires of an unknown,

Keith

 

[KeithPhantom]

The differences were also likely measurable, but far below the threshold of hearing. All equipment has manufacturing tolerances. Numbers on a page can vary, but that doesn't matter if human ears can't hear it.

Well, Ethan Winer establishes that -80 dBFS is a safe threshold of inaudibility (I have blind tested it, but don't have the log, I'll repeat it someday) is inaudible but guys over ASR say anything over -100 or -110 dBFS is way to high.

 

[bigshot]

Check out the videos in my sig. Ethan does a demonstration in them where the most annoying noise he could possibly come up with becomes inaudible under music at about -40dB or so.

A lot of people in audiophilia throw around numbers with no context about what those numbers mean in real world sound. The decibel scale is logarithmic. -40 is massively different than -80dB. If you look at normal music listening levels, 80dB is quite loud. Do you think you could hear something down at 1 or 2dB when Led Zeppelin is playing in your ears at 80dB? When you get up over 100dB, you're close to the threshold of pain and incurring hearing damage.

Science is great, but you also have to keep an eye on how the equipment is actually used in the home to listen to music while sitting on a couch. Real world counts. All it takes to figure out how those numbers relate to reality is a cheap audio app and a few experiments playing around with the settings. You will have a better sense of it all if you hear it for yourself, rather than taking someone else's word for it.

 

[jude]

Actually it was an honest question. About "hacking" I meant that maybe measurements are set-up to have a certain outcome, but I don't think so in both of your cases (differences can be present just for any change in the measurement methodology, which guys over there just explained me) . I would not want to assume anything without having any evidence to back it up. I respect both of you since you both have taught me stuff I didn't understand, and that's something I'm grateful for.

Jude, I'm a guy that actually wants to learn how his stuff works, from the simplest resistor to the most complex FPGA, and it was itching me the fact that I found differences in the same product not just with the Yggdrasil (I only used it as example since it is the most know case of this), also with other products from different manufacturers. Some differences (I would even say most) are attributed to manufacturing divergences between units, but some were striking. I wondered the reasons, but guys at ASR showed me all the variables that increase the marginal deltas between samples.

I love audio (dreamed to study digital signal processing, but chose finance) and I still have my scientific arm always ready for the flexing (always been good in science). I'm humble, I learn and admit my mistakes if I ever make one. For now, I see that many of my questions have been answered, I can lay my doubts down and move to enjoy the music. By the way, thanks for answering, it is flattering to have the owner of a large forum to actually answer questions and inquires of an unknown,

Keith

Sorry for rushing to judge you a rabble-rouser, Keith.

Welcome to Head-Fi.

 

[KeithPhantom]

Check out the videos in my sig. Ethan does a demonstration in them where the most annoying noise he could possibly come up with becomes inaudible under music at about -40dB or so.

A lot of people in audiophilia throw around numbers with no context about what those numbers mean in real world sound. The decibel scale is logarithmic. -40 is massively different than -80dB. If you look at normal music listening levels, 80dB is quite loud. Do you think you could hear something down at 1 or 2dB when Led Zeppelin is playing in your ears at 80dB? When you get up over 100dB, you're close to the threshold of pain and incurring hearing damage.

Science is great, but you also have to keep an eye on how the equipment is actually used in the home to listen to music while sitting on a couch. Real world counts. All it takes to figure out how those numbers relate to reality is a cheap audio app and a few experiments playing around with the settings. You will have a better sense of it all if you hear it for yourself, rather than taking someone else's word for it.

That is what I'm thinking about, even if they are not frequency related or even close, the masking effect by loudness differential should be more enough to mask the weaker signal for virtually every listener (assuming in my ignorance your example). Harmonic distortion is usually related to the fundamental, so it is to be expected to be masked at those levels. This is something I would like to have more research about.

PS: I have watched Ethan's videos already, very informing.

 

[KeithPhantom]

Sorry for rushing to judge you a rabble-rouser, Keith.

Welcome to Head-Fi.

Don't worry, I have been a lurker for years (mind you, 2014?). Thanks for the welcoming.

 

[castleofargh]

If you provide a few examples of such differences it could help make sense of what might have happened(or if it's even worth looking at the deviation depending on the magnitudes measured). and if you think there are trends, having the ability to see them would surely help us make our mind.
I assume that if Amir or some other guy made a mistake or had equipment issues, they'd like to find out about it. I don't imagine people bother to spend so much time following measurement protocols and getting measurement equipment(some really expensive) just to mislead others. Usually when they do, I'd think it's unwillingly and unknowingly.
We're all just humans, even at my so very amateurish level, I can't count the number of times I've measured something under the wrong conditions/settings. And those are the times I noticed!!!! It's even harder to pick up on a problem/mistake when testing a device we just got and know little about. If I go measure stuff I've measured 30 times, of course I'll tend to notice anything out of the ordinary, but with gears I just got... I'll be more likely to assume that it's how the device behaves and not question my results too much.

 

[KeithPhantom]

If you provide a few examples of such differences it could help make sense of what might have happened(or if it's even worth looking at the deviation depending on the magnitudes measured). and if you think there are trends, having the ability to see them would surely help us make our mind.
I assume that if Amir or some other guy made a mistake or had equipment issues, they'd like to find out about it. I don't imagine people bother to spend so much time following measurement protocols and getting measurement equipment(some really expensive) just to mislead others. Usually when they do, I'd think it's unwillingly and unknowingly.
We're all just humans, even at my so very amateurish level, I can't count the number of times I've measured something under the wrong conditions/settings. And those are the times I noticed!!!! It's even harder to pick up on a problem/mistake when testing a device we just got and know little about. If I go measure stuff I've measured 30 times, of course I'll tend to notice anything out of the ordinary, but with gears I just got... I'll be more likely to assume that it's how the device behaves and not question my results too much.

Well, I'll post my link of one of my entries that show some differences. They already have explained then and Jude also explained me with some more information, but thank you for caring about this personal question of mine.

Link to the thread: https://www.audiosciencereview.com/...t-compared-to-other-sources.14435/post-443604

 

[bigshot]

That is what I'm thinking about, even if they are not frequency related or even close, the masking effect by loudness differential should be more enough to mask the weaker signal for virtually every listener

The human ear also adapts to volume levels, so if you are listening to loud music, even after the music stops, you won't hear a soft whisper as well as if you were in a quiet library. Thinking about it, your -80dB number might be correct if the ears have time to acclimate to total silence, but that wouldn't be the figure if you are listening to music. It's important to take the way you listen into account when you refer to figures too. A number in one context might not apply at all in another.

For comparison, the best an LP can produce in terms of signal to noise is about -40 to -50dB, and that can sound very good. If you had a totally silent LP with a very good pressing, you could probably barely hear the surface noise at normal listening volume, but you wouldn't hear the surface under music.

Good enough is never good enough for some folks. CDs were designed to be completely transparent, and they are. But that didn't stop audiophiles from buying SACDs or arguing about whether 24/192 sounds better than 24/96. If you hold up a bigger number, someone will say they can hear the difference because their ears are "trained" and they know a whole lot of stuff about it. The truth though is that human ears are human ears. They hear what human ears can hear and that's all.

The guys throwing around all kinds of specs rarely have a clue how those specs compare to the specs on their ears.

 

[Blackwoof]

Good enough is never good enough for some folks. CDs were designed to be completely transparent, and they are. But that didn't stop audiophiles from buying SACDs or arguing about whether 24/192 sounds better than 24/96. If you hold up a bigger number, someone will say they can hear the difference because their ears are "trained" and they know a whole lot of stuff about it. The truth though is that human ears are human ears. They hear what human ears can hear and that's all.

The guys throwing around all kinds of specs rarely have a clue how those specs compare to the specs on their ears.

Same with people that care way to much about transducers doing 18 ~ 92KHz, Then act as if the 17KHz of the ER4 is bad. When the avg adult human hearing range stops at 16KHz which shouldn't be a shock when Etymotic makes hearing aids too.

 

[bigshot]

Yes. The core frequencies are what determine sound quality, not the frequencies at the extreme bleeding edges of human hearing. I remember people on the other side of Head Fi saying things like "The last 5% of sound quality is the most important." That is not true at all. It shows how little they know about it.