Testing audiophile claims and myths

[71 dB]

I can tell you that nothing has changed.

Interesting. I started using the internet in 1997 (somewhat late, but I am slow with new things) and what the internet was back then was COMPLETELY different from what it is today. That said, I didn't use discussion forums until about 2004 I think because I was scared to express my opions (this fear went away very fast once I started posting). The forums I spent time on had a different vibe. Moderation was different. Instead of having for example forbidden topics (politics, religion etc.) personal attacks where moderated, but as long as you didn't attack someone, it was free speech. About 10 years ago or so I started experiencing forbidden topics. Now that I think of it I felt in the past people represented themselves online while today people are representing ideologies. Social media has created large bubbles where a lot of people are living when in the past it was more like everybody having their personal bubble.

 

[Elegiac]

His study papers don't seem bad, per se, but they're very self-published. A total of 4 references would have had anything I'd ever written laughed out of its submission box. Maybe I have a student mentality in my perspective, and this is the prerogative of age, experience and zero oversight. Also I'm not familiar with an 'average' for this discipline, but looking over a couple of things that Castleofargh has shared as I sit here, they have a few more references than that.

 

[castleofargh]

It's one DAC. There might be a thousand others that cause more audible changes, or not. But about the general idea that an audibly transparent DACs is audibly transparent, well, yeah. As he said, it's the literal meaning of the words.
There is a question of when to set the switch between the original track and the loop back version. If there were moments susceptible to help discrimination, then those should be searched and then the switch should happen near those moments. Switching at less different moments doesn't help discrimination and doesn't count as a test trying to disprove the null hypothesis, IMO. Of course, this assumes there are audibly different passages, and they have been identified. IDK if they even exist in that test, probably not.
And I guess, at a level of scientific rigor that I don't possess, choosing the range of value for the answer to the poll, is having its own psychological influence on the results. Like if there had been only 3 cuts, or if the poll only had 10 options instead of 20, how much of the result would strictly follow statistics, and how much would follow some psychological bias about that 10 instead of 20 initial value given to the listeners? I need to say that I do not have either the answer, nor a solution for the ideal amount that least influences listeners in their response.

As it's a result that doesn't disprove the null hypothesis anyway, I don't feel compelled to scrutinize it too much. It doesn't bring much conclusive power for a more global question about the sound of DACs, so it doesn't matter as much as a test with a positive result.
I can just say that I'm not against the loop back approach, I tend to do that myself when it's too difficult to test something. And after a gazillion people crying about abx being made to fail listeners(usually while not understanding what it's about), showing the same non result another way is a good idea IMO.

 

[Androxylo]

Another dangerous topic, I am not aware if it was already discussed but there is a fresh new event ongoing outside of this board.
We all know that opamps make a difference, it could be subtle, it could be significant, depending of the synergy and the capability of the chain. Assuming we all agree on this, I want to ask why they sound different, going as technical as possible? And, how to measure this difference?

Just few days ago the owner of the audio science board (I don't want to post a link) made a quite flamboyant and arrogant statement that all opamps sound the same because the distortion and noise with a sine wave and a sweep are identical. The post triggered a long discussion that I would call abusive. A very small minority of participants carefully voiced their own positive experience with rolling but were attacked by multiple people at once saying this is delusion and affirmative bias. There is a strong consensus there that nobody can hear thew difference. And that consensus is very far from polite, rather sounds like a religion.

I know some physics but I am not an electrical engineer, I cannot answer this question myself - how to measure the whole amplifier with opamps plugged in the way it will actually expose the audible difference - in soundstage, details, dynamics, emphasis of certain frequency range? How to finish this discussion once and forever with hard facts, not just listening?

 

[BS5711]

The problem with people "hearing" obvious differences with op-amps or anything else that is contentious is that the ability to hear the changes is very rarely ever put to the test, is indeed very much subject to bias, and with no starting point of confirmed audibility there is no basis to start questioning the science and the measurements.

Instead people that bother blind testing inevitably fail and then look to explain why the blind testing is wrong. The chap at the "Passion for Sound" You Tube channel seems to be a recent case in point and is now subject to critique by some of his peers since he seems so convinced that his perception could not be faulty and has gone to some lengths to provide some sort of explanation on a recent video as to why blind testing overall was fundamentally flawed. Unfortunately it seems the logic used against blind testing was full of holes as his initial assessment of what he actually hears versus perceives.

I don't claim to be any kind of an expert but the blind testing I have done is hugely eye opening and completely changed my perception of apparent differences between gear. Also my listening experiences day to day are quite variable which further convinces me that what differences I might think I hear today are likely to be inconsistent tomorrow.

 

[sander99]

Another dangerous topic, I am not aware if it was already discussed but there is a fresh new event ongoing outside of this board.
We all know that opamps make a difference, it could be subtle, it could be significant, depending of the synergy and the capability of the chain. Assuming we all agree on this, I want to ask why they sound different, going as technical as possible? And, how to measure this difference?

Just few days ago the owner of the audio science board (I don't want to post a link) made a quite flamboyant and arrogant statement that all opamps sound the same because the distortion and noise with a sine wave and a sweep are identical. The post triggered a long discussion that I would call abusive. A very small minority of participants carefully voiced their own positive experience with rolling but were attacked by multiple people at once saying this is delusion and affirmative bias. There is a strong consensus there that nobody can hear thew difference. And that consensus is very far from polite, rather sounds like a religion.

I know some physics but I am not an electrical engineer, I cannot answer this question myself - how to measure the whole amplifier with opamps plugged in the way it will actually expose the audible difference - in soundstage, details, dynamics, emphasis of certain frequency range? How to finish this discussion once and forever with hard facts, not just listening?

1. It is not the opamp in itself but the total amplifier design that determines the sound.
2. Audibly transparent (audibly perfect) amplifiers are easy and cheap to make for decades.
3. Audibly better than audibly transparent is not possible.
4. If changing the opamp audibly changes the sound of an originally audibly transparent amplifier then it is an objective downgrade. Why on earth would someone want that?
5. It is of the utmost importance to understand that human hearing perception is a construction by the brain depending on many things besides only the actual sound going into the ears and that because of that it is possible to "hear" (in fact perceive) (even night and day) differences even when there is actually no audible difference in sound.

Only people who are not aware of the "audibly transparent" concept and the fact that it is easily achievable in certain parts of an audio system (amps, dacs, cables, most electronics), and don't know that many reported audible differences are in fact non-existent but actually differences in sound perception that can be caused by many different things, could think there could be any gain in things like changing opamps. And of course it may feel 'flamboyant and arrogant' to such people when they are told the actual facts for the first time and they don't fully grasp them.

 

[XTF1]

We all know that opamps make a difference, it could be subtle, it could be significant, depending of the synergy and the capability of the chain.

Ain’t a good start… What makes you think “we all know” opamps make a difference?

Assuming we all agree on this, I want to ask why they sound different, going as technical as possible?

I’ll be the first one to disagree: opamps response/behavior is almost entirely defined by the surrounding electronics components—what defines the “boundaries” in which the opamp is going to be used. These are not changing when you ‘roll’ an opamp, and unless an opamp is widely inadequate for these intended boundaries (i.e. misused), it’s not going to make any difference.

how to measure the whole amplifier with opamps plugged in the way it will actually expose the audible difference - in soundstage, details, dynamics, emphasis of certain frequency range?

That’s the problem: no measurement is going to prove something if this something doesn’t exist.
We all know the Graal exists… What you’re asking is for this thread to pinpoint its exact location, so you’re proved right once for all…

 

[VNandor]

how to measure the whole amplifier with opamps plugged in the way it will actually expose the audible difference - in soundstage, details, dynamics, emphasis of certain frequency range?

A null test done by subtracting one of the output from the other will reveal all differences between the outputs, audible or not. If the output signals are the same, the result of the subtraction will be near silence, hence the name "null" test. It won't reveal differences in price, marketing fluff and the listeners' preconceptions of the amps though.

 

[Androxylo]

Are those null tests possible, anyone did them with published results? This seems like a way to go, but I never saw it.

 

[2leftears]

We all know that opamps make a difference, it could be subtle, it could be significant, depending of the synergy and the capability of the chain. Assuming we all agree on this, I want to ask why they sound different, going as technical as possible? And, how to measure this difference?

Wrong assumption, we don't all agree on this.

I have rolled opamps myself. I thought I heard a difference but I have no reason to assume that was anything other than perception bias. That was years ago and TBH I didn't do any ABX or null test, just a few basic measurements that showed some of the 'compatible' replacement opamps to be problematic in part of the circuit.

What I learnt in the process is that "audiophile" opamps being marketed as a drop-in replacement for well-known and often-used "consumer-level" opamps, are sometimes anything but!

I have lost count of the YouTube videos where folks play around with op-amp rolling, no ESD prevention measures, adding long stand-offs, no checking for stability, no checking for DC drift, etc. etc. And then they (almost invariably) claim the audiophile opamps to sound better than the mass-produced consumer one. Sure...

Completely independent of the question whether rolling opamps actually makes an audible difference, I have seen the rolling process done wrong so often just because the supplier states it to be "compatible with X,Y,Z". Given that folks generally seem keen to replace the 'slow' integrated consumer opamp with a much 'faster' discrete audiophile one, one would have thought the need to check for circuit stability and temperature-driven DC drift would spring to mind, but apparently not.

And folks, please, ESD protection . Many opamps such as e.g. the OPA6x7 series with FET inputs have only limited built-in ESD protection, and need to be handled with appropriate care.

 

[71 dB]

We all know that opamps make a difference,

Incorrect statement. Many of us know opamps don't make an audible difference when used correctly.

I want to ask why they sound different, going as technical as possible? And, how to measure this difference?

If an opamp sounds different (not transparent), the circuitry around it has been badly designed. It is not the opamp, it is bad engineering. If things have been engineered well, the opamp will sound audibly transparent (the same as every other audibly transparent opamp).

Just few days ago the owner of the audio science board (I don't want to post a link) made a quite flamboyant and arrogant statement that all opamps sound the same because the distortion and noise with a sine wave and a sweep are identical.

The statement may have felt arrogant and flamboyant to you, but such a statement is just as correct as saying 2+2=4. People who know what they are talking about may lack emotional intelligence and especially patience to say things in a way that doesn't hurt the feelings of people who know less, but the facts are facts.

The post triggered a long discussion that I would call abusive.

That wasn't the first time such a thing happened online.

A very small minority of participants carefully voiced their own positive experience with rolling but were attacked by multiple people at once saying this is delusion and affirmative bias. There is a strong consensus there that nobody can hear thew difference. And that consensus is very far from polite, rather sounds like a religion.

Being polite is not the point when we are discussing the facts. People who know get inpatient with stubborn people who are not willing to learn.

I know some physics but I am not an electrical engineer,

I happen to have M.Sc degree in electrical engineering. I am not an expert in designing circuits utilising opamps, but I know what it is about. I say this because someone with a degree doesn't mean the person is an expert in everything included in the degree. The amount of knowledge is insane (hundreds of years worth of cumulated scientific knowledge of humankind). People who haven't studied these things don't realise how clueless they are. What I learned in the university is how little I know and understand.

Do you know how the open loop gain and bandwidth of an opamp affects the closed loop gain? These are the things electrical engineers may need to consider (depending on the application), not how different opamps sound.

I cannot answer this question myself - how to measure the whole amplifier with opamps plugged in the way it will actually expose the audible difference - in soundstage, details, dynamics, emphasis of certain frequency range? How to finish this discussion once and forever with hard facts, not just listening?

I can't really answer that, because that's a badly formulated question. We can measure amplifiers objectively and find out how much distortion they have for example. Then we can compare those figure to known thresholds of audible levels of distortion (not that simple actually, because the type of the distortion and frequency range affect a lot) and have an idea of how audible the distortion can be. If there is a possibility the distortion is audible, then proper DBTs can be done to know if that's the case.

 

[gregorio]

We all know that opamps make a difference, it could be subtle, it could be significant, depending of the synergy and the capability of the chain. Assuming we all agree on this, I want to ask why they sound different, going as technical as possible?

We should all agree that opamps make a difference, everything (or even nothing) makes a difference at some level. However, you then jump to “sound different”, quite a big leap for someone who knows a bit about physics. Opamps operate on an analogue electrical signal, sound on the other hand is a mechanical/kinetic phenomena, physical vibrations/waves travelling through a medium (air in our case). We can quite easily measure differences in analogue signals down to about -120dB and with some math trickery (FFT gain) built-into high quality measuring equipment, differences down to around -160dB to -170dB. However, according to physics, such differences cannot exist as sound in any practical application. If we take a moderately loud but reasonable (not ear damaging) peak level of say 80dBSPL, then a difference measured at say -110dB would therefore be at -30dBSPL but physics dictates that absolute silence (in air) is around -23dBSPL, so our -30dBSPL difference cannot exist as sound, it’s less than half the level of absolute silence! So any question of audibility is moot. In practice of course, speakers/HP do not have infinite resolution and neither does human hearing, so any differences would have to be at least one order of magnitude higher to even be potentially audible.

To answer your question, they don’t sound different, unless they’ve been deliberately (incompetently) designed to sound different.

And, how to measure this difference?

A null test as VNandor suggested will detect all differences. A good measuring device (AP555 for example) will measure specific differences, noise, distortion, FR, etc.

Just few days ago the owner of the audio science board (I don't want to post a link) made a quite flamboyant and arrogant statement that all opamps sound the same because the distortion and noise with a sine wave and a sweep are identical.

That’s neither flamboyant nor arrogant, it’s entirely in line with science and with countless objective measurements of opamps.

How to finish this discussion once and forever with hard facts, not just listening?

This discussion was finished once and forever, opamps were invented in the 1940’s and the discussion was over around half a century ago. The only place it still exists is in the audiophile community and that’s because the audiophile community is driven by the marketing of audiophile equipment manufacturers, many of whom rely on misrepresenting “this discussion”. To justify adding two or three zeroes to the price of an OpAmp, you need something other than “they all sound the same”!!

G

 

[colonelkernel8]

Another dangerous topic, I am not aware if it was already discussed but there is a fresh new event ongoing outside of this board.
We all know that opamps make a difference, it could be subtle, it could be significant, depending of the synergy and the capability of the chain. Assuming we all agree on this, I want to ask why they sound different, going as technical as possible? And, how to measure this difference?

Just few days ago the owner of the audio science board (I don't want to post a link) made a quite flamboyant and arrogant statement that all opamps sound the same because the distortion and noise with a sine wave and a sweep are identical. The post triggered a long discussion that I would call abusive. A very small minority of participants carefully voiced their own positive experience with rolling but were attacked by multiple people at once saying this is delusion and affirmative bias. There is a strong consensus there that nobody can hear thew difference. And that consensus is very far from polite, rather sounds like a religion.

I know some physics but I am not an electrical engineer, I cannot answer this question myself - how to measure the whole amplifier with opamps plugged in the way it will actually expose the audible difference - in soundstage, details, dynamics, emphasis of certain frequency range? How to finish this discussion once and forever with hard facts, not just listening?

I'm sorry, but this is just pure delusion. There was nothing arrogant about the results of the test. Op-amps, if they fall within the spec of the circuit they're being dropped into (which isn't guaranteed), aren't going make an audible difference to the sound, they barely make a MEASURABLE difference to the sound with an Audio Precision 555 series analyzer, a device literally tens of thousands of times more sensitive, precise and accurate than your ears.

 

[Elegiac]

I have had trouble with rolling op-amps. I'm not technically proficient in circuity, so I can't tell what is really happening. The best outcome I've had is putting Burson discrete op-amps in a Burson Funk, because Burson would create their units with compatibility in mind, so that people can spend more money and upgrade.

All I can do is listen and figure it out, and what I've learned to do (with music I'm very familiar with and over an extended period of time) is compare the op-amp I've rolled in to the stock performance of the unit (the Geshellli J2S Dac) and find if it's not linear in frequency response, in presentation. Or it's smearing things, or distorting. The theory being that every op-amp should be linear and transparent (according to its ability to resolve detail). The most basic op-amps seem to be the ones that can be rolled easily, like an OPA2134. MUSES8820 distort audibly. MUSES03 aren't linear, and seem to smear, which in my layman's understanding may have something to do with oscillation. I'm running the unit stock at the moment, but I'm going to try two Burson v5i-D's in there shortly. I know it's foolish, but I'm hoping they'll be linear and that their relatively poorer... SINAD(?) will result in a more pleasing sound.

 

[2leftears]

I have had trouble with rolling op-amps. I'm not technically proficient in circuity, so I can't tell what is really happening. The best outcome I've had is putting Burson discrete op-amps in a Burson Funk, because Burson would create their units with compatibility in mind, so that people can spend more money and upgrade.

All I can do is listen and figure it out, and what I've learned to do (with music I'm very familiar with and over an extended period of time) is compare the op-amp I've rolled in to the stock performance of the unit (the Geshellli J2S Dac) and find if it's not linear in frequency response, in presentation. Or it's smearing things, or distorting. The theory being that every op-amp should be linear and transparent (according to its ability to resolve detail). The most basic op-amps seem to be the ones that can be rolled easily, like an OPA2134. MUSES8820 distort audibly. MUSES03 aren't linear, and seem to smear, which in my layman's understanding may have something to do with oscillation. I'm running the unit stock at the moment, but I'm going to try two Burson v5i-D's in there shortly. I know it's foolish, but I'm hoping they'll be linear and that their relatively poorer... SINAD(?) will result in a more pleasing sound.

I'm not sure why folks talk/worry about linearity of opamps.

In an idealised model of an opamp, the open loop gain is infinite, and linearity is not applicable (well, in theory mathematically you could still assess linearity of an infinite gain if you take the limit of gain to infinity ). In practice the gain is not infinite but very high (some 100dB or higher), and possibly far from ideal linear.

In almost every application of opamps I have ever seen it is the feedback circuitry implemented around the opamp that ensures sufficient linearity, utilising the opamp's extremely high (and not necessarily linear) gain. Pretty much any opamp with sufficient gain inserted in that circuit will result in that circuit exhibiting a more than sufficient degree of linearity.

The trouble you have had with rolling opamps may have been one of stability, or with temperature drift. This may sometimes cause audible issues, but in many cases it may go unnoticed unless the opamp actually runs hot and fails. Really, unless one drops in an opamp guaranteed to be compatible with the circuit (like in your Burson example), some basic measurements confirming stability are in order.

One issue I have encountered myself is with the relatively poor matching of the differential input pair in some discrete opamps. IC opamps have a natural advantage here because the majority of them are of monolithic construction as well as the inherent far better ability to maintain the input pair at a uniform temperature. Hence IC opamps generally offer far superior behaviour with respect to temperature drift and DC offset, which is why so many discrete opamps feature one or two trimpots whereas IC opamps generally don't need them. This can be important where e.g. the opamp is used in a I/V (current to voltage) configuration where the compliance (allowed voltage pull-up) of the current source is small.