You certainly are an audio politician. You basically admitted it to
@Phronesis a little while back. You often don't respond directly to questions or comments, rather you use them as a springboard to get on your soapbox and provide your talking points.
I was asking for, and hoping to get, very specific details about your last comparison test, not a general answer about all of your tests. The reason is that the set of all "blind listening tests" is not a homogenous set, with all the details being basically the same. Some blind listening tests are very easy to perform, and with a paragraph or two instruction could be done well by just about anyone. Some are difficult to perform and require special hardware and/or software, and quite a bit of know-how. Many are not too hard, if you know what you're doing.
The problem I have with the long list of comparisons you say you've performed is the risk of expectation bias or confirmation bias.
We ask people to do blind tests when they tell us they hear a difference between A and B in order to prevent such a bias. Even if we believe them to be honest and sincere, we know that the mere knowledge of what they're hearing can influence their perception, subconsciously! Asking for blind tests is not an accusation of lying or incompetence.
But such biases are not one-way. Just as hearing a difference when there is none can occur, so can not hearing a difference when there is one.
I know, I know... you always give some form of your standard disclaimer; here it's "I'm not looking for differences at the bleeding edge of audibility", and that's fine. You are free to do listening tests however you like, but your preaching that others should do them to high standards, if you aren't, is disingenuous.
So let's see what you said:
1- I just have a consumer SPL meter for my speaker system. A friend of mine is an audio engineer and he brings over equipment when I need it (tone sweep generator, volt meter, etc.) or I go over to his shop and lug whatever I want to test. I let him handle all the technical stuff. We've done a bunch of test together...
2- CD vs SACD,
3- compressed audio vs lossless,
4- different DVD and blu-ray players, ... {snip}
5- ...For quick tests, like when I'm replacing one iPhone with another or to double check if I suspect something has changed, I will level match with tones by ear and do a sighted test, but that isn't when I'm doing careful comparisons. ...{snip}
6- Level matching really isn't that hard. In fact, it goes so quick, I don't pay much attention to it.
7- I'm not looking for differences at the bleeding edge of audibility. I'm looking for clear differences in the range of 1-2dB or more, so it isn't a big deal. ...{snip}
8- ...testing takes just a couple of minutes. We switch for each other, but 90% of the time, neither of us can hear any difference from the start, so it doesn't take long at all.
1- OK, so some tests require special equipment and expertise that you lack. There's nothing wrong with that, and I think it's great you have such a friend. But then you shouldn't hand-wave and tell others to "just do a blind listening test and provide some evidence", without warning them of what they'll need. It is particularly important to warn them, if you don't even know what to tell them to do!
2- How did you synchronize? How did you level-match? How did you deal with different masterings? Keep in mind that in other threads you have said this is a problem (without offering a solution).
3- This is the easiest (not the quickest, as you mentioned). Did you use foobar with the ABX plugin or Mac software (which)? You mentioned a lot of Macs... do you also have a PC? ...or your friend? Do you know if Parallels would work?
4- see #2
5- No problem. When quick and dirty is called for, it is very appropriate. After all, it is quicker, but also dirtier... ,but don't jump on others for doing the same.
6- Sometimes it really easy (like you say), but sometimes not. Your flippant answer calls into question your familiarity with it.
7- Ah yes, your disclaimer that makes so many of your posts like Jello. If anyone picks up one of your answers to look at it, you throw this out there, or something similar, and slip right out of their fingers. It's a nice technique.
8- This is the biggest reason I suspect confirmation/expectation bias. You don't expect a difference, so your effort suffers. If you really could hear a difference, do you take the time to really try? Your carefree level-matching and looking for differences more than 1-2db could mean you do hear a difference, but it doesn't meet you standard of being strong enough, so you dismiss it as "both are audibly transparent".
