[RRod]

People will say the same stuff about DBTing sampling rates or lossy formats, which are tests that are much easier to control. It's not about the difficulty of the test, it's that people don't pass the test, and instead of saying "I failed the test", they want to say "the test failed me!"

 

[DavidA]

  People will say the same stuff about DBTing sampling rates or lossy formats, which are tests that are much easier to control. It's not about the difficulty of the test, it's that people don't pass the test, and instead of saying "I failed the test", they want to say "the test failed me!"

Got a friend like this.  He always says its easy for him to tell the difference between MP3 and WAV, I played some songs sampled at 128vbr, 256vbr 320cbr and the original WAV, he was only able to tell the 128vbr sample from the others, but only 80% of the time.  His reasoning, he was not familiar with my equipment and was too used to the sound of his gear (sony Walkman MP3 player-$100, and sony earbuds-$25) where he can easily hear the differences.  My system that I let him listen on at the time: Bifrost Uber, Lyr2 (Telefunken CCa), and HD-700.  After this he never wanted to try again.

 

[gregorio]

  It's not about the difficulty of the test, it's that people don't pass the test, and instead of saying "I failed the test", they want to say "the test failed me!"

 
That's very well put and goes to the heart of the problem with many audiophiles. It's not they may slightly favour subjectivity over objectivity, it's the absoluteness of it all that's shocking, it's just like a religious fanatic. Their belief is absolute and therefore absolutely unquestionable! It doesn't matter what test, scientific principle, proof or even infant level logic is presented, if it contradicts their belief then it's simply wrong, period.
 
To me, this level of extreme audiophilia is just as insane as religious extremism, although fortunately there are no audiophile jihadists. Nevertheless, it still affects us normal, real audiophiles. What could have been achieved if the industry had concentrated on actual audio improvements rather than the ridiculous non-improvements which are easier to sell to the insane extremists? Such as shame!
 
G

 

[castleofargh]

Spoiler: Warning: Spoiler!

Originally Posted by ColtMrFire /img/forum/go_quote.gif
 
<snip>
 
Double blind tests are completely impractical for most people since hi-fi audio equipment can be prohibitively expensive, most people don't have the money, time or wherewithal, so it's [almost] not even worth talking about unless you're an engineer or scientist or just want to appease your sense of paranoia about being ripped off.
 
<snip>

 
You know guys, this statement is not far off the mark. 
 
For all the debating and fighting about word definitions, not many guys, if any, really double blind their equipment, (except of course, the concentration of fastidious double blind testers in this thread).  I have no doubt we all compare our stuff to other stuff we have, but real double blind tests... I don't think so.
 
I've taken part in a few unscientific double blind tested at meets, but it's a real pain in the ass to do at meets and trying to do it by yourself or in small get togethers, is really time consuming and annoying.
 
Never mind having multiple pieces of expensive equipment to test at the same time.
 
I think what we're actually talking about is developing an unscientific preference for one piece of equipment over another.
 
Here's a test I did for a Portaphile 627 portable amp a number years ago: (Not in the picture but included in the test was a Hornet amp)   Did I double blind anything? No but I developed a preference.  The O2 had more resolution than any of the others.
 

 
Here's another test, comparing DAC and Amp combinations with my T1s.  I set the stuff up on a TV stand and went back and forth.  Double Blind?  Nope, but I was able to develop preferences.  GS-1 amp and NorthStar DAC.
 

 
and one where I was testing three different DACs and two amps.  Double blind anything?  Nope.  But I was able to decide what I liked.  Which, btw, turned out to be horse pucky because....
 
In this case, I did run what became almost a double blind test (because I lost track of what was plugged into where) with the Stello and the Neko DACs, and 2 identical Shuttle computers as sources.  As I toggled back and forth on the GS-1s inputs, after volume balancing in foobar, the result was a disappointing, couldn't really tell them apart.
 

 
That last test was such a pain in the neck that I haven't gone to this trouble in a long time.  Maybe I will in the future.
 

 
Simple cable for the DiffMaker
 

 

is the difficulty a reason to say not to do it or to denigrate the results? because ABX is mighty simple to do and you will read the exact same kind of arguments against it. not everybody has good enough gears to run a jitter test, doesn't mean the test does anything wrong.
I stay on my general idea that all blind tests (double or not) are just a method, a tool, to remove some variables or biases. when you try to get information about one particular variable, if you can remove even one other variable, you're probably already one step closer to getting a proper answer. if you can afford a double blind test that removes a lot of variables, then it's even better.
it's what we do when we measure a given variable, we try to make sure we're not inadvertently measuring the wrong one and we try to end up with only the variable we wished to know more about. I have a question, I look for answers. that's about it.
 
 every single step in the direction of removing variables(be it with measurements or blind testing) painfully demonstrated to me how much of a fool I was before. each step has been a humbling experience telling me something like: "don't you go running your mouth, remember last time you thought you had it all figured out you stupid F".
 
how often did you find someone who got himself a switch, did some tests with it, and then came to say switches are useless and he could do and know more without them?
how often did you see someone getting a multimeter or a sonometer, used it to match loudness and check other stuff, and then after some time came and told you it was better to do without and people shouldn't use those to review gears? doesn't happen all that often right. I believe we agree on this. ^_^
but somehow the argument of no tool, no control(the infamous "just listen") vs almost any objective tool is something you can read about almost every single day on headfi. mind blown by the nonsense.
the nonsense of thinking that somehow doing things wrong is better than doing them well, but also the nonsense of pretending like it's a VS thing. why? the guy is going to die in the next 15 minutes so if he starts a blind test he will never be able to do casual listening? what kind of science fiction goes in the mind of people using that kind of logic? we're saying to do more, not to pick 1 thing and forget all about the rest.
 
people suggesting that I do blind tests and use available measurements, they are simply telling me to multiply my sources of information so that I am better equipped to make a decision, or more confident before making a claim. it's not like doing a blind test will stop me from picking the device I find pretty, or the one with the connectivity I could make more use of. when did any extra knowledge stopped me from thinking straight or picking what I like most if that's what I want? and what if a blind test is inconclusive, so what? the earth won't stop spinning. what is so frightening that some people would actively tell others not to trust those tools? when I think about it I always end up thinking about xenophobia. they're just scared of what they don't understand.
 
now proper tests aren't a lot of fun. and double blind test are indeed very hard to set up well. I understand that someone may not want to waste his time, and decides to stay casual doing what he likes for himself(it's in fact mostly what I do myself). but there is really no excuse to try and drag people into our own laziness with BS excuses while still pretending to be some kind of expert. nobody knows more by doing less.

 

[upstateguy]

   

Spoiler: Warning: Spoiler!

is the difficulty a reason to say not to do it or to denigrate the results? because ABX is mighty simple to do and you will read the exact same kind of arguments against it. not everybody has good enough gears to run a jitter test, doesn't mean the test does anything wrong.
I stay on my general idea that all blind tests (double or not) are just a method, a tool, to remove some variables or biases. when you try to get information about one particular variable, if you can remove even one other variable, you're probably already one step closer to getting a proper answer. if you can afford a double blind test that removes a lot of variables, then it's even better.
it's what we do when we measure a given variable, we try to make sure we're not inadvertently measuring the wrong one and we try to end up with only the variable we wished to know more about. I have a question, I look for answers. that's about it.
 
 every single step in the direction of removing variables(be it with measurements or blind testing) painfully demonstrated to me how much of a fool I was before. each step has been a humbling experience telling me something like: "don't you go running your mouth, remember last time you thought you had it all figured out you stupid F".
 
how often did you find someone who got himself a switch, did some tests with it, and then came to say switches are useless and he could do and know more without them?
how often did you see someone getting a multimeter or a sonometer, used it to match loudness and check other stuff, and then after some time came and told you it was better to do without and people shouldn't use those to review gears? doesn't happen all that often right. I believe we agree on this. ^_^
but somehow the argument of no tool, no control(the infamous "just listen") vs almost any objective tool is something you can read about almost every single day on headfi. mind blown by the nonsense.
the nonsense of thinking that somehow doing things wrong is better than doing them well, but also the nonsense of pretending like it's a VS thing. why? the guy is going to die in the next 15 minutes so if he starts a blind test he will never be able to do casual listening? what kind of science fiction goes in the mind of people using that kind of logic? we're saying to do more, not to pick 1 thing and forget all about the rest.
 
people suggesting that I do blind tests and use available measurements, they are simply telling me to multiply my sources of information so that I am better equipped to make a decision, or more confident before making a claim. it's not like doing a blind test will stop me from picking the device I find pretty, or the one with the connectivity I could make more use of. when did any extra knowledge stopped me from thinking straight or picking what I like most if that's what I want? and what if a blind test is inconclusive, so what? the earth won't stop spinning. what is so frightening that some people would actively tell others not to trust those tools? when I think about it I always end up thinking about xenophobia. they're just scared off what they don't understand.
 
now proper tests aren't a lot of fun. and double blind test are indeed very hard to set up well. I understand that someone may not want to waste his time, and decides to stay casual doing what he likes for himself(it's in fact mostly what I do myself). but there is really no excuse to try and drag people into our own laziness with BS excuses while still pretending to be some kind of expert. nobody knows more by doing less.

 

 
CoA
 
I'm not sure if you agree or disagree with me? 

 
How many guys in this thread, do you think, have or built abx switch boxes? 
 
How many guys are taking measurements to compare to what the manufacturer claims? 
 
My contention is that there are very few nick_charles, or nwav guys around. 
 
I think the general population makes an attempt at a listening comparison and doesn't go much further.
 
Maybe I'm wrong, and I want to be wrong, and it wouldn't be the first time. 
 
So come on guys, post the pics of your abx boxes and the tests rigs you've set up.  No pics means it didn't happen.  I have a hard time believing that anybody would go to all the trouble of setting up one of these tests, with sources, and interconnects and power cables and switch boxes and not even take a cell phone picture of it.
 
If you have DiffMaker or nulling files, give us links so we can all hear them, like the comparison files I have from RRrod.
 

 

 

[DavidA]

I've been using a schiit SYS in reverse to do simple ABX comparisons, the volume control is used to match the input/output levels between the different devices, not the best way since you have to adjust the level every time you switch.  A friend let me borrow his level control box that he had built so I could get the levels matched as close as possible and switch between gears quickly.

 

[RRod]

 

Spoiler: Warning: Spoiler!

 

 
CoA
 
I'm not sure if you agree or disagree with me? 

 
How many guys in this thread, do you think, have or built abx switch boxes? 
 
How many guys are taking measurements to compare to what the manufacturer claims? 
 
My contention is that there are very few nick_charles, or nwav guys around. 
 
I think the general population makes an attempt at a listening comparison and doesn't go much further.
 
Maybe I'm wrong, and I want to be wrong, and it wouldn't be the first time. 
 
So come on guys, post the pics of your abx boxes and the tests rigs you've set up.  No pics means it didn't happen.  I have a hard time believing that anybody would go to all the trouble of setting up one of these tests, with sources, and interconnects and power cables and switch boxes and not even take a cell phone picture of it.

 
I'm not a huge fan of this line of reasoning: that control in tests is either all or nothing. I would be 100% happier with all comparisons done on this site if they were simply volume matched, because that alone gets rid of a potent source of bias. If you then add in the control of "not knowing which device is which", you've gone a long way towards making a causal link between having different devices and hearing different sound. The argument from the other side seems to be that adding in this amount of control is, basically, worthless, because it doesn't establish a large-sample, multiple device, AES administered hootenanny. Of course, even when such tests do occur their results are happily rejected if they don't line up with what people detect during sighted, completely uncontrolled evaluations.

 

[upstateguy]

  I've been using a schiit SYS in reverse to do simple ABX comparisons, the volume control is used to match the input/output levels between the different devices, not the best way since you have to adjust the level every time you switch.  A friend let me borrow his level control box that he had built so I could get the levels matched as close as possible and switch between gears quickly.

 
Please show your set up.  Sorry, Pic or it didn't happen.
 

 

 

Spoiler: Warning: Spoiler!

 

 
CoA
 
I'm not sure if you agree or disagree with me? 

 
How many guys in this thread, do you think, have or built abx switch boxes? 
 
How many guys are taking measurements to compare to what the manufacturer claims? 
 
My contention is that there are very few nick_charles, or nwav guys around. 
 
I think the general population makes an attempt at a listening comparison and doesn't go much further.
 
Maybe I'm wrong, and I want to be wrong, and it wouldn't be the first time. 
 
So come on guys, post the pics of your abx boxes and the tests rigs you've set up.  No pics means it didn't happen.  I have a hard time believing that anybody would go to all the trouble of setting up one of these tests, with sources, and interconnects and power cables and switch boxes and not even take a cell phone picture of it.

 
I'm not a huge fan of this line of reasoning: that control in tests is either all or nothing. I would be 100% happier with all comparisons done on this site if they were simply volume matched, because that alone gets rid of a potent source of bias. If you then add in the control of "not knowing which device is which", you've gone a long way towards making a causal link between having different devices and hearing different sound. The argument from the other side seems to be that adding in this amount of control is, basically, worthless, because it doesn't establish a large-sample, multiple device, AES administered hootenanny. Of course, even when such tests do occur their results are happily rejected if they don't line up with what people detect during sighted, completely uncontrolled evaluations.

 
I'm not sure I fully understand what you mean??  Probably not sophisticated enough. 

 

Spoiler: Warning: Spoiler!

The elephant in the room regarding "not all DACs sound the same" is the "192" upsample switch on the Stello DA100.  Even the manufacturer says it sounds different from the "bypass".  That said, it's doesn't seem a stretch to say that implementation can make a difference how a DAC sounds.  Why it changes the sound is a question for an expert.
 

 

[castleofargh]

  CoA
 
I'm not sure if you agree or disagree with me? 

 
How many guys in this thread, do you think, have or built abx switch boxes? 
 
How many guys are taking measurements to compare to what the manufacturer claims? 
 
My contention is that there are very few nick_charles, or nwav guys around. 
 
I think the general population makes an attempt at a listening comparison and doesn't go much further.
 
Maybe I'm wrong, and I want to be wrong, and it wouldn't be the first time. 
 
So come on guys, post the pics of your abx boxes and the tests rigs you've set up.  No pics means it didn't happen.  I have a hard time believing that anybody would go to all the trouble of setting up one of these tests, with sources, and interconnects and power cables and switch boxes and not even take a cell phone picture of it.
 
If you have DiffMaker or nulling files, give us links so we can all hear them, like the comparison files I have from RRrod.
 

 

eheh, I certainly don't disagree with your depiction of the state of things, and TBH as I just said at the end of last post, I really get why people wouldn't want to bother with a lot of complicated tests.
my real deep problem is about the accepted minimum requirements in audio to start talking like an expert.
list of audiophile requirements to talk about anything in a device:  
1/ I've listened to the damn thing.
end of list...
 
and it's the same list if you're talking about the sound, the components, power, RFI, jitter, signal fidelity, PRAT, synergy, quantum mechanic, or pokemon master. you had the device for 3hours, boom, you graduate from lead expert school. you can make up any BS claim about the device, reject measurements, or blind test, or all of science if you so please. and even be offended if someone points out how ludicrous it all is to do so.
I've heard opera, that didn't turn me into an opera expert. I've used a computer that didn't turn me into a computer expert. what kind of magic turns a guy who listened to a device just sitting in a chair, into someone who can make claims about objective variables, or tell us that doing this is more reliable than a blind test? is it ignorance magic, or arrogance magic? maybe a lot of both.
 
 
 
the heart of the problem isn't to do or not to do complicated tests or to follow scientific methods. the problem is that without those, we lack the means to make most of the objective claims people keep making.  if I'm not sure about something I shouldn't tell people that I am. and if I have no objective mean to verify my objective claim, then I should present my opinion, or my hypothesis. not pretend like I'm sharing a fact. it always comes down to placing events in the proper order, the origin of troubles is someone presenting something as a fact when he has no evidence it's a fact. debating if the fact is real is IMO secondary to debating why in the first place he posted something he couldn't prove, in a way that let people believe he could(because that's what making a claim is). measurements and blind tests can help bring pieces of evidence, but an uncontrolled listening can't bring more than "I felt like...". so that's how people doing casual listening should talk about their experience. at least if they were trying to be honest.
 
about posting pictures, I have a simple switch, else for almost all the things I do, I just record the output, with a mic or in a loop( to be able to abx or analyze the resulting files). so the tools are pretty often limited to... a cable. not sure that would impress a lot of people

.

 

[DavidA]

   
Please show your set up.  Sorry, Pic or it didn't happen.
 
 
I'm not sure I fully understand what you mean??  Probably not sophisticated enough. 

 

Spoiler: Warning: Spoiler!

The elephant in the room regarding "not all DACs sound the same" is the "192" upsample switch on the Stello DA100.  Even the manufacturer says it sounds different from the "bypass".  That said, it's doesn't seem a stretch to say that implementation can make a difference how a DAC sounds.  Why it changes the sound is a question for an expert.
 

I posted this picture a while ago:


The SYS is located next to the Ember, this is how my setup always is, the second SYS is only used once in a while when I want to try something new
 
Another one with headphones out:

 

[upstateguy]

  <snip>
eheh, I certainly don't disagree with your depiction of the state of things, and TBH as I just said at the end of last post, I really get why people wouldn't want to bother with a lot of complicated tests.
my real deep problem is about the accepted minimum requirements in audio to start talking like an expert.
list of audiophile requirements to talk about anything in a device:  
1/ I've listened to the damn thing.
end of list...
 
and it's the same list if you're talking about the sound, the components, power, RFI, jitter, signal fidelity, PRAT, synergy, quantum mechanic, or pokemon master. you had the device for 3hours, boom, you graduate from lead expert school. you can make up any BS claim about the device, reject measurements, or blind test, or all of science if you so please. and even be offended if someone points out how ludicrous it all is to do so.
I've heard opera, that didn't turn me into an opera expert. I've used a computer that didn't turn me into a computer expert. what kind of magic turns a guy who listened to a device just sitting in a chair, into someone who can make claims about objective variables, or tell us that doing this is more reliable than a blind test? is it ignorance magic, or arrogance magic? maybe a lot of both.
 
 
 
the heart of the problem isn't to do or not to do complicated tests or to follow scientific methods. the problem is that without those, we lack the means to make most of the objective claims people keep making.  if I'm not sure about something I shouldn't tell people that I am. and if I have no objective mean to verify my objective claim, then I should present my opinion, or my hypothesis. not pretend like I'm sharing a fact. it always comes down to placing events in the proper order, the origin of troubles is someone presenting something as a fact when he has no evidence it's a fact. debating if the fact is real is IMO secondary to debating why in the first place he posted something he couldn't prove, in a way that let people believe he could(because that's what making a claim is). measurements and blind tests can help bring pieces of evidence, but an uncontrolled listening can't bring more than "I felt like...". so that's how people doing casual listening should talk about their experience. at least if they were trying to be honest.
 
about posting pictures, I have a simple switch, else for almost all the things I do, I just record the output, with a mic or in a loop( to be able to abx or analyze the resulting files). so the tools are pretty often limited to... a cable. not sure that would impress a lot of people

.

 
That's pretty funny....1/ I've listened to the damn thing.  something you'd think everyone takes for granted.
 
And of course I agree with your entire 2nd paragraph.
 
But about the pictures.... go ahead and post a pic of your cable, I posted mine, maybe you can show it connected from your amp to your computer.... you'd be surprised how many guys who talk the talk can't walk the walk and have nothing to post.
 
Here's my other cable, going from the preamp out to the input on the computer
 

 
and while I was at it, I recorded the output from the Stello DA100 in both bypass and 192 upsample.  I used the Woo3 preamp out for this but now that I'm thinking about it I might be able to record directly out of the DAC itself and bypass the amp all together.... so that's just what I did and now I have direct recordings from the DAC in both bypass and 192 upsample.  The Flac files are 16.6mb so if anyone wants to compare them PM me and I can e-mail them to you.
 
Not nearly as cool as DavidA's stuff, but here's what it looked like tonight.
 

 

 
I didn't take a picture of the direct from the DAC set up but I just unplugged the cable from back of the Woo3 and plugged it into where the black cables coming are out of the DAC.

 

[upstateguy]

 

   
Please show your set up.  Sorry, Pic or it didn't happen.
 
 
I'm not sure I fully understand what you mean??  Probably not sophisticated enough. 

 

Spoiler: Warning: Spoiler!

The elephant in the room regarding "not all DACs sound the same" is the "192" upsample switch on the Stello DA100.  Even the manufacturer says it sounds different from the "bypass".  That said, it's doesn't seem a stretch to say that implementation can make a difference how a DAC sounds.  Why it changes the sound is a question for an expert.
 

 

Spoiler: Warning: Spoiler!

I posted this picture a while ago:


The SYS is located next to the Ember, this is how my setup always is, the second SYS is only used once in a while when I want to try something new
 
Another one with headphones out:

 

 
Thank you for posting the pics.....  it's always great to see an elaborate set up like yours.....

 

[DavidA]

@upstateguy, I think your setup look cool too

 

[upstateguy]

  @upstateguy, I think your setup look cool too

 
It's very generous of you to say that DavidA, considering your set up looks like a Head-Fi meet. 

 

[stalepie]

Just go by the equipment that looks the best. It requires intelligence to create good product design that looks good. I even decide sometimes just by whether I like the look of the logo. Peavey was always a cool logo, with those sharp angles. 
 
Schiit is pretty cool, with their S symbol. 
 
beyerdynamic is not so cool, though. They just put a bunch of )))) at the end, whatever that's supposed to mean.