DBT problems --methodology
Sep 7, 2009 at 6:13 AM Post #31 of 63
Quote:

Originally Posted by b0dhi /img/forum/go_quote.gif
...Our mouth/nose on the other hand still serve their functions predominantly by supplying low level, "gut instinct" (most appropriate pun ever?) stimuli. ...


Oh that is wondeful! Post of the month! It gave me a "belly" laugh (a lame pun compared to yours).

I'll answer your serious points when I stop ROLFLing.
 
Sep 7, 2009 at 6:21 AM Post #32 of 63
Quote:

Originally Posted by SmellyGas /img/forum/go_quote.gif
Perhaps a fraction of the population can reliably hear differences in some source material. But who cares!


Those people that can potentially hear the differences care. Say, for example, some of those on a hi-fi forum *cough*

Quote:

My point is that DACs/amps/cables cannot possibly make the LARGE and SUBSTANTIAL audible differences that people claim, because if they did, they would be easily and readily detectable on blind, controlled listening tests.


1 - You are confusing "large" and "substantial". They do not mean the same thing here.

2 - I suggest you go speak to those that claim "large" differences in cables and ask them to define what they mean by large. Please stop derailing this thread by repeating the same questions over and over again, even though they've been answered numerous times.

Quote:

And if DACs/amps/cables only made SUBTLE differences and NOT large/substantial audible differences, then why on earth would I spend thousands of dollars on an upgrade that is so subtle in magnitude that only a fraction of the population, perhaps trained, can reliably hear differences in some source material!!??


You are free to do whatever you want - again, please stop derailing this thread. The thread topic is "DBT problems/methodology", not "Busy people respond to SmellyGas' off-topic demands disguised as questions".
 
Sep 7, 2009 at 6:25 AM Post #33 of 63
Quote:

Originally Posted by SmellyGas /img/forum/go_quote.gif
...My point is that DACs/amps/cables cannot possibly make the LARGE and SUBSTANTIAL audible differences that people claim, because if they did, they would be easily and readily detectable on blind, controlled listening tests


Yes, you have to be right about this, no question, as long as you add "across the whole population" after the word "differences", as I think you clearly mean.

Quote:

Originally Posted by SmellyGas /img/forum/go_quote.gif
...And if DACs/amps/cables only made SUBTLE differences and NOT large/substantial audible differences, then why on earth would I spend thousands of dollars on an upgrade that is so subtle in magnitude that only a fraction of the population, perhaps trained, can reliably hear differences in some source material!!??


Because it makes a difference to you, because you can hear it, or think you can, and you like it, and it makes you feel wonderful, and you can afford it. You really do hear it in your home, even if you fail a blind test. Call this placebo effect?? Who cares? You really hear it.

Compare (rollover for prices)

Neiman Marcus*-*Men's Shop - Tie Shop

with

Neckties : Clothing : Men : Target

(prices shown).

Factor of 10 cannot be explained by workmanship and material alone.

Now I buy my ties from street vendors in Manhattan for even less than Target -- I have to wear ties when I represent my company -- but some of the people I call on buy ties from Neiman Marcus, and these are smart, "with it" people. Why do they do this? They like the way they look and feel in the expensive tie, even though in a lineup (like a blind test) 99.999% of the earth's population (probably more) could not pick out the fellow wearing the expensive tie.

I have a name for this. I call it "life".
 
Sep 7, 2009 at 6:50 AM Post #34 of 63
Quote:

Originally Posted by wavoman /img/forum/go_quote.gif
I have a name for this. I call it "life".


I feel a paradigm shift in statistics coming!!!
 
Sep 7, 2009 at 12:23 PM Post #35 of 63
Quote:

Originally Posted by penguin121 /img/forum/go_quote.gif
This is just begging the question of how such a difference could be considered large. If the difference take so much time/effort or some other special conditions to notice, how would it not be considered a subtle difference?
confused_face(1).gif



Quality of ketchup may not be vastly important to you if you only ate one hotdog a day. Eat more hotdogs a day and quality of ketchup increases in importance. Some people listen to music 5-10 hours a day, in which case they might feel cables to be essential.
 
Sep 7, 2009 at 2:50 PM Post #36 of 63
Quote:

Originally Posted by SmellyGas /img/forum/go_quote.gif
No, it's a positive control. You MUST have a way to demonstrate that your system is capable of identifying differences. If people cannot differentiate between cables/amps/dac's unblinded, and they cannot differentiate between them blinded, how can we be sure our negative results were not due to the inability of our listeners to hear differences in our system? You can't! So you MUST have a positive control, and you must be able to demonstrate that your listeners, on the stereo system provided, are capable of hearing differences.


Whether or not it is a positive control here ultimately depends on whether there exists a real difference between the two set ups. If there isn't a real difference between the two set ups, then it's not a positive control, because there is nothing to differentiate between. My understanding of your experimental design was for one part of the experiment (unblinded), the testers know when they are listening to different equipment, and you ask to see if they can detect the difference, and they can- but then when the experiment is done blinded, then they can no longer detect a difference. I interpret that data as suggesting that no difference between the two set ups existing- thus their ability to tell the difference between the two set ups in the unblinded test didn't exist and we weren't testing in that part of the experiment what we wanted to. Hence it's not a positive control.

Quote:

I disagree. This would only constitute a positive control if you KNEW that the type of audible degradation that occurs in amps/dac's/cables is exactly the same as the one you are introducing as your positive control (e.g. "lower order bits scrambled," to use your example). Otherwise, if listeners can distinguish between scrambled vs. non-scrambled, but they cannot distinguish between cables, critics will say that this was due to listeners being able to hear scrambled bits, but not the type of degradation that occurs in cables.


This really isn't supposed to be a proper positive control- what I am doing this for is to get an objective measurement of how 'different' things have to be, before the listeners can detect a difference. Doing this by starting with the lowest order bits just seemed like an easy way to start off. We could replace 'randomizing lowest order bits', with anything really that would let us create measurable difference between the set ups. The really nice thing about the lowest order bits business though, is we have an exact mathematical relationship between this and dynamic range- so we can generate exact testable hypotheses. I'm sure though we could be a little bit clever, and come up with some other exact testable hypotheses.

Quote:

My point is that DACs/amps/cables cannot possibly make the LARGE and SUBSTANTIAL audible differences that people claim, because if they did, they would be easily and readily detectable on blind, controlled listening tests.

And if DACs/amps/cables only made SUBTLE differences and NOT large/substantial audible differences, then why on earth would I spend thousands of dollars on an upgrade that is so subtle in magnitude that only a fraction of the population, perhaps trained, can reliably hear differences in some source material!!??


For the same reason people buy Ferraris and Porsches when Ford and and Chevy will do. A Ford or a Chevy will get you to the store to buy groceries, drop off your children at school, allow you to go and visit your parents, as well as the Ferrari or a Porsche would. Practically speaking, the speed of the Ferrari and Porsche is lost on most of the population most of the time. But some people really appreciate the subtle differences and improvements.

You're asking a question which may be relevant for you- "Why should I spend a lot of money buying equipment which makes very subtly small change to audio performance?", but the answer to that is really only interesting to you.

The relationship between cost and performance is extremely asymptotic in most cases. Having to spend 2x as much money to get a 5% improvement in audio might not be worth it to you today, but maybe in 1 year after you've won a superball lotery, it will be (just remember who your friends are then *hint hint*). By a similar fashion, most of the population doesn't understand spending hundreds of dollars on a pair of headphones versus the $10 ones you can get at BestBuy. They're cost/performance function asymptotes much more rapidly than yours- and the exact shape of that curve is going to vary according by individual, and by resource status.

But you have definitely cleared up for me what question you are interested in answering.
smily_headphones1.gif
It's just not the same one that I'm interested in.
 
Sep 7, 2009 at 2:53 PM Post #37 of 63
Quote:

Originally Posted by haloxt /img/forum/go_quote.gif
Quality of ketchup may not be vastly important to you if you only ate one hotdog a day. Eat more hotdogs a day and quality of ketchup increases in importance. Some people listen to music 5-10 hours a day, in which case they might feel cables to be essential.


+1 ftw.
 
Sep 7, 2009 at 5:23 PM Post #38 of 63
Quote:

Originally Posted by b0dhi /img/forum/go_quote.gif
2 - I suggest you go speak to those that claim "large" differences in cables and ask them to define what they mean by large. Please stop derailing this thread by repeating the same questions over and over again, even though they've been answered numerous times.


If my very simple and obvious question had been answered anywhere near to my satisfaction, I obviously wouldn't be asking it again. (Hint: responding with "stop derailing this thread, we're busy" like you did doesn't reall constitute a satisfactory response.)[/QUOTE]

Quote:

Originally Posted by wavoman /img/forum/go_quote.gif
Because it makes a difference to you, because you can hear it, or think you can, and you like it, and it makes you feel wonderful, and you can afford it. You really do hear it in your home, even if you fail a blind test. Call this placebo effect?? Who cares? You really hear it.


I'm confused. Nobody doubts that the placebo effect can account for why people hear differences among amps/dac's/cables. But I thought the question was whether cables/amps/dac's THEMSELVES had audible differences? If you didn't care about eliminating the contribution of the placebo effect, why even do BLIND testing?

Quote:

Now I buy my ties from street vendors in Manhattan for even less than Target -- I have to wear ties when I represent my company -- but some of the people I call on buy ties from Neiman Marcus, and these are smart, "with it" people. Why do they do this? They like the way they look and feel in the expensive tie, even though in a lineup (like a blind test) 99.999% of the earth's population (probably more) could not pick out the fellow wearing the expensive tie.

I have a name for this. I call it "life".


I understand you necktie example. Some people buy expensive ties because they like they way they make them feel and they can afford them. Some buy cheap ties because they look exactly the same to 99% of the population. There is absolutely nothing wrong with that. Similarly, if people bought expensive amps/dac's/cables because they LIKE the way the LOOK, they can brag to their friends, and NOT because they thought it made their stereo systems sound better, that would also be fine. HOWEVER, people buy expensive amps/dac's/cables because they THINK it makes dramatic improvements in sound quality. Using your example, this would be like people buying expensive neckties because they thought they LOOKED much better than the cheap ones, despite the fact that they were both manufactured in the same factory and were not distinguishable with the labels removed.

Quote:

Originally Posted by haloxt /img/forum/go_quote.gif
Quality of ketchup may not be vastly important to you if you only ate one hotdog a day. Eat more hotdogs a day and quality of ketchup increases in importance. Some people listen to music 5-10 hours a day, in which case they might feel cables to be essential.


Some people here claim that in order to hear an difference between cables, it requires 5-10 hours of careful listening to each. Now, if you only listen to music 5-10 hours a day, based on this information, you may or may not have been able to distinguish between the two cables anyway during your normal daily listening!!

Quote:

Originally Posted by Clutz /img/forum/go_quote.gif
Whether or not it is a positive control here ultimately depends on whether there exists a real difference between the two set ups. If there isn't a real difference between the two set ups, then it's not a positive control, because there is nothing to differentiate between.


Yeah, strictly speaking, it would not be a positive control. However, you STILL need to demonstrate that your equipment setup + listeners are able to identify and report differences (i.e. validate your experimental apparatus) - you can call it what you wish.

Quote:

My understanding of your experimental design was for one part of the experiment (unblinded), the testers know when they are listening to different equipment, and you ask to see if they can detect the difference, and they can- but then when the experiment is done blinded, then they can no longer detect a difference. I interpret that data as suggesting that no difference between the two set ups existing- thus their ability to tell the difference between the two set ups in the unblinded test didn't exist


This type of experiment has been published. IF you think about it, the independent variable is "blinded/unblinded. It is used as a good way to test the hypothesis that audible differences between amps/dacs/cables are eliminated when listeners are blinded. If the hypothesis is confirmed, then the most likely explanation is that listener perceptions were influenced by their expectations when unblinded = placebo effect.

Quote:

This really isn't supposed to be a proper positive control- what I am doing this for is to get an objective measurement of how 'different' things have to be, before the listeners can detect a difference. Doing this by starting with the lowest order bits just seemed like an easy way to start off. We could replace 'randomizing lowest order bits', with anything really that would let us create measurable difference between the set ups. The really nice thing about the lowest order bits business though, is we have an exact mathematical relationship between this and dynamic range- so we can generate exact testable hypotheses. I'm sure though we could be a little bit clever, and come up with some other exact testable hypotheses.


Oh, I understand. My criticism still stands, however. Reduction is dynamic range is really not an analogous degradation in sound quality that people expect from amp/cable/dac differences, so it really shouldnt be used. But I understand what you're trying to do - you're trying to test your listeners with varying levels of arbitrary signal degradations to detect their threshold ability to differentiate between small differences in sound. Good luck with that.

Quote:

For the same reason people buy Ferraris and Porsches when Ford and and Chevy will do. A Ford or a Chevy will get you to the store to buy groceries, drop off your children at school, allow you to go and visit your parents, as well as the Ferrari or a Porsche would. Practically speaking, the speed of the Ferrari and Porsche is lost on most of the population most of the time. But some people really appreciate the subtle differences and improvements.


That is a terribly non-parallel example! "Subtle" improvements between a Ford and a Ferrari? Are you for real? A Ford and a Ferrari have completely different acceleration times, stopping distances, skidpad g's, lanechange speeds etc. - i.e. the MEASUREMENTS of PERFORMANCE are completely night and day. On the other hand, amps/dac/cables have very little differences in their measurements of performance (frequency response, distortion, etc.). This is why your example is completely non-applicable. Did you think it through at all?

Quote:

You're asking a question which may be relevant for you- "Why should I spend a lot of money buying equipment which makes very subtly small change to audio performance?", but the answer to that is really only interesting to you.


You may be right about that. Some people may be willing to spend thousands of dollars on amps/dac's/cables for the potential to make very small, barely audible, improvements in actual sound quality. However, I think people tend to overestimate the magnitude of possible improvement from these upgrades. This is a good article if you haven't read it: http://www.nousaine.com/pdfs/To%20Tweak%20or%20Not.pdf

Quote:

The relationship between cost and performance is extremely asymptotic in most cases. Having to spend 2x as much money to get a 5% improvement in audio might not be worth it to you today, but maybe in 1 year after you've won a superball lotery, it will be (just remember who your friends are then *hint hint*). By a similar fashion, most of the population doesn't understand spending hundreds of dollars on a pair of headphones versus the $10 ones you can get at BestBuy. They're cost/performance function asymptotes much more rapidly than yours- and the exact shape of that curve is going to vary according by individual, and by resource status.


I'll give you that, sure. However, unless you have $100,000 loudspeakers, or the best electrostatic headphones ever made, then $/% sound improvement (to use your metric), is still greater in upgrading your transducer by a large margin.

Quote:

But you have definitely cleared up for me what question you are interested in answering.
smily_headphones1.gif
It's just not the same one that I'm interested in.


The question you are interested in is really only of interest to a very very small segment of the hi-fi world. Sorry, but most people are interested in "value," or the opportunity cost of providing an improvement. Even *IF* you discovered differences in amps/dac's/cables that could be heard by 25% of trained audiophiles after listening for 1 hour (but not less) on certain classical music tracks, and this amp/dac/cable upgrade cost $5000, I can tell you that very very few people would spend that $5000. Some would, I agree.
 
Sep 7, 2009 at 5:57 PM Post #39 of 63
Quote:

Some people here claim that in order to hear an difference between cables, it requires 5-10 hours of careful listening to each. Now, if you only listen to music 5-10 hours a day, based on this information, you may or may not have been able to distinguish between the two cables anyway during your normal daily listening!!


Blah blah blah blah blah
 
Sep 7, 2009 at 5:58 PM Post #40 of 63
Quote:

Originally Posted by haloxt /img/forum/go_quote.gif
Quality of ketchup may not be vastly important to you if you only ate one hotdog a day. Eat more hotdogs a day and quality of ketchup increases in importance. Some people listen to music 5-10 hours a day, in which case they might feel cables to be essential.


And if you need to consume thousands of hotdogs to begin to tell any difference between the two ketchups, I would consider it just as strange if you then to insist that there were a large difference between the two. I'm not saying subtle differences don't matter to some people, of course they do. I just don't see how a difference that by all accounts will take prolonged periods of careful listening to identify could be considered large. If that would count as a large difference, then what exactly would be a small difference?
 
Sep 7, 2009 at 6:33 PM Post #41 of 63
Why all this semantic debate about small and large? I think you guys are really missing the point if you want to fight my hypothetical with more hypotheticals. The only question I think is important is whether or not cables make a difference.
 
Sep 7, 2009 at 7:34 PM Post #42 of 63
Quote:

Originally Posted by penguin121 /img/forum/go_quote.gif
And if you need to consume thousands of hotdogs to begin to tell any difference between the two ketchups, I would consider it just as strange if you then to insist that there were a large difference between the two.


LOL. It's amazing how people offer what they think are solid parallel examples but don't realize how easily they are shut down.

Quote:

Originally Posted by haloxt /img/forum/go_quote.gif
Why all this semantic debate about small and large? I think you guys are really missing the point if you want to fight my hypothetical with more hypotheticals. The only question I think is important is whether or not cables make a difference.


No, this is a VERY important point. It's a big waste of time to spend the time/energy answering a question that few people would care about. For instance, suppose you find that a 12 year old with large ears can reliably detect a difference in two cables in 50 out of 50 trials, with 1 hour of listening per trial, but none of the other 99 listeners could. Fine, so you've proven that your cables did have an audible difference, but only to 1% of the people you tested and only with considerable difficulty. Ask yourself something - did you really prove something that people care about?

Quote:

Originally Posted by haloxt /img/forum/go_quote.gif
SmellyGas Quote:
Some people here claim that in order to hear an difference between cables, it requires 5-10 hours of careful listening to each. Now, if you only listen to music 5-10 hours a day, based on this information, you may or may not have been able to distinguish between the two cables anyway during your normal daily listening!!

HaloXT wrote:
Blah blah blah blah blah



I apparently made such a good point that you were rendered unintelligible. Seriously, though. If it takes you 5-10 hours of careful listening with specific music/conditions to hear a difference between amps/cables/DAC's, then it follows that you would not be able to reliably distinguish between the two in a shorter 1-2 hour listening session with your usual assortment of music!!
 
Sep 7, 2009 at 7:42 PM Post #43 of 63
Quote:

Yeah, strictly speaking, it would not be a positive control. However, you STILL need to demonstrate that your equipment setup + listeners are able to identify and report differences (i.e. validate your experimental apparatus) - you can call it what you wish.


It's not a matter of strictly speaking- it's simply not a positive control.

Quote:

This type of experiment has been published. IF you think about it, the independent variable is "blinded/unblinded. It is used as a good way to test the hypothesis that audible differences between amps/dacs/cables are eliminated when listeners are blinded. If the hypothesis is confirmed, then the most likely explanation is that listener perceptions were influenced by their expectations when unblinded = placebo effect.


I fully agree- that's the usefulness of that test, but it is not a positive control. That test does nothing to demonstrate that the listeners are capable of resolving actual differences between cables- it just shows that they're willing to report differences, whether they exist or not. To do that you need an experiment that creates known differences between source material in order to ascertain what degree of precision the listeners are able to differentiate.


Quote:

Oh, I understand. My criticism still stands, however. Reduction is dynamic range is really not an analogous degradation in sound quality that people expect from amp/cable/dac differences, so it really shouldnt be used. But I understand what you're trying to do - you're trying to test your listeners with varying levels of arbitrary signal degradations to detect their threshold ability to differentiate between small differences in sound. Good luck with that.


No, apparently you don't understand what I'm trying to do. I used it as an easy example of how someone could control the signal in a non-arbitrary and measurable way, and then be able to see if the population is able to discern the difference. In the previous post I even said that I didn't think it was the right way to do it, that it was just a quick and dirty method I could come up with to explain how one would go about testing it, even if randomizing the lowest order bits is the wrong approach. Further, you haven't demonstrated that it wouldn't work- just that you don't think it would. I'm not saying it is the right approach, but you haven't backed up your objections in anyway, other than your claim that they're different. They may be, they might not be- I don't know- but you haven't proven anything.

Quote:

That is a terribly non-parallel example! "Subtle" improvements between a Ford and a Ferrari? Are you for real? A Ford and a Ferrari have completely different acceleration times, stopping distances, skidpad g's, lanechange speeds etc. - i.e. the MEASUREMENTS of PERFORMANCE are completely night and day. On the other hand, amps/dac/cables have very little differences in their measurements of performance (frequency response, distortion, etc.). This is why your example is completely non-applicable. Did you think it through at all?


The example is parallel- the difference between the examples is scale and perspective. Ferraris and Fords both take you from place to place and they do so in the same fashion. They differ from each other in specific measurements of their abilities to do things, but that is it. Some people think spending money on a Ferrari over a Ford is worth it, others don't. Some people think spending a lot of money on expensive cables is worth it, others don't. I think it's rather galling though that you'd question my thoughtfulness when you said my example was non-parallel (and you're wrong), and when you'd previously used the term 'positive control'. But whatever- if you want to get into this with me, feel free- but all that's going to happen is it's going to stifle an otherwise interesting discussion, and I'm going to go out of my way to point out every single silly thing you say.

Quote:

You may be right about that. Some people may be willing to spend thousands of dollars on amps/dac's/cables for the potential to make very small, barely audible, improvements in actual sound quality. However, I think people tend to overestimate the magnitude of possible improvement from these upgrades.


But you've just changed the question now.


Quote:

I'll give you that, sure. However, unless you have $100,000 loudspeakers, or the best electrostatic headphones ever made, then $/% sound improvement (to use your metric), is still greater in upgrading your transducer by a large margin.


Okay, if you say so- but ultimately that isn't the question under discussion here, so it's a little bit irrelevant.

Quote:

The question you are interested in is really only of interest to a very very small segment of the hi-fi world. Sorry, but most people are interested in "value," or the opportunity cost of providing an improvement. Even *IF* you discovered differences in amps/dac's/cables that could be heard by 25% of trained audiophiles after listening for 1 hour (but not less) on certain classical music tracks, and this amp/dac/cable upgrade cost $5000, I can tell you that very very few people would spend that $5000. Some would, I agree.


Why are you specifying one hour, not less? Why 25%? Why $5000? Why is the question of interest only to a very small percentage of the population? I don't believe that going from $5000 set of speakers to a $10,000 set of speakers would yield the same return as going from $500 amplifier/dac/cable combo to a $5,500 amplifier/dac/cabe combo. I think quite clearly you would be better spending $5,000 on a $500 DAC/AMP/Cable combo than going to $10,000 speakers. Ultimately, each of these options have very different diminishing returns curves, and are also dependent on the other components of the system. Again, I don't understand how this is relevant to the bigger question being discussed here.
 
Sep 7, 2009 at 7:45 PM Post #44 of 63
Quote:

Originally Posted by SmellyGas /img/forum/go_quote.gif
LOL. It's amazing how people offer what they think are solid parallel examples but don't realize how easily they are shut down.


I think it's amazing that you question the parallelism of peoples examples, when apparently you don't know what 'parallel' means.



Quote:

No, this is a VERY important point. It's a big waste of time to spend the time/energy answering a question that few people would care about. For instance, suppose you find that a 12 year old with large ears can reliably detect a difference in two cables in 50 out of 50 trials, with 1 hour of listening per trial, but none of the other 99 listeners could. Fine, so you've proven that your cables did have an audible difference, but only to 1% of the people you tested and only with considerable difficulty. Ask yourself something - did you really prove something that people care about?


Why is it a big waste of time to spend time and energy on a question that few people care about? Because you said so? Do you think it's a large waste of time and energy to learn about sub-atomic particles, because only a few people care? Do you think it's a large waste of time and energy to learn about the mathematics behind knots, because only a few people care? Who made you the final arbiter of what's interesting and uninteresting? Your arrogance is so complete that I think you are incapable of understanding anything from anyone else's perspective.



Quote:

I apparently made such a good point that you were rendered unintelligible. Seriously, though. If it takes you 5-10 hours of careful listening with specific music/conditions to hear a difference between amps/cables/DAC's, then it follows that you would not be able to reliably distinguish between the two in a shorter 1-2 hour listening session with your usual assortment of music!!


That's not true. It might take you 5-10 hours of listening the first time to discern a difference- but that doesn't mean after you've learned to detect the difference that it'd continue to take you 5-10 hours each time after that. That's just goofy.
 
Sep 7, 2009 at 8:24 PM Post #45 of 63
Smellygas, maybe you're not aware but there's a big question being asked on head-fi of whether or not cables make a difference at all. What you're saying has no application to this question. And I'm not really interested at all in your claim that there's no worthwhile difference between amps and dacs and that it must be discerned "EASILY EASILY EASILY".
 

Users who are viewing this thread

Back
Top