or Connect
Head-Fi.org › Forums › Equipment Forums › Sound Science › Audiophile cables, an interesting question.
New Posts  All Forums:Forum Nav:

Audiophile cables, an interesting question. - Page 23

post #331 of 1186
Quote:
Originally Posted by x838nwy View Post

what would be a significant number of tests?

Can someone good at stats let me know what percentage of correct identification is significant? (I mean, if i can tell there's a difference like 55% of the time does that say anything? What about 53%?) actually tell me later after results come in.

The more tests you do, the better your results. Around 15 trials would be a good number.

The significance of your result is known as the p-value, i.e the probability that such a result could occur due to pure chance.

http://home.provide.net/~djcarlst/abx_bino.htm

This is a table of values.

<0.05 is often considered to be significant but replication of these results would be expected to confirm the validity.

post #332 of 1186

I'm contemplating using one of these to switch between sources http://schiit.com/products/sys-passive-preamp

 

I could use the short pyst cable between this and my amp and connect the two cables into different sources. I can hook both up to my gungnir as it has two pairs of RCA outs. This would make the changes quicker and no plugging/unplugging.

 

Does anyone know of an (possibly cheaper) alternative?

 

I'll see when this can get there without going express or whatever. This should be interesting...

post #333 of 1186
Quote:
Originally Posted by x838nwy View Post
 

I'm contemplating using one of these to switch between sources http://schiit.com/products/sys-passive-preamp

 

I could use the short pyst cable between this and my amp and connect the two cables into different sources. I can hook both up to my gungnir as it has two pairs of RCA outs. This would make the changes quicker and no plugging/unplugging.

 

Does anyone know of an (possibly cheaper) alternative?

 

I'll see when this can get there without going express or whatever. This should be interesting...

 

Bugger that. Order placed. If 49.99 + shipping is the price for truth then so be it. If not, at least I have a handy source selector and these small Schiit boxes are wife-friendly (hopefully she'll think it's the Vali or the Modi. Best not have them all in the same room at once).

post #334 of 1186
I’ve done this exercise in the past to see where the pitfalls are. For me it was not a matter of proof but rather to see why blind tests are so difficult to use as proof of an audible change between two devices.

Looking back on my notes I can give you some things you’re going to need to do in order to improve your chances of making any comparisons successful. Blind testing for audible differences by repeated comparisons requires training and takes time to obtain meaningful results, as in lots of hours over a long period of time, like days or weeks. The good news is your ability to determine minute changes will get better with experience. The bad news is you need to continue to work at it until you reach your goal, whatever that may be.

Broken down into stages…

1. Train yourself as to what the differences are that you are listing for between two components under test. Take notes to narrow the comparison process you’re ultimately going to create. In other words, you’re not going into this blind but with as complete an understanding of the sonic characteristics of each device under test as you care to learn. The more you know about them the easier comparisons become. To not do so is the equivalent of expecting a dog to follow a scent without having smelled it.
If you have never done comparisons like this before, it would speed the process to have someone with you who has as they can quickly point out changes for you to hone in on. It all begins with hearing one difference, then comparing, and contrasting that with other devices under test; then the next difference, then the next, and so on, building a mental library so to speak.
2. Locate specific sections of song tracks that exemplify these differences. Note what you hear and the time index.
3. Create a play list of these short portions of specific tracks in a planned order to target differences and shorten test time.
4. Repeat.

Suggestions…

1. Volume should be kept the same during any change, however it can and should be varied (while at the listening position) while a device is under test. Certain aspects become glaring at slightly higher volumes. Once you hone in on a sonic signature, the volume may not need be increased for the same result next time.
2. Quickly changing the devices under test then trying to listen over and over is tedious and stressful- ultimately this method of comparison does not work well. Even if the change is instantaneous as with a remote switch, there is no long-term benefit to this method nor is it a short-cut to discerning audible differences. You must learn differences while always second guessing yourself to confirm.
3. If you’re using speakers, don’t be afraid to move your head or listening position to better hear things within the soundstage. I’ve seen too many people stay fixed in the sweet spot even though what you really want to hear may not be exactly in that spot for different recordings. If you do not have a stereo image or soundstage within your set-up, this MUST be resolved first. Many differences are not huge but are easily heard as changes within the soundstage, for example vocals being more distant, nearer, or cleaner.
4. If using headphones, move the headphones about on your head track to track in an attempt to adjust acoustics, similar to above. With some headphones this may not make a difference but it’s worth a try.
5. Keep in mind you are always training yourself to hear an increasing number of differences in or signatures related to each device under test in order to more easily pick it out when you hear it again.
6. If there is any gap in making a change, say more than a minute, be sure not to make any rash judgments when listening resumes. Listen to a number of tracks to reinstate your memory. We’ll call this the one minute rule.
7. Create 10-15 tracks, and if possible shorten to just the portions where you hear a difference to ultimately have one continuous playback with gaps for quicker comparisons. Note order may matter.
8. Create a scoring method for the above. Maybe choose device A or B for each track section, and add the results. More A’s, must be device A.

Turns out doing blind listening comparison testing is not as easy as it would at first seem, even if you already know the outcome☺. It can be done with 9 out of 10 results, but it takes A LOT of practice.

The more practical approach to understanding the sound characteristics of a device is to listen to it over a period of time; hours, days, even weeks. This averages out to a better understanding of the sonic characteristics of the device within the context of your long-term listening style, and also allows time to assess other portions of the system and how their interaction may relate to what you are hearing. Always remember you are listing to the sum of the parts of an entire system, including the original recording venue and set-up. Tendency is to point at the last change as the culprit when it just might only be the messenger.

Abyss Headphones-  A superior way to listen... JPS Labs- Worlds Finest Audio Products...

The Evolution Of Sound- Premier US Distributor

Reply
post #335 of 1186
Quote:
Originally Posted by x838nwy View Post

Bugger that. Order placed. If 49.99 + shipping is the price for truth then so be it. If not, at least I have a handy source selector and these small Schiit boxes are wife-friendly (hopefully she'll think it's the Vali or the Modi. Best not have them all in the same room at once).


I really like the little schitt passive pre. It's a handy switch for monitors to choose between modi or my fiancée's imac for a source. Im sure you can find a good everyday use for it Once youre done experimenting with it.

Best part: i havent noticed any clicking when it switches. Thats a big plus for blind testing

Cheers
post #336 of 1186
Quote:
Originally Posted by Joe Skubinski View Post

I’ve done this exercise in the past to see where the pitfalls are. For me it was not a matter of proof but rather to see why blind tests are so difficult to use as proof of an audible change between two devices.

Looking back on my notes I can give you some things you’re going to need to do in order to improve your chances of making any comparisons successful. Blind testing for audible differences by repeated comparisons requires training and takes time to obtain meaningful results, as in lots of hours over a long period of time, like days or weeks. The good news is your ability to determine minute changes will get better with experience. The bad news is you need to continue to work at it until you reach your goal, whatever that may be.

Broken down into stages…

1. Train yourself as to what the differences are that you are listing for between two components under test. Take notes to narrow the comparison process you’re ultimately going to create. In other words, you’re not going into this blind but with as complete an understanding of the sonic characteristics of each device under test as you care to learn. The more you know about them the easier comparisons become. To not do so is the equivalent of expecting a dog to follow a scent without having smelled it.
If you have never done comparisons like this before, it would speed the process to have someone with you who has as they can quickly point out changes for you to hone in on. It all begins with hearing one difference, then comparing, and contrasting that with other devices under test; then the next difference, then the next, and so on, building a mental library so to speak.
2. Locate specific sections of song tracks that exemplify these differences. Note what you hear and the time index.
3. Create a play list of these short portions of specific tracks in a planned order to target differences and shorten test time.
4. Repeat.

Suggestions…

1. Volume should be kept the same during any change, however it can and should be varied (while at the listening position) while a device is under test. Certain aspects become glaring at slightly higher volumes. Once you hone in on a sonic signature, the volume may not need be increased for the same result next time.
2. Quickly changing the devices under test then trying to listen over and over is tedious and stressful- ultimately this method of comparison does not work well. Even if the change is instantaneous as with a remote switch, there is no long-term benefit to this method nor is it a short-cut to discerning audible differences. You must learn differences while always second guessing yourself to confirm.
3. If you’re using speakers, don’t be afraid to move your head or listening position to better hear things within the soundstage. I’ve seen too many people stay fixed in the sweet spot even though what you really want to hear may not be exactly in that spot for different recordings. If you do not have a stereo image or soundstage within your set-up, this MUST be resolved first. Many differences are not huge but are easily heard as changes within the soundstage, for example vocals being more distant, nearer, or cleaner.
4. If using headphones, move the headphones about on your head track to track in an attempt to adjust acoustics, similar to above. With some headphones this may not make a difference but it’s worth a try.
5. Keep in mind you are always training yourself to hear an increasing number of differences in or signatures related to each device under test in order to more easily pick it out when you hear it again.
6. If there is any gap in making a change, say more than a minute, be sure not to make any rash judgments when listening resumes. Listen to a number of tracks to reinstate your memory. We’ll call this the one minute rule.
7. Create 10-15 tracks, and if possible shorten to just the portions where you hear a difference to ultimately have one continuous playback with gaps for quicker comparisons. Note order may matter.
8. Create a scoring method for the above. Maybe choose device A or B for each track section, and add the results. More A’s, must be device A.

Turns out doing blind listening comparison testing is not as easy as it would at first seem, even if you already know the outcome☺. It can be done with 9 out of 10 results, but it takes A LOT of practice.

The more practical approach to understanding the sound characteristics of a device is to listen to it over a period of time; hours, days, even weeks. This averages out to a better understanding of the sonic characteristics of the device within the context of your long-term listening style, and also allows time to assess other portions of the system and how their interaction may relate to what you are hearing. Always remember you are listing to the sum of the parts of an entire system, including the original recording venue and set-up. Tendency is to point at the last change as the culprit when it just might only be the messenger.


I mean no disrespect to you, but the above is almost a recipe for how not to come to meaningful conclusions unless I misunderstand.    I am sorry I am short on time today and cannot fully explain why, but will do so later.  One example: While one needs to vary volume to reach a comfortable level for a listening test, item #1 in any such comparison is once you are ready to switch components you must maintain precisely the same volume.    I have tried it personally, it has been tested a number of ways and a number of times.  Almost everyone feels more comfortable in their judgments when done over long times like hours or days.  And in every test with significant results it is short term listening with relatively rapid switching which is the more discriminating method.  Due to echoic memory it is suggested each selection not last longer than 20 seconds before switching.  Again nothing against you personally Joe Skubinsky, but I find most of your suggestions to be wrong.


Edited by esldude - 4/12/14 at 10:55am
post #337 of 1186
The difficulty of obtaining useful results from listening tests is frequently exaggerated by those who for some reason or another, find the results disagreeable.

Here's a link to an excellent how-to from Stuart Yaniger.

http://www.linearaudio.net/images/LA%20Vol%202%20Yaniger(1).pdf

w
post #338 of 1186
My suggestions had nothing to do with the administration of a standardized test. They had everything to do with understanding that hearing subtle differences is not something you can walk into and obtain useful results without knowing what it is you're listening for.

Abyss Headphones-  A superior way to listen... JPS Labs- Worlds Finest Audio Products...

The Evolution Of Sound- Premier US Distributor

Reply
post #339 of 1186
If you have to jump through hoops to even hear a difference, odds are the difference doesn't mean jack diddly to sound quality when listening to music.

A difference is a difference. Set two things next to each other with all other things being even and a difference should be clear. I think that most people who think hearing differences is a difficult thing are training themselves to hear differences that just aren't there.
post #340 of 1186
Quote:
Originally Posted by esldude View Post


I mean no disrespect to you, but the above is almost a recipe for how not to come to meaningful conclusions unless I misunderstand.

You don't misunderstand. Just about every one of those numbered suggestsions would make it MORE difficult to hear subtle differences, not easier.
post #341 of 1186

So, x838nwy, you thought that running your own DBT would somehow clarify the matter? No chance, this is just the beginning… of your nightmare:devil_face:.

 

Joe makes some thoughtful points, which I think are particularly valid if you need to know how the said component is to perform in a range of circumstances (essential for a designer) and I largely agree. However, every single variable in such a test will conspire to confuse your poor overworked brain. So if you just need to know if A is different to B, I think you need to try remove all possible variables , including your circumstance and mood (e.g. time of day).

 

I think that fast-switching DBTs are unbeatable at removing Expectation Bias, and excellent at instantly spotting gross tonal/colouration differences that you typically get with headphones (frequency response that looks like a cross-section of the Himalayas) . But the same tests are unreliable IMO at spotting more subtle-but-significant differences that you get with relatively neutral components like DACs and cables (ruler flat frequency response, negligible THD etc). Not imposible, just unreliable because of the mind tricks that have been fruitlessly debated a million times.

 

What this means is that if you run your fast-switching-DBT and find no difference between components A and B, I simply won't accept that result until you repeat the DBT in a longer term test, of a style that I would deem to be fool proof.

 

The only thing that can be completely guaranteed with such a test is that the side that disagrees with your  end result will nit pick your method to death, and the side that agrees with your end result will quietly overlook the same flaws.

post #342 of 1186

So basically DBTs are completely unreliable since they rely heavily on how well the tester's ears are trained. Best thing for you to do is just test things out yourself and decide from that without being influenced by either side.

post #343 of 1186
Quote:
Originally Posted by TheAttorney View Post
 

But the same tests are unreliable IMO at spotting more subtle-but-significant differences that you get with relatively neutral components like DACs and cables (ruler flat frequency response, negligible THD etc). Not imposible, just unreliable because of the mind tricks that have been fruitlessly debated a million times.

 

What this means is that if you run your fast-switching-DBT and find no difference between components A and B, I simply won't accept that result until you repeat the DBT in a longer term test, of a style that I would deem to be fool proof.

How do you know that these significant differences exist?

If they are so subtle why are they so significant?

post #344 of 1186

audio pretraining is legit in DBT - Double Blind just means controlling the source of information about the selection in each trial so there is no way to tell except by listening  - but the overall test protocol can tweaked for all sorts of hypothesis

 

there are positive and negative controls possible too - include some tests known to be audible, only score the subjects results on the unknowns that get the known audible tests right - detects the hypothetical "nocebo" bias, fatigue, or just plain inability

 

there are incompetent/pseudoscientific and well meaning but naïve listening tests all round - but those passionate about their pet audiophile theories, or taking money for their "professional" subjective reviews of audio equipment have in most cases had 30+ years for most audiophile controversies to refine, evolve scientifically acceptable listening tests proving their "night and day" sighted subjective opinions - and most audiophile verities still haven't made the jump into psychoacoustic text book knowledge yet


Edited by jcx - 4/13/14 at 5:48am
post #345 of 1186

I think some are missing a point here. is the test done for myself, or is it done for truth/everybody to acknowledge.

 

I do my tests for myself and draw my own conclusions. nothing wrong with that as it only concerns me listening to my music. I don't mind sharing what I heard, but then I wouldn't come here telling everybody that I know the truth. because I don't, and didn't have enough control over my experiments to remove uncertainty.

 

dbt is the next best thing available after measurements, and the most simple way to know for real if something is humanly audible or not. real dbt, not whatever switch done by myself. people tend to call dbt a lot of things that are not it.

if a few people come with a clear result from a real dbt done enough times to be statistically valid, and all settings of the experiment correctly done, then yes I will tend to believe in the conclusions of that dbt.

one person telling me he heard something, is just one person giving an opinion. just his pseudo, will be a bias to my trust in his opinion. what w838nwy decided to do will only serve to convince himself of something he wants to hear. it has zero value for us all, except being an opinion. I will gladly read it, it will not change my own. only dbt can do that to me because I trust the method.

New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Sound Science
Head-Fi.org › Forums › Equipment Forums › Sound Science › Audiophile cables, an interesting question.