ABX Reliability - Page 3

Quote:

Originally Posted by Ham Sandwich 

Have you ever tried the Foobar ABX testing plugin?  You can flip rapidly between A, B, X, and Y as many times as you want for each test.  There is no gap or false delay between flipping from one to another.  You can repeat short sections or listen to longer sections.

I'd love to try a similar test with my AKG stock and ALO SXC Cryo cables.  I have two concerns however:

- a cable connector that becomes worn or worse, damaged by repeated switching of cables.

- my headphone remaining in the same position while switching cables.

- genuine volume equalization

- my personal realization that subtle differences are best appreciated after listening for prolonged periods to one signature and then switching to the other.  Switching back and forth makes the difference very difficult to appreciate.  Even after the initially appreciated difference, playing with both signatures to further characterise the difference leads to that loss of discernment.  A very funny thing.  In fact, when this happens I have to make the decision whether or not one signature is in fact better in that I do prefer it or if it's just different in a subtle way.  I do wonder about the long term worth of such differences.

Quote:

Originally Posted by Prog Rock Man 

The Seiveking test to me shows that if you make a huge effort, suspend any enjoyment in what you are doing and concentrate like hell, you may see a difference. But what is the point in that? It has proved that there may be tiny differences that really do not matter and are actually counterproductive because you no longer get any enjoyment out of looking at nice colours.

It's not whether the differences matter or not at this point.  It's whether or not there are at all, any differences.  There being no audible difference has been the strongly defended scientific FACT concerning cables and interconnects.
 

Quote:

Originally Posted by Slaughter 

I'm in total agreement about the cost v. benefit and in some cases cheaper equipment is better or preferred over something more expensive. I just don't understand why ABX is used to prove or disprove what people believe. It's like a lie detector test. Some take its results as fact, some use it as a basis for argument and others despise it. But police still use it and courts don't. Another piece of science gone awry.

I agree here.
 

Quote:

Originally Posted by swanlee 

Great but the ABX still had zero value and in the end was a waste of time as I obviously am not going to swap cables to ones I DON'T like the sound of in my home just because of some ABX test.  I have also switched out equipment and cables with less expensive and uglier choices that in my setup sounded better. Not all of us  upgrade with more expensive prettier equipment some times a better made cheaper product will sound better but in an ABX test I'd get a null result which would have no bearing on my decision.

 

If I always get null results and those null results have no impact on my decision it is a waste of time.

 

Heck just look at my sig I recently replaced my 300$ zune and 300$ xm5 amp with an SFLO:2 that only cost 150$ and has a much less pretty interface. I did this because it sounded better right off that bat, I'm sure I'd get null results in an ABX test, but do I care? Nope it would not provide me with anything worthwhile.
 

Interesting that you'd prefer the uglier and cheaper cables.  I'd not switch behaviour since science claims that you're fooling yourself.  Science and its methods don't always lead to the ultimately correct conclusions.  If you run the same experiment, everytime, 50 times over, why should the results necessarily be different?  In fact, the more you run the same experiment, getting the same result, the more it becomes time to move on to other questions or CHANGE the method of experimenting to prove what it is you're not so convinced science has provided the ultimately correct answer for.



 

Or you could always use Nick Charles solution, pick different cables out of the DAC, and re-AD the signal and compare the AD-ed data.
You still won't know if you could hear a difference but you'll know if there was a difference in the first place, and what type of difference if you know how to correlate digital data and hearing impressions.

I disagree about the memory of sound being short term. We are programmed to remember sounds. If we were not we could not learn language. If we struggle to remember slight differences in tone, pitch or whatever, then those differences are not important.

I have specific tester tracks for auditioning. I know them very well and on each I am listening out for specific things such as background noises, distortion of bass, dynamic shift from verse to chorus, sibilance with cymbals. That way i can listen to one bit of kit one week and another a week later and make an accurate comparison.

After a failed first attempt (didn't really concentrate) I scored 10/10, tried again with a different color: 9/10.

It's important to memorize the colors in the beginning, and in case of my laptop monitor not to move your head. Different angles will give you different colors. :-/

Anyway, a real ABX test would allow me to switch between the colors - guess that would make it a lot simpler.

Ahm, what is the point of this thread?

Quote:

Originally Posted by xnor 

 

 

Ahm, what is the point of this thread?

There are two camps regarding cables. One say they make no difference and ABX/blind tests prove that. The other camp say ABX/blind tests are flawed and there is a difference between cables. This thread is about whether ABX/blind tests are valid. If they are valid, then people who buy and sell expensive bespoke audiophile cables are ripping/being ripped off.

Quote:

Originally Posted by Prog Rock Man 

The other camp say ABX/blind tests are flawed and there is a difference between cables.

Have these folks ever done such a test? Did they fail? Were they disappointed and is that the reason for questioning such tests?

Why should ABX not work?

- Just ABX a 128 kbps mp3 sample - easy. Take a 320 kbps mp3 (considered transparent) and you'll fail.

- Take a very, very long and thin cable and compare it to a normal, short one - easy. Take a cable with similar length and gauge as the normal, short one and you'll fail.

What's wrong with that, and why is transparency so hard to understand? What exactly is flawed? Why shouldn't this test work with any other component in the audio path?

edit:

And what's the point of the linked test? I can measure the effects of rising air temperature in my room on the propagation characteristics of sound waves - I can "see" the difference, but do I enjoy my music less in winter than in summer or vice-versa, no. Is it very hard to make out differences with my hearing? Yes, very much like that color test if you don't amplify the color difference, or if you're colorblind. 

Quote:

Originally Posted by Slaughter 

xnor, you don't get it.... You are the only one who has admitted to passing the test by the way, which should happen in with ABX. As I understand it, 5% of people are suppose to pass an ABX test. Anything less than 95% correct is considered as insignificant results, meaning there is no difference in what you saw or heard. When you scored 9/10, ABX says that you cannot tell the difference between the two images, at least to a degree that has any scientific significance. How does that make you feel about ABX?

Slaughter, don't you get it. In normal ABX tests, the testers are allowed to switch back and forth between their choices as often as they want before making a decision. This test requires that the two colors are memorized first, because it does not allow for switching. It is a DBT test with an agenda, intended to point out a potential flaw that regular ABX tests either will not suffer from or will suffer from in a reduced amount.

Additionally, we're talking about a very minor difference between the colors. Would you pay $1000 for a cable that makes this little of a difference?

I did all 20 earlier today and got 14 right. Not statistically significant on its own, but I know that most of those 6 misses came from the beginning. After making several choices I began to recognize the difference. That's something that a normal ABX test would have allowed from choice 1, and so I would have scored (nearly) 20 out of 20. I also chose one wrong in the middle somewhere, and realized it after I moved to the next question because I recognized the slightly lighter shade.

Quote:

Originally Posted by Ham Sandwich 

Well that's one way to turn that AB test into an ABX test.  Take a screenshot at the beginning of the test showing both A and B.  Then toggle between the screenshot and the sample throughout the test.

 

Have you ever tried the Foobar ABX testing plugin?  You can flip rapidly between A, B, X, and Y as many times as you want for each test.  There is no gap or false delay between flipping from one to another.  You can repeat short sections or listen to longer sections.

 

If your complaint is that audio is transitory and therefore cannot be compared then why even bother with subjective listening reviews.  What are the subjective reviews comparing against?  How could listening impression be compared?  Yes, comparing audio is more difficult that comparing static images.  That doesn't mean you give up and say to heck with statistics and trying to develop statistically sound listening experiments.

You can also do rapid switching back and forth with a hardware solution with a switch box.  But it requires two identical sets of sources sometimes, or at least a decent splitter / adapter cable from the same source when comparing headphone amps or cables.  When comparing different audio player sources, it'll be a bit of hit and miss timing of pressing play at the same time.

In the pic below I used a headphone switch box to compare the two different internal amp cards available with the HM-801.  Since the amp card cannot be quickly swapped back and forth, I loaded the same song files and pressed play at the same time, then volume matched them before comparing.

-Ed
 

Quote:

Originally Posted by Slaughter 

 ABX is a failure.

No.  The particular implementation of an ABX test by Sieveking Sound is a failure.  And it is intentionally designed to be a failure and an improper implementation.  If Sieveking had instead set out to demonstrate that ABX testing is a valid way to test for audible differences in audio the test would have been designed differently, more like the ABX test as done in Foobar.

And you don't need to get 95% right to show a positive result.  What you need is a 95% confidence level that the result was not due to just plain guessing or chance.  So in the end you can say with 95% confidence that the subject did hear a difference and properly identify A and B.  That is completely different than needing to get 95% right.

I'm not going to run the numbers because I don't have my statistics books handy and it's been over 20 years since I took design and analysis of experiments, but I suspect that 14/20 would meet a 95% confidence level for an ABX test.  And thus Head Injury passed the ABX test.

Quote:

Originally Posted by Ham Sandwich 

I'm not going to run the numbers because I don't have my statistics books handy and it's been over 20 years since I took design and analysis of experiments, but I suspect that 14/20 would meet a 95% confidence level for an ABX test.  And thus Head Injury passed the ABX test.

I doubt it. I certainly wouldn't trust a 14/20 in a scientific experiment. If it does, I'd be surprised.

Slaughter, how am I supposed to open it in a second browser if the color is different every time? Surely you don't expect me to keep refreshing a window over and over until the colors are exactly matched? Because even if it's close, a single point more or less in one color will still throw the comparison off, as evident in the two options given. Like I said, the test was designed to create failures.

Slaughter,

Here is a bellcurve example for Adult Intelligence.

According to this I can state there is a 95 percent chance your intelligence falls between 70 and 130 without even seeing you.  There is also a 2% chance you are either very stupid or clever. 

The 95% confidence is not some quirky number used by ABX testers, it is used by millions of scientists and engineers all over the world.

You had better hope it works, it has been used it to predict the strength of aircraft and nuclear reactor materials for the past 40 years.

The article you link to is extremely misleading in this aspect.