Blind cable testing: initial report
Jul 11, 2009 at 7:54 PM Post #46 of 128
Interesting. Thanks for the report.

In my case, these "fresh ears" impressions are mostly attention unfolding. So I know that I would never try to match a "fresh ears" impression with the hifi involved.

It recalls me the first blind test that we did about interconnects, in France. Listening to a soprano with cello, after several trials, the voice was suddenly misplaced a bit on the left side instead of the center of the soundstage.
The operator checked the amplifier balance, but it was ok. Then he disconnected and reconnected the speaker cables, and the voice was back in the center of the soundstage !
After some more trials, I became aware that the soprano was moving in front of her microphone, and that her voice was actually always going from the exact center to the left side, just a tiny bit, several times at some given moments in the track.

Interconnects, balance and speaker cables had actually nothing to do with it.

Keep us informed about your next trials.
 
Jul 15, 2009 at 8:53 PM Post #47 of 128
I had a thought.

I've been calling it an ABBA/ABAB protocol. A and B are randomly assigned to the expensive cable and cheap cable. There are two results.. that is, I answer either ABBA or ABAB.. so I have a 50% chance of getting it right by guessing.

So far I'm 3/4.

Note that in this protocol I don't have to identify A or B. As it happens, though, I did correctly identify A and B in each trial --- assuming you use my impressions from the first two listens. Now, you could say this is cherry-picking the data, but it does fit with my theory that you need "fresh ears" to be sensitive to cable differences.

So, going with that theory, there is now a 25% chance of getting the fully right answer by guessing. There is also a 25% chance of getting a "half-right answer"---that is, identifing A and B based on the first two presentations but failing to identify the 3rd and 4th presentations.

There is a 50% chance of getting it all wrong.

So now the question is: what is the probability of getting the results I did---3 fully right and 1 half-right---by guessing alone?

I don't know how to do the math on this one---it's kind of tricky---so I wrote a program to run a Monte Carlo simulation.

The answer is 12.5%. I have a 12.5% chance of getting the results so far by purely guessing.

So this doesn't reach a 5% significance yet, but it's an interesting angle.
 
Jul 15, 2009 at 10:13 PM Post #48 of 128
Okay I just realized I analyzed that wrong.

This, I believe, is more correct.

Each trial in this test really consists of two trials, or rather two questions that each have two answers. The first question is:

- Considering just the first two listens, what is the identity of A and B?

The second question is:

- Considering the second two listens, would I choose the overall order as ABBA or ABAB?

So in this analysis, we have done 8 trials, and I have gotten 7 right. This reaches a significance level of (8 choose 1) / (2 ^ 8) or 3%.

So I have already succeeded.

Of course, this is a bit of post-hoc analysis and I'm sure most of you will reject it. However, it does fit with my theory that fresh ears are the most sensitive.

Now, what I should do is start a new test with this idea as the new protocol. However, I have yet another idea I would like to test, maybe first. Here is the second new idea:

This test takes two days, a week apart. I pick 8 test tracks. On both Day 1 and Day 2 I listen to the 8 tracks. However, for each track there is a random assignment of cable A (the cheap cable) or cable B (the expensive cable).

As I listen to each track, I write down my impressions of it, and try to assign a score (from 1 to 10) to various aspects of the sound, like the highs, the dynamics, etc.

On Day 2, the cables will be switched. That is, whatever cable was used for test track N on day 1, now the other cable will be used for test track N.

After Day 2 is over, I compare notes. For each track, I see if I rated it more highly with the good cable.
 
Jul 16, 2009 at 12:00 PM Post #49 of 128
i don't see what good it would be to rate from 1 to 10 and compare later. i think you should just continue what are you doing, do more trials. it would be interesting to see the score after 10,20 or 30 trials. it's more important to find out if there is any difference in first place and deal with what is the difference later, once it is proven.
 
Jul 16, 2009 at 3:49 PM Post #50 of 128
Quote:

Originally Posted by mike1127 /img/forum/go_quote.gif
I had a similar experience in the second trial... by the time I got to the third sub-trial (the third time I had heard the test track) I had lost all "freshness of mind." In the second trial I guessed the ordering wrong, but I still was right about the identity of A and B... in other words, in the first two sub-trials I guessed correctly which was the bad cable and which was the good... I just couldn't prove to detect the third and fourth sub-trials correctly.


I don't quite understand here. Are you saying in the second trial you were about to discern differences between A and B such that you knew which one was the cardas and which one was the ratshack. That is to say you at some point was able to determine either A > B or B > A. But when you went on to the 3rd and 4th subtrial you were unable to identify them?
 
Jul 16, 2009 at 6:23 PM Post #51 of 128
Quote:

Originally Posted by m3guy /img/forum/go_quote.gif
I don't quite understand here. Are you saying in the second trial you were about to discern differences between A and B such that you knew which one was the cardas and which one was the ratshack. That is to say you at some point was able to determine either A > B or B > A. But when you went on to the 3rd and 4th subtrial you were unable to identify them?


Just for clarity, I'll restate the protocol. I think you have it, but I just want to be clear. Each trial consisted of four subtrials, which could be either ABAB or ABBA. And A and B were randomly assigned to the Cardas or Rat Shack. Note I'm using the term "subtrial" to mean one listen to A or B, so the single trial consists of four subtrials.

Okay, so here's what happened in the second trial:

Subtrial one: It sounded relatively grainy, poor microdynamics, etc. compared to how I'm used to the system. I say relatively because it's not a huge effect. I suspected the Rat Shack
Subtrial two: Much improved, smooth highs, beauty, etc. I suspected the Cardas.

Subtrial three: Here's where the difficulty began. I had to try to figure out whether the sound had changed from subtrial two. I was not sure. Having just heard the good sound and finding myself involved with it (for example, really "grooving" on the dynamics), I found myself involved with the sound in subtrial three. I was pretty sure, then, it was the Cardas.

Subtrial four: By this time, I had a pretty strong expectation that the sound would be poor now, because I was sure that subtrial three was the Cardas. So I listened and thought the microdynamics were poor.

Then I said to my helper, "Okay based on the first two subtrials, I'm guessing that A was the Rat Shack and B was the Cardas." He said, Yup. Then I said, "Okay, and the ordering was ABBA." He said, Nope. We pulled away the sheet hiding the system and to my astonishment, the cable I had just been "trash talking" in my mind was the Cardas.

My theory, now, is that freshness of mind is very important in hearing differences between cables. It was obvious to me during the first two subtrials that B > A. But by the third subtrial I felt that it was harder to differentiate, partly because I was fatiguing to the musical details, that is, to how I was getting involved with the sound.
 
Jul 16, 2009 at 6:33 PM Post #52 of 128
Quote:

Originally Posted by dex85 /img/forum/go_quote.gif
i don't see what good it would be to rate from 1 to 10 and compare later. i think you should just continue what are you doing, do more trials. it would be interesting to see the score after 10,20 or 30 trials. it's more important to find out if there is any difference in first place and deal with what is the difference later, once it is proven.


The good it would do is this:

If cables matter, as far as I'm concerned they should matter long-term, not just when compared to each other back-to-back. That is, a better cable should bring increased enjoyment each time I listen. If a cable can only be discerned through a back-to-back comparison, then I'm not sure I want to pay $500 for one.

In addition, it appears that I am most sensitive in a fresh listen. I'm concerned that my ABAB/ABBA protocol has a weakness in that I am forced to listen to the same music four times in a row.

In this "two day/rating" test, I only have to listen to the music once each day.

Now, you might be confused about the rating scheme. The point here is not to decide the magnitude of the difference or characterize the difference. The point is to use a specific method to decide whether A is better or worse than B. It might seem overly complicated to have several categories and a rating scale. But think about this: it's what I do anyway! When I'm listening and trying to make up my mind whether a cable is good or bad, I'm actually noting different categories and thinking to myself, "Okay that was pretty good. Okay that wasn't very good." So the rating scheme simply captures what is happening anyway.
 
Jul 16, 2009 at 7:40 PM Post #53 of 128
mike1127, I applaud you for spending the time to come up with a protocol and test it. Thank you for your efforts.

In response to your last post toward my reply, you do say that you were unable to accurate discern the two cables at some point. Then you claim "freshness of mind" is important to hearing differences for you. I don't have any against your claims.

If something should matter long term then why does "freshness of mind" matter? If I were to use a "better" cable for many hours a day, would the "fatigue factor" set in and would I lose "freshness of mind"?

Just in case you need to know, I listen to music off my mbp through an m-audio transit to my esw10jpn. Sometimes I listen in my living room. That setup is mbp to transit to audioquest optical to denon avr-3300 to audioquest gibraltar to paradigm monitor 7 v2. I usually spend at least 6 hours listening to music M-Th and almost no music on the weekend.
 
Jul 16, 2009 at 8:33 PM Post #54 of 128
Quote:

Originally Posted by m3guy /img/forum/go_quote.gif
mike1127, I applaud you for spending the time to come up with a protocol and test it. Thank you for your efforts.


You are welcome.

Quote:

In response to your last post toward my reply, you do say that you were unable to accurate discern the two cables at some point. Then you claim "freshness of mind" is important to hearing differences for you. I don't have any against your claims.

If something should matter long term then why does "freshness of mind" matter? If I were to use a "better" cable for many hours a day, would the "fatigue factor" set in and would I lose "freshness of mind"?


"Freshness of mind" refers to condition of listening to fresh music. So if you spend your six hours listening to the same music 30 times, you won't have freshness of mind. If you listen to 30 different songs, then you will. So the theory is: cables matter under normal listening conditions, which are (A) listening to new songs and listening once to each, (B) listening for enjoyment.

If you really enjoy a song while listening for enjoyment, then you might listen to it multiple times and enjoy it each time. The theory I'm testing is that you will tend to enjoy it regardless of the cable you are using, because you have become really involved with it. Under those conditions you are less sensitive to cable differences. The theory I'm testing is that under these conditions, you will most enjoy it once you have heard the good cable, because that gets you really involved with it.

EDIT: let me make this clearer. Call A the good cable and B the bad cable. Let's say you listen in the order:

A (really enjoy it and get involved) -> B (still enjoy, less sensitive to drop in cable quality) -> A (assuming you are still enjoying the music, you will keep enjoying it)

or let's say you only use cable B:

B (don't enjoy it as much) -> B (not as much) -> B (not as much)

or

B (don't enjoy it as much) -> A (notice improvement, enjoy more and get involved) -> B (still involved, not as sensitive to cable differences)

Something like that. This doesn't have to be a precise theory or predict "enjoyment", but it is a testable theory.

Of course, people have very different listening styles. This whole theory might describe me and a few people like me (maybe you?), while failing to describe other people. Also, those other people also might not be sensitive to cable differences at all.

-Mike
 
Jul 17, 2009 at 10:32 AM Post #55 of 128
Quote:

Originally Posted by mike1127 /img/forum/go_quote.gif
You are welcome.



"Freshness of mind" refers to condition of listening to fresh music. So if you spend your six hours listening to the same music 30 times, you won't have freshness of mind. If you listen to 30 different songs, then you will. So the theory is: cables matter under normal listening conditions, which are (A) listening to new songs and listening once to each, (B) listening for enjoyment.

If you really enjoy a song while listening for enjoyment, then you might listen to it multiple times and enjoy it each time. The theory I'm testing is that you will tend to enjoy it regardless of the cable you are using, because you have become really involved with it. Under those conditions you are less sensitive to cable differences. The theory I'm testing is that under these conditions, you will most enjoy it once you have heard the good cable, because that gets you really involved with it.

EDIT: let me make this clearer. Call A the good cable and B the bad cable. Let's say you listen in the order:

A (really enjoy it and get involved) -> B (still enjoy, less sensitive to drop in cable quality) -> A (assuming you are still enjoying the music, you will keep enjoying it)

or let's say you only use cable B:

B (don't enjoy it as much) -> B (not as much) -> B (not as much)

or

B (don't enjoy it as much) -> A (notice improvement, enjoy more and get involved) -> B (still involved, not as sensitive to cable differences)

Something like that. This doesn't have to be a precise theory or predict "enjoyment", but it is a testable theory.

Of course, people have very different listening styles. This whole theory might describe me and a few people like me (maybe you?), while failing to describe other people. Also, those other people also might not be sensitive to cable differences at all.

-Mike



Relative to my information gathering post, might you clear this up for me.

What's wrong with putting a splitter on your DAC and running two different cables to an amp with two inputs like my GS-1 and then going back and forth so you can compare the decay of a cymbal crash for instance, or the leading edge of a snare drum attack or the size of the sound stage?

USG
 
Jul 17, 2009 at 10:42 AM Post #56 of 128
Quote:

Originally Posted by upstateguy /img/forum/go_quote.gif
Relative to my information gathering post, might you clear this up for me.

What's wrong with putting a splitter on your DAC and running two different cables to an amp with two inputs like my GS-1 and then going back and forth so you can compare the decay of a cymbal crash for instance, or the leading edge of a snare drum attack or the size of the sound stage?

USG



That might be a good idea relative to your listening style or the things you listen to. If you are listening to something like the "beauty in the ambience" it is hard to detect that during quick switching---my experience. Some people theorize that things tend to sound more the same when you are quick switching because you no longer have that "freshness" and your listening style has changed dramatically from "listening for enjoyment" to something more analytical. It's just a theory, and your experience may vary. I would interested to know whether you introspect on your listening process, so that you would tend to notice conditions in which you are less sensitive to differences, or if you rarely make an observation like that.
 
Jul 17, 2009 at 10:57 AM Post #57 of 128
Quote:

Originally Posted by mike1127 /img/forum/go_quote.gif
That might be a good idea relative to your listening style or the things you listen to. If you are listening to something like the "beauty in the ambience" it is hard to detect that during quick switching---my experience. Some people theorize that things tend to sound more the same when you are quick switching because you no longer have that "freshness" and your listening style has changed dramatically from "listening for enjoyment" to something more analytical. It's just a theory, and your experience may vary. I would interested to know whether you introspect on your listening process, so that you would tend to notice conditions in which you are less sensitive to differences, or if you rarely make an observation like that.


floatsmile.png
How long have you lived on the West Coast Mike?
beyersmile.png
Perhaps we could leave "beauty" out of this since no two people are going to view "beauty" the same.
atsmile.gif
She's a fickle lass you know.
bigsmile_face.gif


There's no one there to tell you when to flip the switch... you can quick switch, slow switch or not switch at all.
beyersmile.png


Why would things tend to sound the same? Let's say you are evaluating bass, for example. You switch back and forth and in a second you know which has more bass.....

USG
 
Jul 17, 2009 at 11:46 AM Post #58 of 128
I did a (sort of) double-blind test between vinyl rips of The Beatles. Did I prefer the sound of the Japanese red monos or the 70's monos? I could barely differentiate when I would listen to them one at a time and over a period of time. It wasn't until I put several (of the same) files on shuffle in iTunes, looked away, and went to task. In the end, the detail and bass of the red monos was better. Had I not "switched" back and forth; I never would have discovered the difference.

Of course, all this went out the window with the announcement of the remasters, but what are you gonna do?
redface.gif
 
Jul 17, 2009 at 1:58 PM Post #59 of 128
Quote:

Originally Posted by upstateguy /img/forum/go_quote.gif
floatsmile.png
How long have you lived on the West Coast Mike?
beyersmile.png
Perhaps we could leave "beauty" out of this since no two people are going to view "beauty" the same.
atsmile.gif
She's a fickle lass you know.
bigsmile_face.gif



What does "two people viewing it the same" have to do with anything? I'm comparing my own impressions of two devices. And it's common for devices to differ in beauty.

Quote:

There's no one there to tell you when to flip the switch... you can quick switch, slow switch or not switch at all.
beyersmile.png


Quick-switch refers to the fact that the output switches from one to the other instantly when you throw the switch, not that you go back and forth rapidly.


Quote:

Why would things tend to sound the same? Let's say you are evaluating bass, for example. You switch back and forth and in a second you know which has more bass.....

USG


As I wrote in, "That might be a good idea relative to your listening style or the things you listen to." Some things ARE probably more obvious in quick-switching. I wouldn't know because I never use it, but people who do use it often report that certain details are very obvious.
 
Jul 17, 2009 at 3:13 PM Post #60 of 128
Quote:

Originally Posted by mike1127 /img/forum/go_quote.gif
Quick-switch refers to the fact that the output switches from one to the other instantly when you throw the switch, not that you go back and forth rapidly.


I assumed you meant the other as well. Seems to me that the rapid loss of music memory over even a short time lapse between switches would trump "imagination contamination" or lack of "freshness of mind".
Given this new understanding of what you mean by "quick-switching" I must say that I find your argument weak.
 

Users who are viewing this thread

Back
Top