Originally Posted by Joe Skubinski
I’ve done this exercise in the past to see where the pitfalls are. For me it was not a matter of proof but rather to see why blind tests are so difficult to use as proof of an audible change between two devices.
Looking back on my notes I can give you some things you’re going to need to do in order to improve your chances of making any comparisons successful. Blind testing for audible differences by repeated comparisons requires training and takes time to obtain meaningful results, as in lots of hours over a long period of time, like days or weeks. The good news is your ability to determine minute changes will get better with experience. The bad news is you need to continue to work at it until you reach your goal, whatever that may be.
Broken down into stages…
1. Train yourself as to what the differences are that you are listing for between two components under test. Take notes to narrow the comparison process you’re ultimately going to create. In other words, you’re not going into this blind but with as complete an understanding of the sonic characteristics of each device under test as you care to learn. The more you know about them the easier comparisons become. To not do so is the equivalent of expecting a dog to follow a scent without having smelled it.
If you have never done comparisons like this before, it would speed the process to have someone with you who has as they can quickly point out changes for you to hone in on. It all begins with hearing one difference, then comparing, and contrasting that with other devices under test; then the next difference, then the next, and so on, building a mental library so to speak.
2. Locate specific sections of song tracks that exemplify these differences. Note what you hear and the time index.
3. Create a play list of these short portions of specific tracks in a planned order to target differences and shorten test time.
1. Volume should be kept the same during any change, however it can and should be varied (while at the listening position) while a device is under test. Certain aspects become glaring at slightly higher volumes. Once you hone in on a sonic signature, the volume may not need be increased for the same result next time.
2. Quickly changing the devices under test then trying to listen over and over is tedious and stressful- ultimately this method of comparison does not work well. Even if the change is instantaneous as with a remote switch, there is no long-term benefit to this method nor is it a short-cut to discerning audible differences. You must learn differences while always second guessing yourself to confirm.
3. If you’re using speakers, don’t be afraid to move your head or listening position to better hear things within the soundstage. I’ve seen too many people stay fixed in the sweet spot even though what you really want to hear may not be exactly in that spot for different recordings. If you do not have a stereo image or soundstage within your set-up, this MUST be resolved first. Many differences are not huge but are easily heard as changes within the soundstage, for example vocals being more distant, nearer, or cleaner.
4. If using headphones, move the headphones about on your head track to track in an attempt to adjust acoustics, similar to above. With some headphones this may not make a difference but it’s worth a try.
5. Keep in mind you are always training yourself to hear an increasing number of differences in or signatures related to each device under test in order to more easily pick it out when you hear it again.
6. If there is any gap in making a change, say more than a minute, be sure not to make any rash judgments when listening resumes. Listen to a number of tracks to reinstate your memory. We’ll call this the one minute rule.
7. Create 10-15 tracks, and if possible shorten to just the portions where you hear a difference to ultimately have one continuous playback with gaps for quicker comparisons. Note order may matter.
8. Create a scoring method for the above. Maybe choose device A or B for each track section, and add the results. More A’s, must be device A.
Turns out doing blind listening comparison testing is not as easy as it would at first seem, even if you already know the outcome☺. It can be done with 9 out of 10 results, but it takes A LOT of practice.
The more practical approach to understanding the sound characteristics of a device is to listen to it over a period of time; hours, days, even weeks. This averages out to a better understanding of the sonic characteristics of the device within the context of your long-term listening style, and also allows time to assess other portions of the system and how their interaction may relate to what you are hearing. Always remember you are listing to the sum of the parts of an entire system, including the original recording venue and set-up. Tendency is to point at the last change as the culprit when it just might only be the messenger.