I don't think that playing an instrument helps in judging sound quality, because musicians are located on the other side of the instrument than the audience, and some studies show that usual instruments have a very different spectral balance according to the direction of sound emission.
Think about brass instruments. They are very directive. They mostly radiate sound forward. The sound that the player is used to hear is very different from the "right" sound. Asking a brass player to judge the sound of speakers would be like asking someone who has listened to speakers from behind all his life !
Even violin has a very chaotic response according to the direction. They radiate a lot of high frequency frontwards and upwards, above the audience, where the microphones are often located. Sound engineers must take this into account. A microphone recording violins from above the audience will get harsh sounds compared to the one that the audience listens to, which is in turn different from what the violonist hears.
Soundstage is also very different. A concertist is used to hear instruments all around him or her, including behind. That's not a good reference for judging the soundstage balance !
Sean Olive has published an article about hearing training. His findings show that trained listeners, blind listening to speakers, give much more stable and reproductible ratings than untrained listeners :
Audio Musings by Sean Olive: Part 2 - Differences in Performances of Trained Versus Untrained Listeners
The funny part is that audio reviewers scored lower than audio retailers ! The score is an indication of reliability, but I didn't understand well how the score is calculated. It seems that it is not a DBT performance, but rather a indication of the reliability of the listeners ratings against themselves. Higher scores would mean repeatable ratings, and lower score variable ratings.
In may 2009, Sean Olive said that he was considering publishing his training software :
"Harman's How to Listen" - Hydrogenaudio Forums
I recently posted some samples in order to help people training with ABX tests. They are not representative of all sonic differences that one is supposed to hear, but unlike most ABX tests, they focus on hardware differences rather than mp3 differences. They feature two vinyl vs CD comparisons, one comparison between two different masters, a comparison between two headphone amplifiers, and a comparison between analog and digital mixing.
Very easy : vinyl vs CD. Vinyl taken near the end of a 33 rpm side, where distortion is maximum. The frequency responses differ by about 10 or 15 dB.
http://3141592.pio2001.online.fr/fil...n/005%2021.mpc
http://3141592.pio2001.online.fr/fil...n/006%2022.mpc
Very easy : two different masters, both taken from CD. The frequency responses differ by about 5 or 10 dB.
http://3141592.pio2001.online.fr/fil...Balance-RE.mpc
http://3141592.pio2001.online.fr/fil...Balance-US.mpc
These two tests are not really aimed at testing the hearing ability, but rather to familiarize the listener with the ABX software and procedure, and to make him or her confident with the process of DBT.
More difficult, vinyl vs CD, but taken from the beginning of a 33 rpm side, where distortion is minimal (this is the same vinyl / CD couple as above !). The frequency responses are within 3 dB.
http://3141592.pio2001.online.fr/fil...n/001%2011.mpc
http://3141592.pio2001.online.fr/fil...n/002%2012.mpc
The difference is still plainly audible, but this time, it is small enough to confuse the listener in the ABX process. It is necessary not to rush, to be careful about listening fatigue, and to maintain the attention focused on the difference in order not to make mistakes.
Now, the two difficult tests :
http://3141592.pio2001.online.fr/pic...tz%20K400.flac
http://3141592.pio2001.online.fr/pic...ct%20K400.flac
This is the comparison between two headphone amplifiers. The signal was taken from the output while the amplifier was feeding the headphones. The frequency responses are within +/- 1 dB for musical frequencies, with a +2 dB at 16 kHz in the Marantz file for those who can hear it.
Some listeners find this test more difficult than the next. It may be due to the fact that the difference is mostly located in low frequencies, while the other tests have all significant differences in the high frequencies.
Clue : in the Marantz file, the bass semiquavers running from an ear to the other sound heavier (+1 dB at 100 Hz). In order to succeed the ABX, I had to rise the playback volume.
Another difficult test : an analog mix emulated on Pro-tools. Taken from the digidesign challenge.
http://3141592.pio2001.online.fr/fil...gn/Jazz2A.flac
http://3141592.pio2001.online.fr/fil...gn/Jazz2B.flac
The Jazz2B file has barely +1 dB at 10 kHz, but frequency response might not be the main difference.
All these files have been successfully ABXed by several listeners.