Here are my personal experiences... Of course YMMV.
First and foremost it has to be done "blind". Blindfolded, eyes closed, back to the rig...etc. So I always need the help of someone else (another head-fier) to plug and unplug without me knowing what I am listening too.
Secondly the listening environment needs to be completely silent for critical listening sessions. So unfortunately this negates most of the larger convention hall venue head-fi meets. Unless of course you are using IEMs.
Yes volume levels need to be equal. Louder sounds will naturally sound more dynamic. Really loud sounds (concert level) will sound more compressed. On top of that, the equipment+headphones will exhibit different levels of harmonic distortions, and introduce other "signature" resonant overtones-undertones at different output levels. It ALWAYS makes me nervous hot-plugging TRS connectors, so out of respect to others' equipment (any my own too!!) I ALWAYS plug/unplug with the volume at 0%. Unfortunately by the time I have my head-fi buddy "re-normalize" listening volume, my brain to some degree, has re-acclimated and adjusted to the signature of rig or piece of gear I am comparing.
I also need to play the same exact music passage/tune. Again, almost impossible to do without the help of a buddy... unfortunately.
Edited by kramer5150 - 2/11/14 at 10:12am