Wow, Rob. That was a great summary of what you go through. I am so thankful for that; it gives me a much better feel as to how this beast of many heads gets tamed. And for what to look (listen) for when the next iteration of the WTA, etc gets released. Thanks much
P.S. Oh, and in the world of "geniuses think alike" I have both that Norah track and the Moby 18 transitions from We Are All Made Of Stars -In This World-In My Heart (great trio of dynamic but different sounds) in my demo list. Interesting that you use Norah for both female vocals AND that double bass. Nice! Wonder if those ab changes also surface on the 24/192 or DSD (Analogue Productions, that is) versions of that track.
I forgot to mention about perspective, or trying to scale the magnitude of an improvement. Doing the sound stage depth test, I find that its very easy for the the ear/brain to hear very small changes in depth perception - for example, listening solely to the depth on the organ track, you can easily tell if one is deeper than the other - if A say is deeper, then B sounds "flat as a pancake" even though the organ may sound 100 feet away on A and is probably 95 feet on B but the relative shift in placement on the AB test is easy to hear. It is very easy for small changes to take on the perspective of an enormous change. So to get changes into perspective, I use the general test, and gauge my reactions to the change. For example, if you listen the next day, and don't notice the improvement, then you know what one was doing the day before was pretty small. So in essence you measure how long it takes for the improvement to fade and for one to get used to the sound.
Hearing an improvement after:
1 day - small improvement
1 week - substantial worthwhile improvement
1 month - very substantial
3 months - very rare breakthrough events
more than 3 months - earth shattering change
For example, the Olimex USB isolator was in the 1 week category - I was still noticing the improvement one week on. So one week on I felt like I had still had a new toy, and enjoying the extra warmth and depth. But a month later, I was no longer thinking the system sounds better, even though if you went back it would sound much worse.
Now this won't work if the product has a big break-in period, or if you are constantly making small changes. It won't work too if your system is variable (good at certain times, bad at others). My designs do not have a big break-in period, as I know why the components age and I try to reduce the sensitivity to aging through design - this also improves SQ as well, as a component will never age to perfect performance. That said, Hugo does have a break-in, but this is more due to one getting used to the sound - breaking in your brain if you like!
I have only ever had one instance of the earth shattering change....