I wouldn't get it either, even if I had the disposable income commensurate with it. If nothing else, DAC-technology is still evolving at a reasonably fast pace, a top of the line DAC purchased 7 years ago would be less than mid-end now.
My advice/'philosophy': a) take your time to get to know what you have and don't lose yourself in sites like this, there's always 'better' and many wizards promoting things they may not even have heard, far less compared to other gear. The taller the claim, the higher the burden of proof. b) Invest in quality music material after all, that's what this is all about and if that stinks, no expensive gear will fix it. c) Learn what your sound signature preferences are, once again, this takes time. d) Many roads lead to Rome, what's right for me/my music may not be right for you. e) Audition gear there's no substitute for your own ears. I had to purchase some items unheard due to my situation living in 2 countries and in retrospect, some things I'd have done differently.
For the rest, personally I like source/amp to be as neutral as possible and adapt the sound to my liking via a judicious choice of headphones. Just my opinion. Also, neutrality is not the end all and certainly does not equate euphony in all cases.
If I were you I'd take a break from tinkering, enjoy the music and then go out and audition something radically different to see how it compares. Try a tube amp/dac for instance, that'll provide significant contrast.
I agree with everything you've said. In particular I like to think of having a wide range of headphones with different sound signatures as a very expensive EQ.
I will be investing in a valve amp, particularly since I read they can work particularly well with the HD650's.