I had the exact same experience as you guys.
I found the Reference 5.32 to be too dark for me. As my experience afterwards showed me, I'm really "sensitive" to darkness.
Overall it might not be that dark and certainly much less than the Mad Dogs or the HE-400 (if it makes sense to compare DAC to Headphones... :D).
Stock, the sound lacked "bite" with my HE-500 (which are not the fastest headphones around, I reckon). I then performed the cap-mod as indicated in the manual to increase brightness.
It was still dark but overall better in the areas I found the stock version to be lacking. The thing is, I never could really focus on the sound to make a proper opinion of the sound. I was disturbed by something (not audible) that simply caused pain in my head and forced me to open my eyes.
I don't know if the unit was faulty or if, as Kingwa mentioned, the frequency response in the 1-4kHz was just not to my liking and provoked that disturbance. Honestly I didn't hear any flaw in these frequencies, the voicing was normal to my ears (no recess or anything).
I ended up returning the Ref5 and ask for upgrading it to another DAC.
Since I found the Ref5 to be lacking dynamics, I upgraded to the NFB-27 which should fit the bill.
Dynamics it indeed has! Lots of it! I never felt it was harsh sounding, definitely brighter than the Ref5 but, in my book, closer to neutral than anything else.
The detail retrieval is excellent, so are treble and bass extensions.
Yet, once again, I'm not fully relaxed when listening to it... It's nowhere near the level of the Ref5 but, still, I feel stressed.
My opinion now is that the bass is just too tight for me and it hard presses my small head.
I'm no cable believer but it is plugged into a cheap power strip with a cheap AC cable. The unit seems really sensitive to power noise: each time I turn the light ON/OFF, the DAC stops for few seconds!
I still need to investigate this.
Bottom line: if you want a Ref5 natural sound with the NFB-7 voicing, get yourself a Metrum DAC ;)