Headphones, source, amp, and DAC, in that order.
The headphones ultimately decide what you hear and how you hear it in the end with their own little colorations and presentations, though some of them need amps to sound proper (or even function at all, if we're talking electrostatics).
The source...well, that should be obvious. Garbage in, garbage out, as they say.
The DAC? As long as I'm not hearing any hissing or buzzing that obviously shouldn't be there (like with integrated motherboard audio or front-panel audio jacks), I'm fine. At least, I don't see the benefit for paying hundreds more over an X-Fi Titanium HD right now when those same hundreds of dollars could be used to buy better Stax gear...