If you were using higher end headphones, like the HD600 or K701 open headphones, or something that had very high current needs, like a Denon, a better headphone amp might be needed. But, the internal headphone amp on the Duet 2 is actually very good, as it uses very low distortion op-amps (the chips that do the amplification), the Texas Instruments OPA1612 to be exact, which is much better in quality than what is in many under $400 amplifiers anyway. I notice a difference between the built in amp and the Duet 2 combined my portable Meier-Audio Stepdance (which uses 5 single-channel OPA1611 in a special configuration), but the difference is not night and day, and the Duet 2 actually is the warmer sounding amp of the two, with the Stepdance being more clinical and better-controlled in the bass regions, and both being capable of similar levels of clean output with a nice, three-dimensional soundstage. My point is, that while you may benefit from an outboard headphone amp, the Duet 2 is fine for the headphones you have. My experience with the Duet 2 is that it is even capable of powering the AKG K701's and Sennheiser HD600's very nicely, even though it isn't as good sounding as the Stepdance with my Sennheiser HD598, which seems to want a bit more control in the bass regions than the Duet 2 gives it. The headphone out of the Duet 2 is very neutral in my opinion, but has enough warmth to not sound clinical, and when driven within reason, is very clean as well. I hope this helps.
--Eric