Output impedance is not the only metric that you will see an improvement when using a dedicated headphone-only amp. Distortion, noise, etc... will also see an improvement, though these improvements may vary depending on the model. Your question is basically the same as "why do I need to upgrade to a 2015 BMW sports car when my 1973 Honda Civic still gets me to the supermarket and back?" So... with that, I hope you'll arrive at the answer that works for you.
6/3/15 at 10:51pm