I would generally and always stay away from any kind high impedance outputs to avoid mismatch and change of FGs, even when some headphones like planars are not affected. All headphones I owned responded best at <1 Ohm - in technical terms.
I can see someone using a higher output to give his headphone more warmth (and lose detail in the bass for instance for more bloom) but if a headphone is that strident or bothersome then it should be replaced before experimenting with more gear. I wouldn't keep a Grado and do everything to make it sound "right". Waste of time and money. Look at how many people want to get the HD 800 "right" when in the end it's just not their signature. Yet they want to be in the "game" () and like it because a majority does, because the press does. There's so much psychology involved in wanting to like things. Let go, it's ok to not like the Sennie, the Fostex, the Hifiman or whatever.
The downside to a technical ideal in the chain is the price. After <1 Ohm output impedance you want noise and hiss-free signal with balanced armatures or sensitive headphones in general. That's where prices can go quite up, though in the end you can keep such a device for many many years.
If money allows then the cheapest soundcard with the same software features/dsp should be paired with a clean external device.