This amplifier discussion is interesting.
To add a few thoughts of my own:
Fortunately, there is a fairly wide selection of relatively high sensitivity speakers available today at affordable price points.
If one is listening to music with a near-field/ desktop setup, or in a smaller room, you really don't need very much power to get a satisfying SPL, even with modest-sized speakers.
*True* class-A operation, with either solid-state or tube amplification requires very hefty parts and lots of heat to dissipate --no way around it.
A 20-watt pure class-A amplifier is a BIG thing. And, 25-50 watts in class-A: ridiculously expensive and bulky if you want to do it properly.
A well-designed amplifier that stays in class-A for maybe the first couple watts, and then transitions into class-A-B up to, say, a 5-6 watt output, this should be all you need to drive essentially any electrodynamic headphone on the market today to levels beyond what is safe for your hearing --and IMO actually quite sufficient for most desktop/near-field listening too.
To build a 20-watt amplifier which sounds about as good as a very carefully designed 5-watt amplifier, there is a huge jump in parts cost.
BTW: for a couple years my main personal home rig was a 1-watt SET tube headphone/desktop amp driving a pair of large Tekton Design 'Lore' floor-standing loudspeakers (10" main driver). The Lores are approx 95dB -Watt/meter sensitive, and would "rock out" quite satisfyingly, with big solid bass and good dynamics. Granted, not suitable for pressurizing a discotheque or dance hall, but hey...
Efficient/sensitive loudspeakers really are a game changer as far as required amplifier power.