Let's see: 1 watt = 98db, 2 watts =101db, 4 watts = 104 db, and 8 watts = 107 db at each speaker. That's the equivalent of listening to two power lawnmowers 3 feet from each ear.
It isn't just the volume (decibels) available from the speaker. There needs to be enough power on tap to 'control' the speaker and provide good dynamics and bass. That is often much more power than the mathematics suggests. Having at least 20dB on tap beyond your normal listening level is a good idea.
Good example - the Auralic Taurus should have enough juice to drive the HE6 - and it will. It will even go loud with it, but the sound is thin, weak and anemic compared to what a more powerful amp will do (like the WA5 or 430HA).
I like 300B tube amps for headphones - 8W will drive anything (including the HE6) very well indeed. Probably massive overkill for anything but the HE6.