An amplifier's gain and its power are two different things as you know.
To make it clear, if you have a 10kW amp with unity gain, speak gain=0dB or 0x, and your source just delivers 2V, then it will easily drive low ohmic headphones. Will the p blow your ears? Depends of the efficiency, so how many volts does it need to play a certain volume.
If you have a sissy amp with "just" 0,1A or 100mA, but have a voltage swing of 30V, then it will not be appropriate for your 32Ohm cans, but will drive your 150 ohmic headphones with grace.
Power P=U²/R and P=I²*R. The lower power is the limiting real power. (Of cause the Ohmic Laws are only for steady volts, but you get the idea.)
Ah, so, your headamp will deliver the voltage and current to your headphone to the limit it is capable of. Ears are easily destroyed, even with today's standards of weak amps.