Many headphone produce over 100 dB SPL with just 1 mW of power. Assuming it's 100 dB SPL @ 1 mW and that the average RMS amplitude of your music is -15 dB you're still at an imo loud 85 dB SPL.
Into 32 ohms the corresponding volume position would be around 7:30, into 300 ohms about 9:30.
If you're listening to classical music you might have to turn up the volume more, but if you're listening to dynamically compressed stuff you will run into permanent hearing damage.