I'm currently at 3 o'clock volume at high gain and low impedance on my WA33 EE JPS to drive my Abyss 1266s. Don't worry, you aren't going to blow up anything. I think music sounds significantly better using low gain, but it doesn't drive the TCs loud enough, even with my DSP volume maxed out in Roon. I'm waiting on my preamp to arrive to help increase the input volume so I can hopefully go back to the low gain setting. I don't think you are imagining anything at all. I used the WA33 as a preamp for my old HSA-1b + SR1a combo and I could definitely hear a difference, especially between high and low gain settings.
Also, I don't know how your DAC controls the volume, but I recently learned you should never use a digital volume control. With each 6dB attenuation you lose one bit of resolution. DSP = very convenient, but bad for resolution.