When you are operating any amplifier at higher gain, you also raise the noise floor closer to audible. How much of a difference depends on the amplifiers design, often it isn't really an issue. Saying that, if you are getting adequate volume from the lower gain setting, typically keeping the gain as low as needed to be effective is the way to go. I am not sure why beyond the increase in noise floor (which again may be very modest) the sound of the amplifier section would change at all. Ultimately the power flows through the same design circuitry so it is also possible that this is placebo effect in action.
Do you still play music at the same volume as you did before, as far as you can tell? The other thing to consider is that our brain is hard-wired to perceive louder as more engaging and dynamic, unless it is too loud of course. So often when people report things like this, even a small increase in playback volume will stimulate your hearing brain more and it is generally perceived as beneficial when the only thing that has changed is the amplitude of the signal your brain is working with. I guess it could be some design factor that causes the high gain to have more dynamics, but seems like an odd thing to do, but who knows, it might have been unintentional?