OK. With some major concentration, using Grado 325's, there does seem to be a very, very, marginal change which I can notice when switching from the wrong setting to the correct setting. It sounds like a tiny increase in amplitude....possibly a clarity improvement.
I don't notice the change at all when switching from the correct setting to the wrong setting.
This would make a perfect blind listening test project.
Sit with your back to the amp, headphones on and playing your favourite, best known music....and get someone else to move the switch. Be sure that you don't know whether the switch is up or down at the start of the session.
Have that person keep track of your observations after each switch back and forth.
Repeat the process several times.
And try to stay calm and relaxed so it doesn't ruin your judgement.
I'd do it, but i live alone.