Well, I've reminded myself why I listen to rock and not classical music in the office; I get too involved in classical music. I made the "mistake" of trying to compare low & high primarily using my absolute favorite piece of music (which is, of course hackneyed, etc.) The Choral movement of Beethoven's 9th. I used the HDTracks 24/96 version on my iMod, just in case there was a scintilla of extra clarity that even these lead-encased lobes on my head could pick up. I spent nearly a half hour with my eyes closed and a stupid grin on my face only stopping to flip the two switches and hit "back" on the iMod. Ahh, what beauty, what majesty, the drama, the pathos!
OK, back to the matter at hand. For that piece of music, the only difference I could tell was that on low (adjusting for volume) the soundstage seemed larger, if that makes sense. Switching to high brings the instruments "closer" to my head. I don't know what "harsh" means audiophilliclly, but perhaps the high tones were ever so slightly more shrill on high?
I next compared it using "Shake your Rump" (Beastie Boys Paul's Boutique ripped 24/96 from someone's original vinyl) and I cannot hear much of a difference (correcting for volume). I get the same punch in the bass, the same resolution. If I HAD to say there was a difference, and this may be SOLELY a factor of my imagination, it would be again, that perhaps on high, the soundstage is smaller. If on high, the stage is a circle 10 ft from my head, on low its 15 or 20. I don't even know if that makes sense, and I tried to isolate and hear if I could perceive any more separation of voices on low vs high, but I'm not sure I do, and I'm positive I'd fail an A/B/X test on that.
So, at least, there is no harm to the music, IMO, going to low, and it allows the use of IEMs without opening up the amp
Hope that helps.