I just completed my purely subjective, opinionated, it's-all-about-me test of my 4s and the new 5. I was ready to do the "early upgrade" to get the 5 *IF* it sounded as good (or better) than my 4s. The primary use of my iPhone is music (moreso than phone, IM, browsers, etc.), so good sound is imperative.
I took my mid-grade (don't want to get into a brand/quailty argument) IEMs to the Apple store. I went to the iPhone 5 area, got one, and turned off all the sound and EQ settings (sound check, eq, etc.). I then looked through the songs on the 5 and found a song I also have on my 4s (don't want to get into another style/quality argument here). The 4s has iOS 6 btw.
I then played that song for a while on the 5, then re-plugged immediately to my 4s playing the same song. I did this several times. I then did it again with a different 5 next to the one I had been using (just in case).
Subjective opinion: Just as some others have posted in this thread, the 4s sounded "warmer" and had a fuller sound stage to my ears. I also noticed the 5 had that clear (some call it "digital"), but more 2 dimensional cold sound; not necessarily a negative thing, just different. The 4s was also louder at identical volume settings.
Result: I will not be upgrading to the 5. This is purely subjective and YMMV, of course. To my ears, there's a significant difference and the 4s is significantly better for me.