Listening to some more songs on both the 2SE and 4SR, the 4SR does sound finer (resolution) to my ears. The vocals are more present and pops out a little more. If both were the same price I would pick the 4SR every single time. Frequency response wise they are both pretty much the same so I guess this, for me, shows there are things other than FR that aren't measured or measurable.
Will most people notice? I think for most people, audio is an afterthought and not a main focus so for the vast majority, the 4SR would not be worth the extra cost.