A lot of users own HiFiMan products for a long time before they realize that the cups can swivel. Not sure you are having this issue, but if you swivel the cups, you won't have issues behind your ear. My HE-4's don't swivel very well, they are really tight, but after messing around a bit, I finally got the right amount of swivel on both sides and I don't have to mess with it anymore.
Yea it's more due to the curvature of my head. The cups do swivel but the cups still need a slight curve to them. I don't think the gap makes a big difference in my case.