Hi,
I can't comment on the 500 but I do have the 560v4 which even though I love, have been relegated to their stand since getting the XS.
IMO the XS are definitely an upgrade from the 560 and for me the most noticeable differences between the two are the soundstage which I find to be quite special on the XS. It's wider, more open and spacious and gives me some absolute 'wow' moments in my listening sessions. The bass, which I find to be comparatively as punchy as the 560 but with extra depth and definitely more bassy. The mids/upper mids seem more detailed with vocals and some instruments sounding more pronounced which I felt were slighlty lacking sometimes on the 560.
All subjective of course
.
I listen to most genres of music but primarily EDM (Rufus du Sol, Ben Bohmer, Yotto etc..) which I find the 'Hifiman sound' is perfect for this genre and with the XS on my head I'm pretty much in my audio happy place.
My Hifiman journey so far has been HE35x -> 400SE -> 560v4 -> EdXS, with my next stop being the Arya (maybe) but for now I'm more than happy where I'm at with the XS and not in any hurry to get there.
Cheers