So you're extrapolating from your own inability to hear a difference; thus you say "for a lot of recordings there's no way anyone can hear a difference".
That's hundreds of millions of people you're talking about - some, just some, might have more sensitive ears than you do.
Not at all, my friend. Chill. No reason to get your knickers in a twist
For example: if a recording is at any stage compressed to 320kbit mp3 - and then pressed on a CD - the FLAC from that CD will sound exactly as crappy(or good) as the MP3 - hence my point. Doesn't apply to most "professional" mixed music though but still, these things do exist.
Anyhow - we are all good. HiRes sounds better (under the premise it's not up sampled crap) and some people can very well hear and appreciate the difference, no doubt.