It doesn't have to do with friends. There are lots of stories on this thread that have nothing to do with that.
It doesn't imply that one needs to have above average hearing in order to appreciate the difference. It has to do with experience and knowing how all the components of music come together, both technically and subjectively, which trained listeners have and inexperienced people don't. This is my entire point to begin with; many people can't tell the difference between low-fi and high-fi not because of any physical differences, but simply because they have no idea what to be listening for or how to listen to music technically because they're usually not as invested or interested in music. Listening to music as an average Joe and listening to music as an audiophile are two completely different processes and experiences.