When I first began listening to lossless files and changing my collection to lossless from itunes AAC, I did not hear too much of a difference honestly. At that time I used good mid/high level headphones like beyerdynamic dt990 32 ohm and denon d5000 out of my computer using foobar. The only initial changes that I noticed after a few minutes of ABX was flac files semmed to have more sound/noise at all times, compared to AAc where I was used to the missing samples, and thus seemingly emptier sound of songs. The other main thing was flac files seemed to have crispier treble, and in general a tad higher treble (although I think the higher treble sound is due to the playback source of my default computer dac/amp, the no-name internal ones). That was all at first.
eventually after a few months I got used to flac files, and began hearing slight extra details in all that extra noise that was there initially. Then when i played AAC again, it felt noticeably emptier sounding, like something, very little something, was just missing. Otherwise the only thing that can go wrong with flac is how you actually get flac songs. like if you rip from a cd, and use a good ripper, then you should be ok for consistant sound quality. But if you are like me and occasionally torrent some stuff, then the flac files can vary in actual sound quality, because they reveal anything that exists on the files. so I have some albums that sound perfect, and others that are too sibilant, or others that are too quiet. small variants but, the sibilant ones can be annoying. Compared to torrenting an mp3 or AAC file, witht he missing samples, it kinda fixes some of those minor varients. If you buy all your music and rip from a cd, then non of that should matter lol. bandcamp has great flac files, I love it when bands post their music there. or use hdtracks (but thats way pricier compared to bandcamp which can be free at times or very cheap).
So I guess after that essay of text, the overall differences are in the sound signature + fullness of the song, and less of the quality of the sound itself. as in the quality from 320 to lossless will be exactly the same in terms of clarity and sharpness and distortion free, just with practise you will definitely start hearing where the compression removed certain samples of frequencies, and you start only using flac, unless you dont get bothered by it enough. I have mostly flac, with some 320. And all my flac, besides like 3 albums, is all in 16 bit 44.1, so just go with flac if you can handle the file sizes, or stick with 320 to save space.
hope this helps on a less technical level :)