I can only tell a difference if I use my SR-009 with the the dCS Scarlatti as my source that is running through my Weiss DAC 202 with everything connected via Nordost Supreme Reference cables. When I use my SRS-4170 or Ultrasone Edition 10 headphones and the mbl system of the 1621 cd transport and the 1611 HR DAC with the Transparent Musiclink Super cables, the music just doesn't seem to have as much bite in 320 as it does in FLAC. There is certainly just a touch of gray in the blackness. Although the mbl is a fine piece of equipment, it just doesn't quite punch out of it's weight class.
Oh wait, there is probably a placebo difference but I doubt anyone in the homo sapien species can hear a difference. Nevermind.
That pretty much sums it. There are definite measurable differences but whether you will hear them with your gear/music depends on innumerable factors.
A recording professional with even a decent soundcard/monitors will be able to hear the difference - their livelihood depends on hearing much smaller differences than 320 mp3 vs 16/44.1 FLAC.
"SR-009 with the the dCS Scarlatti as my source that is running through my Weiss DAC 202 with everything connected via Nordost Supreme Reference cables" is similarly a VERY high resolution setup that should expose the differences easily.
On the average portable setup the differences are probably insignificant due to the hardware and background noise and outweighed by practical considerations.
IMO if you can get a recording in FLAC or WAVE do so; if you can get 24/96 even better - this is your master copy and you want as good quality even if it is mostly redundant. Hard drives are cheap - there is no need to split hairs over whether you can pass a DBT between formats. Use the best quality format that is practical.