Trying to hear the difference between 320 mp3 vs flac (16/24-bit, 44.1/88.2)
May 30, 2013 at 8:08 PM Thread Starter Post #1 of 75

matthewacbroad

New Head-Fier
Joined
May 30, 2013
Posts
18
Likes
11
I'm not sure how to hear the difference between mp3 / flac. If I plug my headphones directly to my PC onboard "Realtek HD" and enable 24-bit 192,000khz well I be able hear the difference?

playing FLAC 24-bit / 88.2khz Daft punk files. Than play 320k mp3 version of it, all I hear is a volume difference and little bit less highs
 
May 31, 2013 at 12:34 AM Post #2 of 75
You can't do it in you system. Easyest way is to ask some body with good gear - you my find mate in you area- or attend good shop, meeting, show or some thing like this.
Vlad
 
May 31, 2013 at 3:17 PM Post #3 of 75
You can do it on that system easily enough. Find some samples that cause problems with lossy encoders. There is a quite short but informative discussion of the kind of sounds that are a problem at http://www.hydrogenaudio.org/forums/index.php?showtopic=64724

You can also search for some of the famous killer samples such as trumpets, eig_essence and castanets, and very easily hear how lossy compression can go wrong even at high or maximum bitrates.

No special playback hardware is required, A Realtek HD integrated audio card and a budget headphone is fine and will be good enough even if the PC has a noisy headphone out. The differences can be very gross. Very revealing equipment may be useful in revealing less obvious differences.

The easiest way to make a fair comparison is to use an abx plug in or app. In windows try foobar with abx plug in. In Linux you can use http://abx-comparator.berlios.de/ Not sure about Mac but there is surely something useful.

Of course it helps to know what to listen for and how, and to be good at it (because it's not exactly a relaxing and enjoyable task) . You can get a free listener training package from Harman Kardon at http://harmanhowtolisten.blogspot.com
 
May 31, 2013 at 4:00 PM Post #4 of 75
Lossy audio isn't supposed to sound noticeably different at such high bitrates, even with the best gear dare I say. There might be some instances where you can hear a difference (see julian67's post), but generally, it's supposed to be transparent. People who say they can easily hear a difference are probably suffering from the placebo effect. Double blind tests are necessary. You said the lossy MP3 had attenuated highs, but are you sure you're not imagining it?
 
May 31, 2013 at 4:58 PM Post #5 of 75
Welcome to the world of supposition, probability and uncertainty, decorated with a dash of unqualified and uninvited psychological evaluation.

Warning: If you spend years trying to convince yourself your mp3s sound identical to the CD, without ever really being sure, then this can happen to you too.
 
May 31, 2013 at 6:04 PM Post #7 of 75
To assert that people are pretending is a grosser expression of bias than anybody mistakenly thinking they heard a difference that actually arose from level difference or expectation. Their bias might be real but it is an honest mistake and easily mitigated. Your bias is one that assumes malice and dishonesty in others. This is much more difficult to remedy.

http://www.filefactory.com/file/55ii4976hfzh/n/eig_essence_flac

There is a killer sample for you. If you convert that to mp3 and cannot abx it at 320 CBR or -V 0 your next step is seeing a doctor to get your hearing checked out.

That is offered as an easy example and for illustration. You can easily find other samples for yourself if interested.
 
May 31, 2013 at 6:42 PM Post #8 of 75
Quote:
http://www.filefactory.com/file/55ii4976hfzh/n/eig_essence_flac

There is a killer sample for you. If you convert that to mp3 and cannot abx it at 320 CBR or -V 0 your next step is seeing a doctor to get your hearing checked out.

 
I already tried  to ABX this sample with encoded version before at vbr v0, without success.
I find it dubious when someone claim it can abx  "easily", 
If someone else want to do the the test, you are welcome. Show me  the abx log, I'm not interested  by 5 cent philosophy.
 
Oh and if you download from this link , don't forget to uncheck "Download with FileFactory's download manager", my antivirus blocks it as suspicious.
 
May 31, 2013 at 7:38 PM Post #9 of 75
Here is the lame -V 0 mp3

http://www.filefactory.com/file/3x32qc5ii44x/n/eig_essence_mp3

and some file info

Code:
General
Complete name : eig_essence.mp3
Format : MPEG Audio
File size : 147 KiB
Duration : 4s 153ms
Overall bit rate mode : Variable
Overall bit rate : 289 Kbps
Writing library : LAME3.99r

Audio
Format : MPEG Audio
Format version : Version 1
Format profile : Layer 3
Mode : Joint stereo
Duration : 4s 153ms
Bit rate mode : Variable
Bit rate : 289 Kbps
Minimum bit rate : 32.0 Kbps
Channel(s) : 2 channels
Sampling rate : 44.1 KHz
Compression mode : Lossy
Stream size : 146 KiB (100%)
Writing library : LAME3.99r
Encoding settings : -m j -V 0 -q 0 -lowpass 22.1 --vbr-new -b 32

abx log, foobar2000 running under wine.

Code:
foo_abx 1.3.4 report
foobar2000 v1.2.3
2013/06/01 00:13:38

File A: I:\mp3\problem_samples\eig_essence.flac
File B: I:\mp3\problem_samples\eig_essence.mp3

00:13:38 : Test started.
00:16:19 : 01/01 50.0%
00:16:39 : 02/02 25.0%
00:17:04 : 03/03 12.5%
00:17:43 : 04/04 6.3%
00:18:57 : 05/05 3.1%
00:19:22 : 06/06 1.6%
00:19:36 : 07/07 0.8%
00:19:59 : 08/08 0.4%
00:20:16 : 09/09 0.2%
00:20:29 : 10/10 0.1%
00:20:59 : 11/11 0.0%
00:21:11 : 12/12 0.0%
00:21:19 : 13/13 0.0%
00:21:32 : 14/14 0.0%
00:22:31 : 15/15 0.0%
00:23:20 : 16/16 0.0%
00:23:24 : Test finished.

 ---------- 
Total: 16/16 (0.0%)

This is probably one of the most easily abx'ed samples you could ever find.

If you genuinely can't abx it then here is what to listen out for:

The impacts are smeared, that is they have a thicker, less well defined sound. This is most apparent on the final few hits. If you're not sure what to listen for then a really good method is to make a lower bitrate encode where the fault is just too big to ignore. With eig a 128 CBR will do the trick. Then go back to the -V 0 or a 320 CBR and you'll easily notice the artefact is still there.

If you genuinely can't hear the difference even after "learning" the artefact with a lower bitrate version of this I think it is time to check your playback hardware and maybe get your hearing checked. I'm not being facetious; it does suggest a problem.

edit: the Harman Kardon training pack I linked in an earlier post is very helpful. One of the techniques is to start with very obvious differences and step by step make them less obvious until you can't tell. It's a great way to make doing abx easier and more reliable but imo it is still about as enjoyable as cutting myself shaving.
 
May 31, 2013 at 9:44 PM Post #10 of 75
Quote:
 This is most apparent on the final few hits. If you're not sure what to listen for then a really good method is to make a lower bitrate encode where the fault is just too big to ignore. With eig a 128 CBR will do the trick. Then go back to the -V 0 or a 320 CBR and you'll easily notice the artefact is still there.

If you genuinely can't hear the difference even after "learning" the artefact with a lower bitrate version of this I think it is time to check your playback hardware and maybe get your hearing checked. I'm not being facetious; it does suggest a problem.

 
Well, I managed to ABX  lossless / vbr v0  after memorizing/learning  the artifact from the 128 kps version.
However it's not trivial , I need repeated listen before I make choices on ABX test, and I made few mistakes.
Took me twice your time, for 15 attempts (18 min vs 9 min):
 
Code:
 [color=#2f4f4f]foo_abx 1.3.4 report foobar2000 v1.2.4 2013/06/01 01:47:36 File A: C:\Users\Mehdi\Desktop\eig_essence.flac File B: C:\Users\Mehdi\Desktop\eig_essence vbr 0.mp3 01:47:36 : Test started. 01:48:13 : 01/01 50.0% 01:49:38 : 02/02 25.0% 01:50:29 : 03/03 12.5% 01:51:23 : 04/04 6.3% 01:52:37 : 05/05 3.1% 01:53:39 : 05/06 10.9% 01:54:53 : 06/07 6.3% 01:55:40 : 06/08 14.5% 01:56:27 : 07/09 9.0% 01:59:32 : 08/10 5.5% 02:00:30 : 08/11 11.3% 02:01:34 : 09/12 7.3% 02:02:37 : 10/13 4.6% 02:03:44 : 11/14 2.9% 02:05:32 : 11/15 5.9% 02:05:43 : Test finished. ---------- Total: 11/15 (5.9%)[/color]
 
 
 
From what I've understood , the "drums"  (for lack of better name) are echo-ed, and I must focus my attention on the echo only.
The "echo" sound less tight in vbr v0 version, but the "main non-echoed impact" is just too distracting for me.
Off course with 128kps bitrate, it's much easier to abx.
 
Anyways I don't have "bionic ears", and when I listen to music, I focus my attention on melody not isolated sounds.
I realized also that artist often use lossy samples in their compositions, and unfortunately some compilations such like buddha bar are using more and more lossy tracks.
Nevertheless, I'm a kind a lossless freak myself , but  if someone doesn't have everything in lossless, it shouldn't ruin his enjoyment.
 
May 31, 2013 at 11:05 PM Post #12 of 75
Well, I managed to ABX  lossless / vbr v0  after memorizing/learning  the artifact from the 128 kps version.
However it's not trivial , I need repeated listen before I make choices on ABX test, and I made few mistakes.
Took me twice your time, for 15 attempts (18 min vs 9 min):
...


From what I've understood , the "drums"  (for lack of better name) are echo-ed, and I must focus my attention on the echo only.
The "echo" sound less tight in vbr v0 version, but the "main non-echoed impact" is just too distracting for me.
Off course with 128kps bitrate, it's much easier to abx.

Anyways I don't have "bionic ears", and when I listen to music, I focus my attention on melody not isolated sounds.

I realized also that artist often use lossy samples in their compositions, and unfortunately some compilations such like buddha bar are using more and more lossy tracks.
Nevertheless, I'm a kind a lossless freak myself , but  if someone doesn't have everything in lossless, it shouldn't ruin his enjoyment.


Hi extrabigmehdi

Thats an informative and interesting test and description. Sounds like you enjoy abx about as much as I do :wink:

I didn't focus on the echoes, I focused on the actual impacts becoming smeared. Assuming we're choosing our descriptive words fairly well then I think this shows there are at least two problems with the lossy version.

If you do a little listening training you will hit 15/15 or 20/20 on a sample like that every time. I hadn't done an abx for several months, and find it really boring and quite annoying, and also I don't like that kind of music at all. If I paid attention I suspect I could do that test with little or no repeat listening and that 9 minutes would go down to next to nothing. I'd be surprised if you couldn't do the same.

I don't have bionic ears. I'm 45 and my hearing is, as far as I can tell, in the normal range. I had a quite involved hearing test for employment about 10 years ago (in a job where a hearing defect is not allowed) and they didn't congratulate me or open any champagne, they just said "Your hearing is normal." and I went through to the interview stage.

This eig sample is quite interesting. It is unusually easy to abx at very high bitrate, at least in mp3 and ogg vorbis (and I think opus but not sure) because lossy encoders tend to have terrible trouble with transients, resulting in smearing and pre-echo. You had some trouble at first and hadn't noticed a quite severe error that you now know to be present and audible. You and I noticed different errors. It would hardly be a surprise if there were some difference neither of us noticed.

This is a very simple piece and there are no sounds which might mask the errors. Most music is far more complex but does also contain plenty of transients (though not usually so well defined). I think you can see how likely it is that we listen to lots of lossy audio that does differ audibly from source but which we assume is "transparent" or "non-different" but is hard enough to abx that for many people an abx produces a null. Now you know what that artefact sounds like you can hear it every time in good old normal listening btw, try for yourself.

eig is also easy because the compression adds obvious noises. You can easily hear "the problems". Now consider that other compression artefacts don't add noise or distortion, but do more subtle stuff like reduce stereo separation or shorten the decay of notes or very slightly change relative apparent loudness of different tones differently ("bass is less weighty" is something people say) and so on. How do you rate your chances of picking up on that in normal listening while believing your mp3 is as good as the original? If you know the music in lossless form you will probably occasionally notice things or more likely respond to things without quite being able to pin it down. It's easy to put it down to bias, especially if abx gives you nothing but a headache and stronge urge to do something else.

Imagine if that piece was something you really knew, that you had listened to the CD tens or hundreds of times. Then you listened to the "transparent" -V 0 mp3. Chances are that you would then notice it sounded like crap. But what if the only version you ever knew was the mp3? You might never realise how crappy it was.

btw a modern codec like aac does a much better job than lame or vorbis with a sample like eig. Last year I also tried the Fraunhofer mp3 encoder with eig and that stank as well. If you need lossy compression and your hardware supports it then aac is far less likely to fail quite so badly. Just don't expect perfection.

Uncomfortable parting thought:

Think about how being accustomed to compression affects your ability in the long term to even be able to tell what is good quality and what is bad, what is music, what is unwanted distortion, and what is missing.
 
May 31, 2013 at 11:56 PM Post #13 of 75
Quote:
I'm not sure how to hear the difference between mp3 / flac. If I plug my headphones directly to my PC onboard "Realtek HD" and enable 24-bit 192,000khz well I be able hear the difference?

playing FLAC 24-bit / 88.2khz Daft punk files. Than play 320k mp3 version of it, all I hear is a volume difference and little bit less highs

 
@ OP - Julian left you some really good comments.
 
For you to test yourself - if you follow this link (http://www.head-fi.org/t/655879/setting-up-an-abx-test-simple-guide-to-ripping-tagging-transcoding), it'll walk you through setting up a very basic test.  The software is all free - all it will cost you is time.
 
The most important thing to realise when people make claims about what is audible and what is not (with different formats) - is that everyone's hearing is unique.  So the best thing you can do is find your threshold - then you can confidently choose the best codec/container that works for YOU.
 
Personally - with my older ears - I know mine is aac ~ 200 kbps (vbr).  But that's come from a lot of testing.
 
So far I have found very few people who can successfully and consistently abx aac256 from lossless in a completely blind level matched comparison - using same original source.  There may be some who can - but if you can't - don't worry about it - it just means your hearing is more comparable with the vast majority of us.
 
Jun 1, 2013 at 5:33 AM Post #14 of 75
So far I have found very few people who can successfully and consistently abx aac256 from lossless in a completely blind level matched comparison - using same original source.  There may be some who can - but if you can't - don't worry about it - it just means your hearing is more comparable with the vast majority of us.


There are other possible explanations.

Do you know/remember that challenge young children make to their friends? "Pat your head and at the same time rub your stomach". A few seconds later there are kids with rotating, swiveling arms and hands comically failing to do two completely simple things at the same time, shortly followed by some intensive practice so as not to look silly again and, much more importantly, so as to be able to fearlessly issue the same challenge to others. The point is that making a three way comparison is not a simple task. It is simple and easy to describe but that is not the same thing as simple and easy to perform. It is something you will probably never do outside the artificial construct of this test. You have no experience of it, no expertise, probably not even any prior life experience that offers a complete set of directly transferable skills. It feels difficult and unnatural because it is unnatural and difficult. It is so difficult that it requires coaching/training and practice to acquire any measurable ability or reliablity. Probably normal listening is anything but simple if one examines the way the ears react, the brain/mind responds and manages the ears and so on. But it feels simple in that it requires some attention but is something we do easily and contentedly. But making a three way comparison does not feel simple, does require complete attention and involves mental and possibly physical exertion. Hearing and listening are not a simple mechanical processes depending only on the physical response of the ear. If you give the brain two tasks which are strikingly different it is unreasonable to insist that they are the same, or to insist that the reactions and perceptions that arise must be equivalent and directly comparable, or even that they might be. Training can mitigate this to some extent but can't make the different experiences the same.

You can certainly say that any positive abx result is proof of a perceptible difference. What you absolutely cannot do is claim that a null is a proof of transparency. This is particularly true of untrained listeners performing personal listening tests. If they have been led to believe that nulls have meaning then they are just as much victims of bs and bias as people who have been persuaded that pebbles and green marker pens change the sound.

If you have trained listeners of sufficient and verified ability and reliablity then you can start to have a real interest in the nulls, but still can't guarantee that your testing process fully encompasses the normal every day human listening process. Your big benefit comes in identifying differences more quickly through enhanced ability and better description. There are no guarantees that you found all the differences people will legitimately notice in other circumstances.

I would suggest that if you don't detect a difference in abx testing but you think you do detect differences in normal listening then this cannot be reliably taken as an indication of bias and may well be exactly what it seems - a difference.
 
Jun 1, 2013 at 6:39 AM Post #15 of 75
That's where you an I differ Julian - but that's OK.
 
If I can't hear the difference (after many, many tests) then what I can't hear, I don't worry about.  it actually makes life easier for me.
 
If anyone still thinks they can  reliably and consistently tell the difference  despite consistently failing blind abx - then they are welcome to still claim  a difference.  It is their ears.  I personally will remain sceptical until I see proof - but that is my right to do so.   All I encourage others to do is simply do the tests.  It costs nothing but time.
 
What they make of their own data after running the tests is up to them.  All I can apply (and only apply to me) is knowledge of my own hearing threshold.  And I know for me - that properly encoded aac ~ 200 kbps is to all intents and purposes transparent to me.
 
That's all that matters.
 

Users who are viewing this thread

Back
Top