BLIND TEST: 128kbps mp3 vs Lossless

Feb 4, 2012 at 11:31 AM Thread Starter Post #1 of 180

Crazy*Carl

500+ Head-Fier
Joined
Aug 8, 2009
Posts
852
Likes
28
I see there is some threads on mp3 vs flac.  This will clear stuff up very easy and objectively, by allowing you guys to actually know for yourself.  This is done via a blind test, without having placebo affects get in the way. (Yes I am one of those guys, please don't turn this thread into something bad.  If you think blind tests are bad, DO NOT POST)
 
I took the original lossless flac file and decoded to to a wav.  I made a copy, then encoded that wav into a 128kbps mp3 file, then back to wav, so the files are both the same size to mask the type.  I will eventually reveal the file types.
 
Song: The Goo Goo Dolls - Black Balloon (50 second clip).
 
A:   http://www.mediafire.com/?gpb35926lk9ac9j

B:   http://www.mediafire.com/?jcgnknv1wrpy7c4
 
Feb 4, 2012 at 11:56 AM Post #2 of 180
I'm really excited to see the results of this!  I'm one of those who can never tell a 320k mp3 from a FLAC consistently, and I'm definitely one of those who both loves and trusts DBTs
 
 
I'd guess the original file is B, and the mp3 is A.  
 
Feb 4, 2012 at 1:31 PM Post #3 of 180
interesting, I guess I will take the test on my wake up with a not-so-revealing system. Guys, for those with multiple equipments, post your gear used it shall be a interesting test. 
 
For me it would be 
 
Creative SB X-fi HD -> Edifier R2000T active monitors(FR ~50HZ- ~21KHZ)
 
Status: done, used replaygain, it was a pretty difficult test with my equipment, track with 128kbps was a pretty subtle difference. Did the test fully blind with my brother switching tracks on jriver media center and a randomnizer.
 
Feb 4, 2012 at 1:47 PM Post #4 of 180
I did the test before El_Doug posted so no placebo effect for me. It did take me a few tries of A/Bing but ultimately decided that B is lossless and A is 128kbps.
 
Macbook Pro -> Amarra 2.3 -> StageDac -> Classic -> Beyer T1
 
Feb 4, 2012 at 2:22 PM Post #5 of 180
It's unfortunate that people posted their results. It would have been a great test, otherwise.
 
Feb 4, 2012 at 3:10 PM Post #7 of 180
If people want to cheat, they can easily do it anyway with an audio editor (the mp3 will obviously lack high frequency content on an FFT display). And even knowing which file is which, an ABX test is still meaningful to determine if there are audible differences.
 
 
Feb 4, 2012 at 3:15 PM Post #8 of 180
Feb 4, 2012 at 3:22 PM Post #9 of 180
I've been breaking in my new DACmini w/the LCD-2. This test falls right into what I was doing in updating my music library on my macbook. For some albums where I had duplicates, one being mp3 and the other the updated flac/wav, I couldn't tell a difference. And others where I could tell a difference, the difference was more subtle than dramatic.
 
I didn't read the rest of the post until after I tried the test, and I think B is lossless b/c it sounded fuller at around the 10 second mark where the song first hits hard and had a slight edge on impact.
 
Feb 4, 2012 at 4:09 PM Post #10 of 180
If people want to cheat, they must REALLY be bored
 
Quote:
If people want to cheat, they can easily do it anyway with an audio editor (the mp3 will obviously lack high frequency content on an FFT display). And even knowing which file is which, an ABX test is still meaningful to determine if there are audible differences.
 



 
 
Feb 4, 2012 at 6:18 PM Post #11 of 180
SUP GUYS
 
Files were off by like 0.4db or so, too easy. You can get big volume differences when encoding to these low bitrates, but when it's encoded to something high, it only changes by 0.02db or so, if any. Keep that in mind when conjuring up more tests.
 
foo_abx 1.3.4 report
foobar2000 v1.1.8
2012/02/05 09:34:29

File A: C:\Users\Alice\Desktop\A.wav
File B: C:\Users\Alice\Desktop\B.wav

09:34:29 : Test started.
09:34:59 : 01/01  50.0%
09:35:08 : 02/02  25.0%
09:35:30 : 03/03  12.5%
09:35:51 : 04/04  6.3%
09:36:10 : 05/05  3.1%
09:36:22 : 06/06  1.6%
09:36:31 : 07/07  0.8%
09:36:38 : 08/08  0.4%
09:36:51 : 09/09  0.2%
09:37:00 : 10/10  0.1%
09:37:02 : Test finished.

 ----------
Total: 10/10 (0.1%)
 
So I hit em up with that replaygain(after converting to flac to support them tags) to make it more challenging, was somewhat difficult because I don't like listening to boys, so I struggled until I got the bright idea to only listen to the last 15 seconds of the song with no vocals.
 
foo_abx 1.3.4 report
foobar2000 v1.1.8
2012/02/05 09:41:44

File A: C:\Users\Alice\Desktop\A.flac
File B: C:\Users\Alice\Desktop\B.flac

09:41:44 : Test started.
09:42:18 : 01/01  50.0%
09:42:41 : 01/02  75.0%
09:43:55 : 02/03  50.0%
09:44:07 : 02/04  68.8%
09:44:36 : 03/05  50.0%
09:44:50 : 04/06  34.4%
09:45:10 : 05/07  22.7%
09:45:20 : 06/08  14.5%
09:45:38 : 07/09  9.0%
09:46:06 : 07/10  17.2%
09:46:18 : 08/11  11.3%
09:46:27 : 08/12  19.4%
09:48:00 : 09/13  13.3%
09:48:32 : 10/14  9.0%
09:48:43 : 11/15  5.9%
09:49:17 : 11/16  10.5%
09:49:26 : 11/17  16.6%
09:50:03 : 12/18  11.9%
09:50:25 : 13/19  8.4%
09:51:22 : 14/20  5.8%
09:51:38 : 15/21  3.9%
09:51:55 : 16/22  2.6%
09:52:05 : 17/23  1.7%
09:52:17 : 18/24  1.1%
09:52:24 : 19/25  0.7%
09:52:33 : 20/26  0.5%
09:52:48 : 21/27  0.3%
09:53:01 : 22/28  0.2%
09:53:09 : 23/29  0.1%
09:53:16 : 24/30  0.1%
09:53:24 : Test finished.

 ----------
Total: 24/30 (0.1%)
 
 
Feb 4, 2012 at 10:07 PM Post #13 of 180


Quote:
 
So I hit em up with that replaygain(after converting to flac to support them tags) to make it more challenging, was somewhat difficult because I don't like listening to boys, so I struggled until I got the bright idea to only listen to the last 15 seconds of the song with no vocals.



Exactly, I hear almost no audible difference with the vocals only the instruments, then again I don't really listen to male vocals either.
 
Feb 4, 2012 at 10:20 PM Post #14 of 180
There was an overall sense of "naturalness" to the second track, I found.  I can't put my finger on it, but track A seemed crisper, but almost in an artificial way.  That, and the cymbals, were the only places I detected a difference.  The vocals sounded identical to me, no matter how hard I tried to find a difference
 
Quote:
Exactly, I hear almost no audible difference with the vocals only the instruments, then again I don't really listen to male vocals either.



 
 
Feb 4, 2012 at 10:39 PM Post #15 of 180
It might be useful to mention that people shouldn't scroll down before doing the blind test. I almost scrolled down just to see what comments people had made about the idea of this test, but realized people might have listed their answers. Either that, or we need a spoiler function to block out the answers until scrolled over.
 
I kind of had the same feel as others. I tried to tell the difference by listening to the vocals, but couldn't. I'm surprised at how difficult it is to explain the difference, but the instrumental sections just sounded fuller on B. I favored B from the first listen, but couldn't figure out why. The following few listens I felt like the cymbals and drums in particular just had more range. After a few times listening, it just slowly started to sound the same or something sounded randomly weird, so I think I just listened too many times and started imagining differences. The vocals always sounded the same to me.
 
It would be neat if there was an automated process for creating test files like this using your own music without being aware of which file is which. Of course, you would need an answer key of some sort so you could find the solution once you've done the blind test. I feel like it took an effort to know the song and identify differences. I think it would have been easier to tell the difference if it was a song I was really familiar with.
 

Users who are viewing this thread

Back
Top