Unscientific Testing of MP3s and Flac
Jul 17, 2011 at 10:56 PM Thread Starter Post #1 of 8

prescient

Head-Fier
Joined
Oct 31, 2006
Posts
73
Likes
41
Got bored and decided to conduct some very unscientific tests.  Test 1 was to see if my hearing is shot so I listened to some test tones to determine whether I could hear high frequency sounds.  I passed that at 17K.  That's not really great but then again I'm not a teenager these days so I'm satisfied.  
 
Next up lets ABX some music!  So I wrote a quick little program to ABX some WAV files.  The plan was to convert a flac of a high quality audio recording to a WAV, and also convert a flac of a high quality audio recording, Chesky Records' Isn't She Lovely, to 128kbps mp3 and then to a WAV.  So that's done, but I don't really know the song that well so I figure I'll give it a listen on both the MP3 version and Flac version and see if I can pick out any differences.  Well hell I'm not sure if the details that I'm picking out are real or illusory.  So I pull up the recordings in Audacity and start looking at the spectrum and see the drop off as expected in the MP3 within my hearing range so I should hear something, and I think I do but not enough to bother wasting 30mins of my life ABXing em.
 
Long story short is that I've come to the conclusion that the quality of the original recording is much more important than the actual format.  This isn't ground breaking or new but it was fairly interesting to satisfy my own curiosity. I also inverted one of the tracks to theoretically subtract them from each other (not an audio engineer so I'm not sure this is valid) and it was pretty interesting to see what the difference is that I can't seem to reliably make out.
 
Who knows maybe the gear I'm using isn't up to snuff (E-MU 0404 USB & HD580 or SRH 840).  I know I listened to a really high end speaker system once and concluded that I had never heard a good stereo before.  I would actually like to try this test with a DAC1 and HD800s.
 
Jul 17, 2011 at 11:04 PM Post #2 of 8
At 128kbps, MP3 is already quite good with very few artifacts. At 320kbps they're about perfect. Save for the few and rare killer tracks, one cannot differentiate 320kbps and true lossless with the same setup.
Maybe unless you're operating a Sennheiser HE90.
 
Jul 17, 2011 at 11:17 PM Post #3 of 8
I never considered doing ABX testing, but I probably should. I would fail though, since my equipment is so terrible.
 
I think many people use FLAC/ALAC simply because it is easier to manage. The computer has all of the lossless files, and then today's media players have enough storage for lossless music, so there is pretty much no reason NOT to use lossless on both ends, as it saves the time of converting to OGG or MP3. Except maybe if someone has a very large collection.
 
Jul 17, 2011 at 11:21 PM Post #4 of 8
Yeah, I guess.
Though I have two libraries, one FLAC (v5 I think) and one 320kbps MP3. The one reason would probably be "storage space" although I am having a hard time filling up an 8GB card with my MP3 files, and I will be getting a 40GB player tomorry... Hmm... Easier to maintain... I see now... Thanks for mentioning it!
 
Jul 17, 2011 at 11:32 PM Post #5 of 8


Quote:
I never considered doing ABX testing, but I probably should. I would fail though, since my equipment is so terrible.
 
I think many people use FLAC/ALAC simply because it is easier to manage. The computer has all of the lossless files, and then today's media players have enough storage for lossless music, so there is pretty much no reason NOT to use lossless on both ends, as it saves the time of converting to OGG or MP3. Except maybe if someone has a very large collection.


I completely agree with this, but I was curious about what I was hearing in some tracks.  I had listened to some MP3 tracks and previously was convinced that they sounded terrible due to the format.  I'm pretty satisfied that I was completely wrong about that.
 
 
Jul 17, 2011 at 11:42 PM Post #6 of 8
Yeah the differences are mighty small, I'm sure you'll be able to nail the ABX no problemo if you had better gear.
 
I once thought there was a big difference with 128k and lossless, I did some blind testing and now I don't even care anymore.
 
I still rip CD's in FLAC though, partly because I'm only going to do it once, but mainly because I want all the bits that I paid for =P.
 
Jul 20, 2011 at 2:35 AM Post #7 of 8
the only way ive been able to tell was listing the tight cymbals and how they decay.  i was actually able to hear this on XB500 which are NOT made for quality i'll be getting my k240 studio's in a few weeks so i'll try again.  It was actually 320 vs 128 so probably more of a difference but same concept should apply.
 
Jul 20, 2011 at 4:01 AM Post #8 of 8
My advice - always rip to lossless (either FLAC or ALAC) depending on what your home set-up is.  Use the lossless for PC/MAC where space isn't an issue - and make sure you have a back-up in the same format.
 
For your portable (assuming 32Gb or under), transcode to either V0 MP3, 320 MP3 or 256 AAC (again depending on format).
 
Reason:
 - lossless for archiving, because if you ever change to a player that needs a different format, you can simply transcode again from the lossless file without having to rerip your media (eg changing from iPod to Cowon).
 - lossless for PC because today space shouldn't be an issue - so you may as well listen to the best quality you have
 - lossy, but hq lossy for portable - because most of us wouldn't be able to tell the difference between lossless and 320mp3 / 256 alac in a blind test - and it gives you the extra space on a limited portable device.
 
On my 32gb Touch G4, I have everything in 256 aac - which gives me 254 full albums, or 3256 songs - way more than I need,
 
Of course if you have a high capacity player - go lossless on both and save the problem of transcoding altogether.
 
For those who insist they can tell the difference - try the following:
 
Rip a CD (one you know well) to FLAC/ALAC
Convert same CD to 256aac or 320 mp3
 
Pick a song, load both versions into foobar using the abx plugin, normalise them (equalise the volume).  Now run the blind test and see how you go.  I've tried this several times, and the software confirms that I can't tell the difference.  There are bound to be a few that can tell the difference - but I'm willing to bet 99% of us can't.  Until you do the test though, you're always questioning yourself.  Once you do the test at least you know.  In a way it's actually liberating.
 

Users who are viewing this thread

Back
Top