Lossless vs mp3 ABX results. (Among other ABX's)
Oct 1, 2011 at 9:00 AM Post #16 of 119
Thanks again. If you still have patience for my hare-brained ideas, I would be supremely appreciative (as opposed to merely very grateful) if you could experiment to how large a role background noise plays in your ability to distinguish things - just set something automated to be noisy a few rooms away (washing machine, hairdryer or something) and see if a bit of background noise makes life a lot harder. 
 
 
Oct 1, 2011 at 1:11 PM Post #17 of 119
Quote:
Thanks again. If you still have patience for my hare-brained ideas, I would be supremely appreciative (as opposed to merely very grateful) if you could experiment to how large a role background noise plays in your ability to distinguish things - just set something automated to be noisy a few rooms away (washing machine, hairdryer or something) and see if a bit of background noise makes life a lot harder. 


Yeah I think I'll leave an ipad or something playing white noise/rain or something close by and see how that affects the results.
 
I'm just trying to do a shootout with the O2's and the SR-202 at the moment, and I'm having an extraordinarily hard time finding a track whose mp3 is transparent to the SR-202.
 
This is the closest thing I have to a result: SR-202 for the first set, followed by the O2.
It's also the most difficult track I've come across so far, there is just one background instrument(some kind of synthesised xylophone) and 2 female vocals. No lyrics either, it's just gentle chanting(?)/going "lalala", so there isn't much to pick up in terms of dynamics that I normally get from the sibilant sounds. Lots of sustained notes due to the duet mask what little dynamics there are in this track. It's not on youtube so I can't link it =/.
 
foo_abx 1.3.4 report
foobar2000 v1.1.8
2011/10/02 03:08:15

File A: F:\Music\Binaria various\01 epoca.flac
File B: C:\Users\Alice\Desktop\01 epoca.mp3

03:08:15 : Test started.
03:09:11 : 01/01  50.0%
03:09:24 : 02/02  25.0%
03:10:17 : 02/03  50.0%
03:10:35 : 03/04  31.3%
03:10:53 : 04/05  18.8%
03:11:07 : 05/06  10.9%
03:11:18 : 05/07  22.7%
03:11:31 : 06/08  14.5%
03:11:46 : 07/09  9.0%
03:11:56 : 07/10  17.2%
03:12:09 : 07/11  27.4%
03:13:01 : 08/12  19.4%
03:13:10 : 09/13  13.3%
03:13:18 : 10/14  9.0%
03:13:33 : 11/15  5.9%
03:14:43 : 11/16  10.5%
03:14:51 : 11/17  16.6%
03:15:11 : 12/18  11.9%
03:15:34 : 13/19  8.4%
03:16:11 : 13/20  13.2%
03:16:15 : Trial reset.
03:17:35 : 01/01  50.0%
03:17:44 : 02/02  25.0%
03:17:59 : 02/03  50.0%
03:18:23 : 03/04  31.3%
03:18:36 : 03/05  50.0%
03:18:50 : 04/06  34.4%
03:19:01 : 04/07  50.0%
03:19:37 : 04/08  63.7%
03:20:07 : 04/09  74.6%
03:20:34 : 05/10  62.3%
03:20:47 : 06/11  50.0%
03:20:56 : 07/12  38.7%
03:21:10 : 08/13  29.1%
03:21:27 : 09/14  21.2%
03:21:41 : 10/15  15.1%
03:21:55 : 11/16  10.5%
03:22:07 : 11/17  16.6%
03:22:27 : 12/18  11.9%
03:22:39 : 13/19  8.4%
03:22:59 : 14/20  5.8%
03:23:11 : Test finished.

 ----------
Total: 27/40 (1.9%)
 
Given some more trials, I *think* I would be able to ABX that track successfully with the O2's. I called it at 20 per set otherwise I would learn the differences and the result may be invalid for the other headphone coming up. The little experience I gain ABXing with that track is kinda balanced from the fatigue I sustain as the test goes on. Such a simple testing idea does seem quite complicated to carry out doesn't it?
 
These two headphones have fairly different signatures, it's interesting because the Lambda is more treble happy, and I assumed the extra treble might help me out, but I don't think that was the case.
 
The smaller and more pinpoint soundstage of the O2 kind of helped me detect any differences in dynamics, as the differences are more....focused in one point inside my headstage.
 
It's interesting to see that I was failing all over the place with the Lambda, but the bulk of my failures were at the start for the O2. This might just be pure gibberish, but I think the fails start of the test with the O2's may be me adjusting to the different sound signature, as the second half of the test was more straightforward and correct.
 
Fascinating, isn't it? I'm going to have to do a lot more testing.
 
@Brooko
 
Same track as above, it was easy listening genre(I think). AAC vbr ~218kbps(but I set it to 255kbps @ encode) vs 320cbr mp3.
 
foo_abx 1.3.4 report
foobar2000 v1.1.8
2011/10/02 03:28:04

File A: C:\Users\Alice\Desktop\01 epoca.mp3
File B: C:\Users\Alice\Desktop\01 epoca.m4a

03:28:04 : Test started.
03:29:11 : 01/01  50.0%
03:29:21 : 02/02  25.0%
03:29:40 : 02/03  50.0%
03:29:52 : 03/04  31.3%
03:30:00 : 04/05  18.8%
03:30:12 : 05/06  10.9%
03:30:24 : 06/07  6.3%
03:30:43 : 07/08  3.5%
03:31:14 : 07/09  9.0%
03:31:24 : 08/10  5.5%
03:31:34 : 08/11  11.3%
03:31:51 : 09/12  7.3%
03:32:05 : 09/13  13.3%
03:32:50 : 09/14  21.2%
03:33:04 : 10/15  15.1%
03:33:19 : 11/16  10.5%
03:33:40 : 12/17  7.2%
03:33:54 : 13/18  4.8%
03:34:16 : 13/19  8.4%
03:34:28 : 14/20  5.8%
03:34:38 : 15/21  3.9%
03:35:00 : 16/22  2.6%
03:35:25 : 16/23  4.7%
03:35:41 : 17/24  3.2%
03:35:49 : 17/25  5.4%
03:36:51 : 17/26  8.4%
03:37:06 : 17/27  12.4%
03:37:25 : 17/28  17.2%
03:37:47 : 17/29  22.9%
03:38:00 : 18/30  18.1%
03:38:02 : Test finished.

 ----------
Total: 18/30 (18.1%)
 
Aaaaand, that's a fail =P. So for that particular track, the aac ~218kbps is comparable to the 320kbps mp3. I'll have a go with a more complicated track soon, but it looks like the aac is better bang for buck. Not surprising though as its a vbr compared to the cbr.
 
Oct 1, 2011 at 2:50 PM Post #18 of 119
Wow this is hard. I just tried it with one of your tracks Kimi no Shiranai Monogatari, I cannot distinguish between FLAC and 320 mp3. I would think HD800 is good enough. Which part are you listening to? Can you give me some minute:second?
 
Oct 1, 2011 at 3:11 PM Post #19 of 119
I tried it at 1:00 and onwards, there are no compression artifacts to listen for, I had to lock onto the vocals.
 
If you know that singer well, then you might have come across my name before.
tongue.gif

 
Oct 1, 2011 at 8:20 PM Post #20 of 119
I give up, really cannot tell the difference. I wonder if it's because my DAC automatically resample everything to 24/96. The resampling might have blurred the difference that it's really difficult to tell them apart if there are no artifacts.
 
Oct 1, 2011 at 8:29 PM Post #21 of 119

 
Quote:
@Brooko
 
Same track as above, it was easy listening genre(I think). AAC vbr ~218kbps(but I set it to 255kbps @ encode) vs 320cbr mp3.
 
(Snip)
 
Aaaaand, that's a fail =P. So for that particular track, the aac ~218kbps is comparable to the 320kbps mp3. I'll have a go with a more complicated track soon, but it looks like the aac is better bang for buck. Not surprising though as its a vbr compared to the cbr.



Thanks!  Confirmed what I'd heard and read.  Gives me peace of mind and more room on my DAP.  Win!
 
Oct 2, 2011 at 8:04 AM Post #22 of 119
Ha I got it! I got out some of my old mp3s, and my god those are TERRIBLE. I re-converted them again with LAME 3.98 from my lossless backup, and the new mp3 sounds sooo much better, even though they have the same bit rate :eek: How did I live back then lol! I don't remember when I got those mp3s, but they were definitely more than 6 years old, some might even be 10+ years old. Shows you how much LAME encoder improved over the years.

I feel pretty good using mp3s now TBH. With the new mp3s, I can only hear small differences on selected tracks (ie well recorded & mastered tracks), with volume 80db+, on my expensive equipment. My normal listening volume is 60-70db, sometimes even lower. I think it's safe to say mp3s are good for 99% of the time for me. On most tracks, if I'm listening casually, I won't even notice the difference. It's only on well recorded tracks, you feel that the mp3s lack "presence".
 
Oct 2, 2011 at 8:49 AM Post #23 of 119
It's only on well recorded tracks, you feel that the mp3s lack "presence".


Ahh, presence, that's the word I was looking for. ^^

Yeah I feel like I'm not missing anything at all when using mp3's, I just wanted to see if it was possible to distinguish them. For all intents and purposes, mp3's are more than transparent enough for everyday listening if there aren't any insane compression artifacts.

It's more of a feeling really, rather than hearing. No wonder it's so bloody difficult to ABX them. When I mentioned 'easy' it was in relative terms, I wouldn't want to bet my left nut on being successful every time I try, even on the easiest tracks.

I'm going to take a (hopefully) long break from ABX testing, as it's REALLY draining, I'll return one day with some more interesting results. Maybe whenever I get new headphones I'll see if I can nail some my harder or impossible tracks.
 
Oct 4, 2011 at 10:47 AM Post #25 of 119
My husband also disputed that I could tell the difference between FLAC and lossy on well-mastered tracks. IIRC I was correct better than 80% of the time on the artists I listen to frequently. Obviously, poorly-mastered tracks on Spotify at 320 still sound like garbage.
 
Oct 4, 2011 at 2:15 PM Post #27 of 119
Legend has it that the mp3 codec was tuned to recreate a passable version of Suzanne Vega's a capella rendition of "Tom's Diner" from her 1987 album. source
 
This calls for a test
 
https://rapidshare.com/files/1487408860/Tom_s_Diner.rar
 
Included in the link are five different versions of a clip from the song: a flac and both fraunhoffer and lame encoded 128 and 320 versions.
 
Deadlylover, I know that you're powering down from testing, but if you would be so kind as to give this one a go, I'd love to see how Herr Brandenburg's reference stands up
popcorn.gif

 
Oct 4, 2011 at 11:56 PM Post #28 of 119
once you go flac , you never go back.
 
my 2 cents.
 
Oct 5, 2011 at 12:16 AM Post #29 of 119
Quote:
Deadlylover, I know that you're powering down from testing, but if you would be so kind as to give this one a go, I'd love to see how Herr Brandenburg's reference stands up
popcorn.gif


The fluffyhouser 128 320 and lame 128 we're easy.
 
I failed the lame 320 and I really don't think I will be able to do it even if I kept trying for two reasons. Firstly, I think lame @ 320 has that particular track down no problem, and secondly, I really want to bash my head against the wall because of that track. No offense intended because I think it's a fine song, I just absolutely hate it.
 
Bleh, audiophile music, makes me cringe. I need to go cleanse my ears, I feel violated.
tongue.gif

 
I'll try it again in the future when I do another run of tests, hopefully with my rig upgraded. It will be a good test to see if better gear can help resolve it or not.
 
Oct 5, 2011 at 12:45 AM Post #30 of 119
 
Quote:
 
Bleh, audiophile music, makes me cringe.
 
 


I think music existed before audiophiles ...
 

Anyway, it's nice you did this test with regular, you know, loudness enhanced, digital, studio music... dispells the 'myth' a little bit about high quality live recordings (acoustic, unplugged) having the most perceptability... sure, they must have the most perceptability for dynamic range, but perhaps not so much for other factors.
 
What I'm saying is, my hunch is a song made entirely digitally will still receive an altered sound from compression, along the same lines as high quality music, and your test is very much leaning towards the digital type of music, so I think your test is worthy of some credibility for mp3-320 versus lossless in the digital flavour of music.
 
In my opinion, MP3 files are convinient, as long as they're small, such as 192. To me 320 is a slightly annoying compromise between convinience, and purity[the strive for fidelity].
 
Now if 540kbps MP3 existed, that would be truly pointless/annoying.
 
Now, I wonder what the perceptability is like, of 90's techno music recorded from a cassette tape, in Flac versus MP3? =p
 
 
Edit: If my above post is confusing, the jist of what I'm saying is is I think all music is worthy of Flac.
 
Even if you don't own a flacship... but more on that later. :wink:
 
On the other hand, for the mere "enjoyment" of music, and convinience of small files (not that that matters with todays internet speeds and 1TB storage space?) then mp3 128~192 is pretty much fine.
 
One thing I have at the back of my mind however, was a test someone linked to on head-fi saying the results showed that the participants preferred mp3 to lossless? Could that be something along the lines of how we prefer the sound of oversampling to non-oversampling? I think there could be a potential conflict with "the preferred sound of compression"... however, that still keeps my point valid that 320 is useless in light of 192 as a better standard of compression for convinience and supposed "preferred sound" (dot dot dot).
 
 
Edit: Once again, my above post is mostly theorizing, and the jist of what I think is I think the world should be covered in fibre optic cables and the European Union and Chinese government should set 24bit / 192kHz FLAC as the new standard for music distribution and we will all live happily ever after!!!!
 
 
 

Users who are viewing this thread

Back
Top