Head-Fi.org › Forums › Equipment Forums › Sound Science › ABX Test of 320kbps VS FLAC - Results
New Posts  All Forums:Forum Nav:

ABX Test of 320kbps VS FLAC - Results - Page 2

post #16 of 64
6/28/09 at 1:44pm
I'm going to try this kind of comparison, but not using foobar2000 because my computer rig is nowhere near as good as my Naim CDP. Instead I'm going to burn a CD, and each track will either come from an MP3 or a WAV. I'm a programmer so I'm going to write software that chooses the source for each track randomly.

My computer rig is this:

Audiophile 2496 digital coax out -> CAL Sigma II DAC -> (very long and sh**ty cable) -> DNA Sonett (amp)

For CD listening I have

Naim CD5x -> Cardas Neutral Reference -> DNA Sonett


I ripped a CD to MP3 190 kbs, and using ordinary classical music with no cymbals or any simmilar high frequencies, I must say that the difference didn't jump out at me. (This was using my computer rig.) I would say it's less of a difference than I heard in the cable blind tests I did last week. I know that seems paradoxical, but who knows? Anyway putting it on my main rig may be more revealing.
Reply

Head-Fi's Best Sellers

post #17 of 64
6/29/09 at 3:25pm
Thread Starter 
Quote:
Originally Posted by Earwax View Post
It's interesting that higher flac bitrates didn't predict which pieces you'd have trouble ABXing. That's not at all what I would have guessed.

I am curious about a couple of things. Can you describe the type of difference you heard between the Shostakovitch tracks? How long did it take you to do all those trials? Was this all done in one day or spread out over time?
The differences that I heard were mostly in depth of sound and, I guess, realism. On the less "active" sections of the piece it was impossible to differentiate them. On the complex sections, I found it much easier. I believe I was 14/14 on that test before testing the slower sections of the piece.

Violins were the most helpful to me in determining which one sounded more real, but I really had to listen for the very end or very beginning of each note: that was where the differences were located.

The trials took me quite a while, maybe about 3 hours or so. It couldn't be more; it might actually be a little less, but that time includes all the breaks I took, some of which were rather long. The test was done in one day. I did discover some important things about audio testing in general.

The more times you hear the same track, the more easily fatigued you become. Listen to the two versions too many times, and they begin to sound exactly the same, despite a minor difference you may have noticed during the first five replicates. Because of this, I had to constantly test different sections.

Quote:
Originally Posted by progo View Post
Actually, while the flac bitrates don't mean anything, they might correlate to loudness. In general, hot CDs I've ripped are usually over 1000 kbps but the classical and jazz can be 400-700 kbps. But the numbers don't mean anything in the end. Just the saved amount of disk space.
I'd definitely agree with that. The newer the CD is, the higher bit rate I get off of it in FLAC. The only reasonable conclusion is that the higher the bit rate, the hotter the CD is. All of my good CD's: the first pressings, classical music, recordings mastered by the artist, etc. - All of them have relatively low bit rates.

Quote:
Originally Posted by Headdie View Post
1. When you heard a difference between MP3 and FLAC, was it a continuous difference (meaning that you could hear it all time), or was it a ponctual difference (meaning that you could hear it at some places/passages only) ?

2. I understand that MP3-320 is a very good format, but the encoder could be the weak link. It would means that encoding with WMP or LAME would not yield to bit identical MP3 files. Is that tested and confirmed ?
Thanks,
1. I could hear the difference during some small intervals only. Usually less than a second or so in length.

2. The encoder and program definitely make a difference. At first, I tried some 320 kbps AAC files I had on iTunes. A long time ago, I had burned some CD's using iTunes into that format.

Since I still have the same CD's in lossless form, I compared the two. The ABX test was not required. The difference I heard was substantial enough that it was highly unlikely it was all in my head, unless I'm just crazy.
Reply
post #18 of 64
6/29/09 at 5:01pm
Quote:
Originally Posted by elrod-tom View Post
I'd tend to agree. I appreciate the work you put into this, but if it's done on the typical PC headphone jack I suspect that it's not allowing a lot of details to come through that might otherwise be more revealing of differences.
That is certainly possible. What I'd like to know is if listening via this PC headphone jack is the typical way in which the OP listens to music. That's important because he was convinced there was an obvious difference between the two types of encoding prior to this test.
Reply
post #19 of 64
6/29/09 at 7:50pm
Thread Starter 
Quote:
Originally Posted by Arjisme View Post
That is certainly possible. What I'd like to know is if listening via this PC headphone jack is the typical way in which the OP listens to music. That's important because he was convinced there was an obvious difference between the two types of encoding prior to this test.
Normally I don't listen to my headphones through the headphone jack. I have some horrible computer speakers that shall go unnamed. When comparing FLAC files to 320kbps, I heard an audible difference from those speakers.

That makes little sense, considering that my IEM's are vastly superior, and that one would think the better the equipment, the easier it would be to differentiate the two kinds of formats. Obviously, it was all in my head.

Most of my listening is off of my portable player. I don't have a home setup yet, but I do commute about 4 hours a day to get to and from school, and I get to listen to music the whole way (lucky me!). All my files on that player are lossless, and it'll stay that way regardless of these test results.
Reply
post #20 of 64
7/10/09 at 1:23am
I also ABXed flac files and mp3 file once (256 and 320 kbit/s, lame). I used Tony Joe White songs and played them though my DAC to my big speakers (see my signature). At first, I couldn't tell the difference and failed the comparison test. As I listened more and more, I noticed the difference in the sound of drums and just the overall feel of the music. So, to the end of my listening test, I scored 8/10 or even 10/10. 256 kbit files were easier to tell from FLAC. 320 kbit files were harder to differentiate.

My friend refused to be tested when I offered it to him. He was afraid that he won't be able to maintain his status of being a mp3-hating audiophile
Reply
post #21 of 64
7/10/09 at 1:40am
Quote:
Originally Posted by mike1127 View Post
I'm going to try this kind of comparison, but not using foobar2000 because my computer rig is nowhere near as good as my Naim CDP. Instead I'm going to burn a CD, and each track will either come from an MP3 or a WAV. I'm a programmer so I'm going to write software that chooses the source for each track randomly.

My computer rig is this:

Audiophile 2496 digital coax out -> CAL Sigma II DAC -> (very long and sh**ty cable) -> DNA Sonett (amp)

For CD listening I have

Naim CD5x -> Cardas Neutral Reference -> DNA Sonett


I ripped a CD to MP3 190 kbs, and using ordinary classical music with no cymbals or any simmilar high frequencies, I must say that the difference didn't jump out at me. (This was using my computer rig.) I would say it's less of a difference than I heard in the cable blind tests I did last week. I know that seems paradoxical, but who knows? Anyway putting it on my main rig may be more revealing.

I think your setup for testing is highly non-ideal. I assume "190 kbs" means 190 CBR. You may know that VBR>ABR>CBR. Therefore, you should probably use V2 to achieve roughly the same bitrate with higher SQ. Also, you may want to consider using V0. Make sure you're using the newest version of LAME (3.98 right now). Also make sure you set your encoder to "slow mode" (if available) to maximize SQ. Odds are the difference will be even less or non-existent for the vast majority of tracks. I personally find very few tracks are non-transparent at v2.
Reply
post #22 of 64
4/7/10 at 7:55pm
Do 320kbps mp3 files really sound better? Take the test! | NoiseAddicts music and audio blog

try this. it should tell you how revealing your set up truly is.
Reply
post #23 of 64
4/7/10 at 8:34pm
so VBR is on par with 320 and pretty close to lossless?
Reply
post #24 of 64
4/8/10 at 12:03pm
Quote:
Originally Posted by Saintkeat View Post
Do 320kbps mp3 files really sound better? Take the test! | NoiseAddicts music and audio blog

try this. it should tell you how revealing your set up truly is.
About worthless without a non-lossy control sample though.
Reply
post #25 of 64
4/8/10 at 1:15pm
Quote:
Originally Posted by Saintkeat View Post
Do 320kbps mp3 files really sound better? Take the test! | NoiseAddicts music and audio blog

try this. it should tell you how revealing your set up truly is.
I guessed it right after 3-5 listenings of each clip but this is a quite hard test

As for high quality mp3 vs flac i haven't taken any test but i think i will fail alot.
Reply
post #26 of 64
4/8/10 at 1:56pm
i think most will fail unless they have a dac paired with customs or some good headphones.

i'm quite happy listening to alac now. once i started noticing characteristics of lossless i could spot it easily.. like how i get a lot of clarity because of the huge reduction in 'noise'. by that i mean highs and lows bleeding into each other. gotta find more lossless files to rebuild library
Reply
post #27 of 64
4/10/10 at 10:50am
I own SE530's and they are brilliant. They should have been able to highlight how deficient the computer headphone output is and discourage you from using this methodology. Nevertheless, I couldn't differentiate 320 from lossless when I tried it blind - that was a long time ago though.
Reply
post #28 of 64
4/10/10 at 12:41pm
Quote:
Originally Posted by krmathis View Post
About worthless without a non-lossy control sample though.
You can set up an ABX test however you want though, you don't have to depend on this guy's test. Just load up Foobar, and test flac against whatever bitrate you're curious of. Most people can tell on 50% of 128 kb/s mp3 samples some minor difference between it and flac. Remember though, that Foobar is not asking you which one sounds better, it's just asking for you to separate them into two categories. If you answer the test trying to say which sounds better, you might get even poorer results.

I think, thought maybe not on this forum, it's generally pretty well known lossless are archival formats for transcoding into current lossy formats, and that 360 kb/s is extremely bloated. There are a few killer samples, maybe 1 in 10,000 that are noticeable on any level of mp3 encoding (not sure about other encoders), but for the rest of them basically no one can tell the difference between 192 kb/s and lossless on most modern encoders. These days, 160 kb/s probably isn't suffering either, though I haven't really had the opportunity to see many tests on it. 128 is definitely what everyone is fighting for now, to make transparent that is.

If this all sounds new to you, remember you're on a forum that is always trying to bring back old, obsolete technologies, claim they sound incredible, and then promoting enough demand for a substandard product (based purely on rustic appeal) for a small business to start hiring engineers to start making products they know are inferior because they can sell them for insane prices to some yokels It's hilarious.


Yeah, there's merit to saying a laptop is not the ideal music player, especially if you don't do any of your music listening on your laptop, but most people here I think make a big enough fuss about the difference between lossy and lossless that this is a good way to make them realise the difference, if any, is smaller than their laptop setup is able to show. If there's no difference on the laptop, that doesn't exactly prove there's a difference on your rig.



PS: He did do a lossless vs mp3 blog. WAV is even more lossless than FLAC I'm hoping to start a craze on this forum where people start getting elitist about their untaggable WAV files and refuse to use FLAC for SQ reasons. Sound Test: Difference between Wav vs MP3 | NoiseAddicts music and audio blog
Reply
post #29 of 64
4/10/10 at 1:00pm
It indeed is very, very hard to pick out differences between lossless and 320 kbps mp3's and for most files it is impossible to tell A from B.
Reply
post #30 of 64
4/10/10 at 4:49pm
Quote:
Originally Posted by Chef View Post
You can set up an ABX test however you want though, you don't have to depend on this guy's test.
Sure. But this trhead is about FLAC vs. MP3, not MP3 at a certain bitrate vs. MP3 at different bitrate.

Quote:
WAV is even more lossless than FLAC
Totally wrong!
WAV (the audio file format) can contain lossy audio, like MP3, GSM, and others. While FLAC is always 100% lossless to the source.
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Sound Science

Head-Fi's Best Sellers

Head-Fi.org › Forums › Equipment Forums › Sound Science › ABX Test of 320kbps VS FLAC - Results