Berlioz
100+ Head-Fier
- Joined
- Apr 8, 2009
- Posts
- 249
- Likes
- 14
I realize that there is another thread just like this one in this sub forum, but I didn't want to hijack the thread, and I was curious enough to do my own test.
The structure of the test went like this:
I used the ABX Comparator on foobar2000. It is an optional utility during installation of the program.
How it works:
You choose two files to test. You can pick what section of the song to examine, and how short it is etc.
There are two sets of two choices each: [A and B] & [X and Y]
You can compare any of them to one another at any time, as many times as you like. Basically, you need to match them up in pairs.
One of the [A and B] choices will be lossless, and one will be the 320kbps version.
Same thing goes for the [X and Y] choices.
I would compare the A and B tracks, and decide which one sounded better. Then I would do the same with the X and Y tracks. If B sounded better than A, and X sounded better than Y, I would pick the option "B is X".
Sorry if that's a little confusing for some people; that's the best way I can word it.
I chose four pieces:
1) Symphony No. 1 by Dmitri Shostakovitch - As with all of his work, this dynamic piece is packed with energy. I was curious if I could separate the two versions based on transients.
2) Gold Rush by Dragonette - Electro Pop that is full of compressed sounds, particularly in the low range. I thought it would be a good test of the type of mastering that most people listen to on the radio.
3) Rambaldo!...Ah! M'aiutate! - A section of an Italian Opera La Rondine by Giacomo Puccini. Vocals and the backing orchestra constantly strive for top spot across the entire piece. This recording won the 1997 Gramophone Award for "Record of the Year".
4) Sunshine of Your Love by Cream - This song got me into music in the first place, so I couldn't leave it out! Clapton's riffs are overdriven yet articulate at the same time. Coupled with his vocals, it's another good test of dynamics in my opinion.
Preparation:
For equipment, I was simply using my Shure SE530's connected to the headphone jack on my computer.
All of the tracks were copied twice using EAC. Once in a 320 kbps .mp3 format using the LAME encoder. The second time, in .flac using the FLAC encoder bundled with EAC.
There would be 40 replicates of each test, per song, so 160 trials altogether. I could compare the tracks as many times as I liked, and I took breaks about every 10 trials (trust me, I needed them!).
All of the MP3's are compressed to have a bitrate of 320 kbps.
The bitrates for the FLAC files are as follows:
Symphony No. 1 - 427 kbps
Gold Rush - 1001 kbps
Rambaldo!...Ah! M'aiutate! - 628 kbps
Sunshine of Your Love - 818 kbps
Foobar2000 also calculates your probability of randomly guessing and getting the same results,although it only goes to 1 decimal place. To calculate the values on your own, with more decimal places, use the binomial probability equation.
Results:
Symphony No. 1 -
Correctly differentiated them 28/40 times. Probability of guessing: 0.8%
I was very pleased with this result. It was extremely hard to differentiate the selections, even though I focused on a variety of sections of the song. The least complex sections that I tested gave me the most trouble.
Gold Rush
Correctly differentiated them 18/40 times. Probability of guessing: 78.5%
This made me sad. I have always sworn that Martina Sorbaras voice sounded sexier on FLAC. Oh well. I honestly couldn't make out any difference between the two versions. About a dozen or so replicates in I was already getting discouraged. It was impossible to hear any difference.
Rambaldo!...Ah! M'aiutate!
Correctly differentiated them 22/40 times. Probability of guessing: 31.8%
I had high hopes for this one. At this point, I began to realize what others have been saying. I tried as hard as I could, but I could not discern any differences whatsoever. I would hear A to have crisper vocals than B, and hear the same quality in Y. I would pair A up with Y....and be wrong!
Oh well, there's still another test left!
Sunshine of Your Love
Correctly differentiated them 24/40 times. Probability of guessing: 13.4%
Again, it was impossible to tell the difference. I took a lot of breaks, and felt well rested, but it seemed to make no difference.
Conclusion:
Only one of the results had enough of a difference to be statistically significant. In doing this test, I discovered just how close 320kbps and flac really are. Even with Symphony No.1, I was focusing on the music so ****ing hard, to the point where it wasn't even enjoyable anymore.
It's important to note, that this is only one set of tests, and really doesn't mean anything in the big picture. Maybe my hearing sucks. Maybe my equipment is so poor I can't hear all the little nuances.
However, the fact alone that it was so hard to tell them apart in the first place...there's not nearly as much difference between the two as I found in sighted "tests".
I take back any statements I have ever made as to whether I can hear a difference between FLAC and 320kbps MP3s. I didn't know what the hell I was talking about. It's funny, because as soon as I play them again the .flac one sounds so much crisper. But I know it's all in my head now.
I hope that anyone reading this enjoyed my experiment, and maybe you'll be curious enough to do a test of your own.
Any comments/questions are welcome.
The structure of the test went like this:
I used the ABX Comparator on foobar2000. It is an optional utility during installation of the program.
How it works:
You choose two files to test. You can pick what section of the song to examine, and how short it is etc.
There are two sets of two choices each: [A and B] & [X and Y]
You can compare any of them to one another at any time, as many times as you like. Basically, you need to match them up in pairs.
One of the [A and B] choices will be lossless, and one will be the 320kbps version.
Same thing goes for the [X and Y] choices.
I would compare the A and B tracks, and decide which one sounded better. Then I would do the same with the X and Y tracks. If B sounded better than A, and X sounded better than Y, I would pick the option "B is X".
Sorry if that's a little confusing for some people; that's the best way I can word it.
I chose four pieces:
1) Symphony No. 1 by Dmitri Shostakovitch - As with all of his work, this dynamic piece is packed with energy. I was curious if I could separate the two versions based on transients.
2) Gold Rush by Dragonette - Electro Pop that is full of compressed sounds, particularly in the low range. I thought it would be a good test of the type of mastering that most people listen to on the radio.
3) Rambaldo!...Ah! M'aiutate! - A section of an Italian Opera La Rondine by Giacomo Puccini. Vocals and the backing orchestra constantly strive for top spot across the entire piece. This recording won the 1997 Gramophone Award for "Record of the Year".
4) Sunshine of Your Love by Cream - This song got me into music in the first place, so I couldn't leave it out! Clapton's riffs are overdriven yet articulate at the same time. Coupled with his vocals, it's another good test of dynamics in my opinion.
Preparation:
For equipment, I was simply using my Shure SE530's connected to the headphone jack on my computer.
All of the tracks were copied twice using EAC. Once in a 320 kbps .mp3 format using the LAME encoder. The second time, in .flac using the FLAC encoder bundled with EAC.
There would be 40 replicates of each test, per song, so 160 trials altogether. I could compare the tracks as many times as I liked, and I took breaks about every 10 trials (trust me, I needed them!).
All of the MP3's are compressed to have a bitrate of 320 kbps.
The bitrates for the FLAC files are as follows:
Symphony No. 1 - 427 kbps
Gold Rush - 1001 kbps
Rambaldo!...Ah! M'aiutate! - 628 kbps
Sunshine of Your Love - 818 kbps
Foobar2000 also calculates your probability of randomly guessing and getting the same results,although it only goes to 1 decimal place. To calculate the values on your own, with more decimal places, use the binomial probability equation.
Results:
Symphony No. 1 -
Correctly differentiated them 28/40 times. Probability of guessing: 0.8%
I was very pleased with this result. It was extremely hard to differentiate the selections, even though I focused on a variety of sections of the song. The least complex sections that I tested gave me the most trouble.
Gold Rush
Correctly differentiated them 18/40 times. Probability of guessing: 78.5%
This made me sad. I have always sworn that Martina Sorbaras voice sounded sexier on FLAC. Oh well. I honestly couldn't make out any difference between the two versions. About a dozen or so replicates in I was already getting discouraged. It was impossible to hear any difference.
Rambaldo!...Ah! M'aiutate!
Correctly differentiated them 22/40 times. Probability of guessing: 31.8%
I had high hopes for this one. At this point, I began to realize what others have been saying. I tried as hard as I could, but I could not discern any differences whatsoever. I would hear A to have crisper vocals than B, and hear the same quality in Y. I would pair A up with Y....and be wrong!
Oh well, there's still another test left!
Sunshine of Your Love
Correctly differentiated them 24/40 times. Probability of guessing: 13.4%
Again, it was impossible to tell the difference. I took a lot of breaks, and felt well rested, but it seemed to make no difference.
Conclusion:
Only one of the results had enough of a difference to be statistically significant. In doing this test, I discovered just how close 320kbps and flac really are. Even with Symphony No.1, I was focusing on the music so ****ing hard, to the point where it wasn't even enjoyable anymore.
It's important to note, that this is only one set of tests, and really doesn't mean anything in the big picture. Maybe my hearing sucks. Maybe my equipment is so poor I can't hear all the little nuances.
However, the fact alone that it was so hard to tell them apart in the first place...there's not nearly as much difference between the two as I found in sighted "tests".
I take back any statements I have ever made as to whether I can hear a difference between FLAC and 320kbps MP3s. I didn't know what the hell I was talking about. It's funny, because as soon as I play them again the .flac one sounds so much crisper. But I know it's all in my head now.
I hope that anyone reading this enjoyed my experiment, and maybe you'll be curious enough to do a test of your own.
Any comments/questions are welcome.