Head-Fi.org › Forums › Equipment Forums › Sound Science › Lossless vs 128kbps mp3 vs 320kbps mp3 blind test
New Posts  All Forums:Forum Nav:

Lossless vs 128kbps mp3 vs 320kbps mp3 blind test

post #1 of 103
1/15/13 at 5:35pm
Thread Starter 

Statistics have now been gathered, but I will leave the files in case anyone wants to ABX them. The key and statistics are in a spoiler at the bottom of this post.

 

Hey everyone,

 

I converted and volume matched some files for a blind test. I'd appreciate your participation, but please follow these guidelines so we can eliminate some unwanted variables entering the test:

1. Put your answers in SPOILERS. Do not read anyone's answers before listening and responding yourself.

 This is so nobodies answer is influenced by anyone else.

2. This is a LISTENING test. Not a looking test. No peaking. If you peak you are a bad person.

3.  Don't do any bizarre equalization or any other fancy tricks. Listen to them like you would normally listen to your music.

 

When you listen to each group I'd like for you to answer the following:

- How do you rank each of the files?

- Were you guessing?

   - If you weren't guessing, what differences did you hear?

   - How difficult was it?

 

On with the test! 

 

For each group, one file is lossless, one is a 128kbps mp3, another is a 320kbps mp3. They are randomly scrambled, so one group's "A" might not be the same type as the other group's "A".

 

 

Group 1 - "Mighty River" by Railroad Earth

Zipped files: http://www.mediafire.com/download.php?u44qxs3j43azgo7

Individual files(for those paranoid about zip compression :) ) (Click to show)

 

Source: http://www.mediafire.com/download.php?apisoaeabmjpbmn

A: http://www.mediafire.com/download.php?kwoi8pi8hplea82
B: http://www.mediafire.com/download.php?kxjbujfi135plpy
C: http://www.mediafire.com/download.php?wckkgtuk4t3l5c4

 

 

 

File 2 - "Fahrenheit Fair Enough" by Telefon Tel Aviv

Zipped files: http://www.mediafire.com/download.php?hpi9n4d2i3544di

Individual files (Click to show)

 

Source: http://www.mediafire.com/download.php?91i462ih79gnwae

A: http://www.mediafire.com/download.php?j5g3dr31vvzasvw

B: http://www.mediafire.com/download.php?c521p8rx2myu1c3

C: http://www.mediafire.com/download.php?5clv3q643do8o2e

 

 

File 3 - "Falling Awake" by Shpongle

Zipped files: http://www.mediafire.com/download.php?aib6uc1hw1ng73o

Individual files (Click to show)

 

Source: http://www.mediafire.com/download.php?i9syghdbn62dfhe
A: http://www.mediafire.com/download.php?ebfuuvya2x3y74q

B: http://www.mediafire.com/download.php?f7006wt04kc6w4f

C: http://www.mediafire.com/download.php?mz64z6d5i84tc7j

 

 

 

 

While I'd appreciate it if you listened to all three groups, if you don't have the time feel free to only give your feedback on one or two. I tried to make the groups pretty different from each other.

 

Happy Listening!

 

 

Warning: Spoiler! Key and Statistics! (Click to show)

KEY

Group 1:
A.320 B.Lossless C.128
Group 2:
A.Lossless
B.320
C.320
Group 3:
A.128
B.320
C.Lossless
STATISTICS
Group 1
A(320kbps): 4/11(36%) were correct. Only 1 person said they were guessing, and they guessed right.
 
B(Lossless): 3/9(33%) were correct. 1 person said they guessed, and they guessed wrong. One person said that it was the opposite bitrate.
 
C(128kbps): 7/10(70%) were correct. Two people said it was the opposite bitrate. 
 
Group 2
A(Lossless):3/9(33%) were correct.  Two of the correct answers said they were guessing. Two people said it was the opposite bitrate.
 
B(320kbps):1/8(12.5%) were correct. The person who answered correctly said it was a guess. Two of the wrong answers were said to be guesses.
 
C(320kbps): 4/9(44.4%) were correct. Three correct answers were said to be guesses. One wrong answer was said to be a guess.
 
Group 3
A(128kbps): 7/11(63.6%) were correct. One person said it was the opposite bitrate. 
 
B(320kbps): 5/10(50%) were correct.

C(lossless): 5/10(50%) were correct. Four people said it was the opposite bitrate.
 
 
One answer for each group said they couldn't tell a difference.
 
I also posted this on another forum, only a few people answered though.
One person guessed all of group 1 wrong, and they were unsure about their answers.
Another person guessed all of group 3 right, and they described what differences they heard.
One person said they couldn't tell a difference.
One person said they didn't give a schiit.
 
 
 
Conclusion
 
These are some fairly interesting results. Group 2 was a little weird, and not completely fair because I unintentionally lied about there being 3 types of file. But the largest amount of people said that their answers were guesses for this one, so hooray for you folks.
 
I found group 3 to be the most interesting. Almost half the people who ranked it said that the lossless file was 128kbps! At the same time, it was just behind the first group in the amount of people guessing correctly on the 128kbps.
 
So, what does any of this actually prove? Absolutely nothing statistically speaking, although there did seem to be a higher percentage of people guessing the 128kbps file right. Hopefully it gave some people a better idea of what compression really sounds like.
 

Edited by chewy4 - 1/21/13 at 4:47pm
Reply

Head-Fi's Best Sellers

post #2 of 103
1/15/13 at 5:52pm
Warning: Spoiler! (Click to show)

I'm only listening to one right now but will edit later. I need to go to sleep really, haha.
My choice is 'Fahrenheit Fair Enough' by Telefon Tel Aviv:

A = 128kbps
B = 320kbps (guess)
C = Lossless (guess)

2^

 

Just from initial listens. I think I'm probably wrong on all counts... Listened a few times again and changed my perceptions based on certain parts of the track.

 

Really struggled to be honest.

 

3:

Warning: Spoiler! (Click to show)

"Falling Awake" by Shpongle.
Damn you not providing us with the higher treble parts :p

A = Lossless
B = 128kbps
C = 320kbps

 

1:

Warning: Spoiler! (Click to show)

"Mighty River" by Railroad Earth.
I just hope Winamp is opening them in order of selection

A = 320kbps?
B = Lossless
C = 128kbps

 

EDIT:

 

Mine were all mixed up and its hard to correct too, so I've put numbers where each one is


Edited by musical-kage - 1/18/13 at 6:15pm
Reply
post #3 of 103
1/15/13 at 10:29pm

I already know I can't tell the difference between lossless and 320 so I will just pick out what I think was the 128

 

 

Warning: Spoiler! (Click to show)

128:

1A
2C first one that stood out IDK why
more acoustic guitars  -.- 3A

 

 Mann, you just had to use 2 acoustic guitar and an electronics track, 99% of my music have none of that, its pretty difficult since I am not used to it at all

 

I didn't double check my picks, I think my ears have grown too accustomed to try and re-examine the tracks


Edited by KamijoIsMyHero - 1/15/13 at 10:31pm
Reply
post #4 of 103
1/15/13 at 11:44pm

Thanks for getting the files together Chewy, I enjoyed the samples and I liked the 1st one so much I looked up Railroad Earth on MOG - very good stuff. Pass or fail, I'm really happy to have some new music to try out. I love folk!

 

Warning: Spoiler! (Click to show)

 

Group1

A>B>C, decided based on acoustic reverb and clarity of strumming

Group 2

B>A>C, decided based on airiness and richness

Group 3

B>A>C, decided on acoustic reverb and clarity of strumming

 


Edited by Strangelove424 - 1/16/13 at 8:27pm
Reply
post #5 of 103
1/16/13 at 12:44am
Warning: Spoiler! (Click to show)

Only tried the first song - "Mighty River". At first just casually listened to the files in foobar. They seemed reasonably similar, especially A and B. But something seemed to be wrong with C. So I ABX-d B and C. 10/10 and pretty convinced of the difference. Not a massive difference, but detectable when paying attention. 

I used headphones - Beyer DT880 and E-MU 0202 USB sound card.
So, C is definitely 128 kbit/s. It just has that familiar impureness of transients, some sort of wrong echo around them. I won't bother with A vs B.
So it seems, that 128 kbit/s still is different from the original and I would say, in a detrimental way. I wouldn't choose C as the one I would want to listen.
 
So the ABX result here:
foo_abx 1.3.4 report
foobar2000 v1.1.15
2013/01/16 10:25:42
 
File A: C:\Documents and Settings\Owner\My Documents\1_B.wav
File B: C:\Documents and Settings\Owner\My Documents\1_C.wav
 
10:25:42 : Test started.
10:26:20 : 01/01  50.0%
10:27:21 : 02/02  25.0%
10:28:14 : 03/03  12.5%
10:28:52 : 04/04  6.3%
10:29:55 : 05/05  3.1%
10:30:37 : 06/06  1.6%
10:31:23 : 07/07  0.8%
10:31:58 : 08/08  0.4%
10:32:11 : 09/09  0.2%
10:32:39 : 10/10  0.1%
10:43:41 : Test finished.
 
 ---------- 
Total: 10/10 (0.1%)

 

Reply
post #6 of 103
1/16/13 at 4:36am

Yeah thanks for this Chewy, it was really enlightening actually, and some of the best test material I've heard!

I listened to the whole of Shpongle's album this morning. Couldn't find the Telefon Tel Aviv track on Spotify/Deezer or Rdio though :(

Reply
post #7 of 103
1/16/13 at 9:48am
Thread Starter 

No problem, glad you guys liked the files. Hopefully enough people take this so that I don't have to wait too long to put the results up.

 

Sorry I wasn't able to put any metal or the likes in there for you KamijolsMyHero, the closest stuff I had was either not lossless or of not that great recording quality. I mentioned in another thread that I wanted to avoid stuff with any hiss in the background and that's hard to avoid when distorted guitars come into play.

 


Quote:
Originally Posted by musical-kage View Post
Couldn't find the Telefon Tel Aviv track on Spotify/Deezer or Rdio though :(
 

The remaining member of Telefon Tel Aviv hasn't exactly been trying to get their stuff out there since the other guy died, so you'll probably have trouble finding it on any of those streaming services(some of their stuff is on lastfm but not of that album, which is completely different than all their others). Good luck finding it lossless legally too, I ended up buying the CD for about $30 used off discogs. I think most is on YouTube though if you want to listen to it.


Edited by chewy4 - 1/16/13 at 9:49am
Reply
post #8 of 103
1/16/13 at 10:50am
Quote:
Originally Posted by chewy4 View Post

No problem, glad you guys liked the files. Hopefully enough people take this so that I don't have to wait too long to put the results up.

 

Sorry I wasn't able to put any metal or the likes in there for you KamijolsMyHero, the closest stuff I had was either not lossless or of not that great recording quality. I mentioned in another thread that I wanted to avoid stuff with any hiss in the background and that's hard to avoid when distorted guitars come into play.

 


 

The remaining member of Telefon Tel Aviv hasn't exactly been trying to get their stuff out there since the other guy died, so you'll probably have trouble finding it on any of those streaming services(some of their stuff is on lastfm but not of that album, which is completely different than all their others). Good luck finding it lossless legally too, I ended up buying the CD for about $30 used off discogs. I think most is on YouTube though if you want to listen to it.


I should have made sure Winamp put the songs in order. With one of the sets, it kept the X1,X2,X3 in Winamp. The other two opened up with the file names so you couldn't see if they were A,B,C etc.

 

Ah, I know :( Even their website doesn't go anywhere as far as I can tell. It just has the announcement and then, it won't go any farther.

Reply
post #9 of 103
1/16/13 at 10:59am
Thread Starter 

You should be able to have it display the actual file name instead of the title IIRC, but I should probably wipe the tags from the files. I'll do that and re-upload tonight so nobody else runs into that problem.

Reply
post #10 of 103
1/16/13 at 2:32pm
No point in peeking, you're just lying to yourself (Click to show)

For the first one is seems to be A>B>C

C had this weird stereo effect like the song was alternating between left and right speakers
For 2nd one after a few mins I just gave up. The loud squeaking hurt my ears.

 

Reply
post #11 of 103
1/16/13 at 3:33pm
Warning: Spoiler! (Click to show)

Could not hear any difference in group 1 and 2.

Reply
post #12 of 103
1/16/13 at 3:34pm

I can't seem to submit a single post anymore without a typo. In my answer box, the second Group 2 should be labeled Group 3. I don't want to edit my original post though. I also neglected to write down my observation that it was a difficult test, which confirms my belief that well-mastered music sounds good no matter what, and compressed junk sounds bad no matter what.  

Reply
post #13 of 103
1/16/13 at 5:13pm
Thread Starter 
Quote:
Originally Posted by Strangelove424 View Post

I can't seem to submit a single post anymore without a typo. In my answer box, the second Group 2 should be labeled Group 3. I don't want to edit my original post though. I also neglected to write down my observation that it was a difficult test, which confirms my belief that well-mastered music sounds good no matter what, and compressed junk sounds bad no matter what.  

It would probably help if you edited since I'll forget about that when it comes time to collect the statistics. 

Don't worry I'll trust that you didn't change your answers. normal_smile%20.gif

 

I threw the groups in their own zip files so people don't have to click a million links. Probably should have done that in the first place.

Reply
post #14 of 103
1/16/13 at 7:14pm

Wow!  This is absolutely wonderful.  Thank you so much for making this comparison available.  I love your song choices, too.  If you ever see a post of mine making a recommendation for any music that you might not have heard before, jump on it.  I think our tastes are very similar, or at least eclectic enough to enjoy a little about almost everything.

 

 

I ranked each file within its group from worst sounding to best sounding.  So anything with a 3 is my guess at the lossless file, 2 would be the 320kbps mp3, and 1 is my guess at the 128kbps mp3.

 

 

Warning: Spoiler! (Click to show)

 

Group 1
 
A-3
B-1
C-2
 
Group 2
 
A-1
B-3
C-2
 
Group 3
 
A-1
B-3
C-2

 

Reply
post #15 of 103
1/16/13 at 8:29pm
Quote:
Originally Posted by chewy4 View Post

It would probably help if you edited since I'll forget about that when it comes time to collect the statistics. 

Don't worry I'll trust that you didn't change your answers. normal_smile%20.gif

Ok, I edited it.

Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Sound Science

Head-Fi's Best Sellers

Head-Fi.org › Forums › Equipment Forums › Sound Science › Lossless vs 128kbps mp3 vs 320kbps mp3 blind test