Lossless vs 128kbps mp3 vs 320kbps mp3 blind test

[chewy4]

Statistics have now been gathered, but I will leave the files in case anyone wants to ABX them. The key and statistics are in a spoiler at the bottom of this post.
 
Hey everyone,
 
I converted and volume matched some files for a blind test. I'd appreciate your participation, but please follow these guidelines so we can eliminate some unwanted variables entering the test:
1. Put your answers in SPOILERS. Do not read anyone's answers before listening and responding yourself.
 This is so nobodies answer is influenced by anyone else.
2. This is a LISTENING test. Not a looking test. No peaking. If you peak you are a bad person.
3.  Don't do any bizarre equalization or any other fancy tricks. Listen to them like you would normally listen to your music.
 
When you listen to each group I'd like for you to answer the following:
- How do you rank each of the files?
- Were you guessing?
   - If you weren't guessing, what differences did you hear?
   - How difficult was it?
 
On with the test! 
 
For each group, one file is lossless, one is a 128kbps mp3, another is a 320kbps mp3. They are randomly scrambled, so one group's "A" might not be the same type as the other group's "A".
 
 
Group 1 - "Mighty River" by Railroad Earth
Zipped files: http://www.mediafire.com/download.php?u44qxs3j43azgo7

Spoiler: Individual%20files(for%20those%20paranoid%20about%20zip%20compression%20%3A)%20)

 
Source: http://www.mediafire.com/download.php?apisoaeabmjpbmn
A: http://www.mediafire.com/download.php?kwoi8pi8hplea82
B: http://www.mediafire.com/download.php?kxjbujfi135plpy
C: http://www.mediafire.com/download.php?wckkgtuk4t3l5c4
 

 
 
File 2 - "Fahrenheit Fair Enough" by Telefon Tel Aviv
Zipped files: http://www.mediafire.com/download.php?hpi9n4d2i3544di

Spoiler: Individual%20files

 
Source: http://www.mediafire.com/download.php?91i462ih79gnwae
A: http://www.mediafire.com/download.php?j5g3dr31vvzasvw
B: http://www.mediafire.com/download.php?c521p8rx2myu1c3
C: http://www.mediafire.com/download.php?5clv3q643do8o2e
 

 
File 3 - "Falling Awake" by Shpongle
Zipped files: http://www.mediafire.com/download.php?aib6uc1hw1ng73o

Spoiler: Individual%20files

 
Source: http://www.mediafire.com/download.php?i9syghdbn62dfhe
A: http://www.mediafire.com/download.php?ebfuuvya2x3y74q
B: http://www.mediafire.com/download.php?f7006wt04kc6w4f
C: http://www.mediafire.com/download.php?mz64z6d5i84tc7j
 

 
 
 
While I'd appreciate it if you listened to all three groups, if you don't have the time feel free to only give your feedback on one or two. I tried to make the groups pretty different from each other.
 
Happy Listening!
 
 

Spoiler: Warning%3A%20Spoiler!%20Key%20and%20Statistics!

KEY
Group 1:

A.320	B.Lossless	C.128
Group 2: A.Lossless	[size=inherit]B.320[/size]	[size=inherit]C.320[/size]
Group 3: A.128	B.320	C.Lossless

STATISTICS
Group 1
A(320kbps): 4/11(36%) were correct. Only 1 person said they were guessing, and they guessed right.
 
B(Lossless): 3/9(33%) were correct. 1 person said they guessed, and they guessed wrong. One person said that it was the opposite bitrate.
 
C(128kbps): 7/10(70%) were correct. Two people said it was the opposite bitrate. 
 
Group 2
A(Lossless):3/9(33%) were correct.  Two of the correct answers said they were guessing. Two people said it was the opposite bitrate.
 
B(320kbps):1/8(12.5%) were correct. The person who answered correctly said it was a guess. Two of the wrong answers were said to be guesses.
 
C(320kbps): 4/9(44.4%) were correct. Three correct answers were said to be guesses. One wrong answer was said to be a guess.
 
Group 3
A(128kbps): 7/11(63.6%) were correct. One person said it was the opposite bitrate. 
 
B(320kbps): 5/10(50%) were correct.

C(lossless): 5/10(50%) were correct. Four people said it was the opposite bitrate.
 
 
One answer for each group said they couldn't tell a difference.
 
I also posted this on another forum, only a few people answered though.
One person guessed all of group 1 wrong, and they were unsure about their answers.
Another person guessed all of group 3 right, and they described what differences they heard.
One person said they couldn't tell a difference.
One person said they didn't give a schiit.
 
 
 
Conclusion
 
These are some fairly interesting results. Group 2 was a little weird, and not completely fair because I unintentionally lied about there being 3 types of file. But the largest amount of people said that their answers were guesses for this one, so hooray for you folks.
 
I found group 3 to be the most interesting. Almost half the people who ranked it said that the lossless file was 128kbps! At the same time, it was just behind the first group in the amount of people guessing correctly on the 128kbps.
 
So, what does any of this actually prove? Absolutely nothing statistically speaking, although there did seem to be a higher percentage of people guessing the 128kbps file right. Hopefully it gave some people a better idea of what compression really sounds like.
 

 

[musical-kage]

Spoiler: Warning%3A%20Spoiler!

I'm only listening to one right now but will edit later. I need to go to sleep really, haha.
My choice is 'Fahrenheit Fair Enough' by Telefon Tel Aviv:

A = 128kbps
B = 320kbps (guess)
C = Lossless (guess)

2^
 
Just from initial listens. I think I'm probably wrong on all counts... Listened a few times again and changed my perceptions based on certain parts of the track.
 
Really struggled to be honest.
 
3:

Spoiler: Warning%3A%20Spoiler!

"Falling Awake" by Shpongle.
Damn you not providing us with the higher treble parts

A = Lossless
B = 128kbps
C = 320kbps

 
1:

Spoiler: Warning%3A%20Spoiler!

"Mighty River" by Railroad Earth.
I just hope Winamp is opening them in order of selection

A = 320kbps?
B = Lossless
C = 128kbps

 
EDIT:
 
Mine were all mixed up and its hard to correct too, so I've put numbers where each one is

 

[KamijoIsMyHero]

I already know I can't tell the difference between lossless and 320 so I will just pick out what I think was the 128
 
 

Spoiler: Warning%3A%20Spoiler!

128:
1A
2C first one that stood out IDK why
more acoustic guitars  -.- 3A
 

 Mann, you just had to use 2 acoustic guitar and an electronics track, 99% of my music have none of that, its pretty difficult since I am not used to it at all
 
I didn't double check my picks, I think my ears have grown too accustomed to try and re-examine the tracks

 

[Strangelove424]

Thanks for getting the files together Chewy, I enjoyed the samples and I liked the 1st one so much I looked up Railroad Earth on MOG - very good stuff. Pass or fail, I'm really happy to have some new music to try out. I love folk!
 

Spoiler: Warning%3A%20Spoiler!

 
Group1 [size=12.0pt][/size]
A>B>C, decided based on acoustic reverb and clarity of strumming
Group 2
B>A>C, decided based on airiness and richness[size=12.0pt][/size]
Group 3
B>A>C, decided on acoustic reverb and clarity of strumming[size=12.0pt][/size]
 

 

[cer]

Spoiler: Warning%3A%20Spoiler!

Only tried the first song - "Mighty River". At first just casually listened to the files in foobar. They seemed reasonably similar, especially A and B. But something seemed to be wrong with C. So I ABX-d B and C. 10/10 and pretty convinced of the difference. Not a massive difference, but detectable when paying attention. 
I used headphones - Beyer DT880 and E-MU 0202 USB sound card.
So, C is definitely 128 kbit/s. It just has that familiar impureness of transients, some sort of wrong echo around them. I won't bother with A vs B.
So it seems, that 128 kbit/s still is different from the original and I would say, in a detrimental way. I wouldn't choose C as the one I would want to listen[size=1em].[/size]
 
[size=1em]So the ABX result here:[/size]
foo_abx 1.3.4 report
foobar2000 v1.1.15
2013/01/16 10:25:42
 
File A: C:\Documents and Settings\Owner\My Documents\1_B.wav
File B: C:\Documents and Settings\Owner\My Documents\1_C.wav
 
10:25:42 : Test started.
10:26:20 : 01/01  50.0%
10:27:21 : 02/02  25.0%
10:28:14 : 03/03  12.5%
10:28:52 : 04/04  6.3%
10:29:55 : 05/05  3.1%
10:30:37 : 06/06  1.6%
10:31:23 : 07/07  0.8%
10:31:58 : 08/08  0.4%
10:32:11 : 09/09  0.2%
10:32:39 : 10/10  0.1%
10:43:41 : Test finished.
 
 ---------- 
Total: 10/10 (0.1%)

 

 

[musical-kage]

Yeah thanks for this Chewy, it was really enlightening actually, and some of the best test material I've heard!
I listened to the whole of Shpongle's album this morning. Couldn't find the Telefon Tel Aviv track on Spotify/Deezer or Rdio though

 

[chewy4]

No problem, glad you guys liked the files. Hopefully enough people take this so that I don't have to wait too long to put the results up.
 
Sorry I wasn't able to put any metal or the likes in there for you KamijolsMyHero, the closest stuff I had was either not lossless or of not that great recording quality. I mentioned in another thread that I wanted to avoid stuff with any hiss in the background and that's hard to avoid when distorted guitars come into play.
 

Couldn't find the Telefon Tel Aviv track on Spotify/Deezer or Rdio though

 


The remaining member of Telefon Tel Aviv hasn't exactly been trying to get their stuff out there since the other guy died, so you'll probably have trouble finding it on any of those streaming services(some of their stuff is on lastfm but not of that album, which is completely different than all their others). Good luck finding it lossless legally too, I ended up buying the CD for about $30 used off discogs. I think most is on YouTube though if you want to listen to it.

 

[musical-kage]

Quote:

No problem, glad you guys liked the files. Hopefully enough people take this so that I don't have to wait too long to put the results up.
 
Sorry I wasn't able to put any metal or the likes in there for you KamijolsMyHero, the closest stuff I had was either not lossless or of not that great recording quality. I mentioned in another thread that I wanted to avoid stuff with any hiss in the background and that's hard to avoid when distorted guitars come into play.
 

 
The remaining member of Telefon Tel Aviv hasn't exactly been trying to get their stuff out there since the other guy died, so you'll probably have trouble finding it on any of those streaming services(some of their stuff is on lastfm but not of that album, which is completely different than all their others). Good luck finding it lossless legally too, I ended up buying the CD for about $30 used off discogs. I think most is on YouTube though if you want to listen to it.

I should have made sure Winamp put the songs in order. With one of the sets, it kept the X1,X2,X3 in Winamp. The other two opened up with the file names so you couldn't see if they were A,B,C etc.
 
Ah, I know Even their website doesn't go anywhere as far as I can tell. It just has the announcement and then, it won't go any farther.

 

[chewy4]

You should be able to have it display the actual file name instead of the title IIRC, but I should probably wipe the tags from the files. I'll do that and re-upload tonight so nobody else runs into that problem.

 

[badhomaks]

Spoiler: No%20point%20in%20peeking%2C%20you're%20just%20lying%20to%20yourself

For the first one is seems to be A>B>C
C had this weird stereo effect like the song was alternating between left and right speakers
For 2nd one after a few mins I just gave up. The loud squeaking hurt my ears.
 

 

[Kaffeemann]

Spoiler: Warning%3A%20Spoiler!

Could not hear any difference in group 1 and 2.

 

[Strangelove424]

I can't seem to submit a single post anymore without a typo. In my answer box, the second Group 2 should be labeled Group 3. I don't want to edit my original post though. I also neglected to write down my observation that it was a difficult test, which confirms my belief that well-mastered music sounds good no matter what, and compressed junk sounds bad no matter what.  

 

[chewy4]

Quote:

I can't seem to submit a single post anymore without a typo. In my answer box, the second Group 2 should be labeled Group 3. I don't want to edit my original post though. I also neglected to write down my observation that it was a difficult test, which confirms my belief that well-mastered music sounds good no matter what, and compressed junk sounds bad no matter what.  

It would probably help if you edited since I'll forget about that when it comes time to collect the statistics. 
Don't worry I'll trust that you didn't change your answers. 

 
I threw the groups in their own zip files so people don't have to click a million links. Probably should have done that in the first place.

 

[sonitus mirus]

Wow!  This is absolutely wonderful.  Thank you so much for making this comparison available.  I love your song choices, too.  If you ever see a post of mine making a recommendation for any music that you might not have heard before, jump on it.  I think our tastes are very similar, or at least eclectic enough to enjoy a little about almost everything.
 
 
I ranked each file within its group from worst sounding to best sounding.  So anything with a 3 is my guess at the lossless file, 2 would be the 320kbps mp3, and 1 is my guess at the 128kbps mp3.
 
 

Spoiler: Warning%3A%20Spoiler!

 
Group 1
 
A-3
B-1
C-2
 
Group 2
 
A-1
B-3
C-2
 
Group 3
 
A-1
B-3
C-2
 

 

[Strangelove424]

Quote:

It would probably help if you edited since I'll forget about that when it comes time to collect the statistics. 
Don't worry I'll trust that you didn't change your answers. 

Ok, I edited it.