Tricky blind audio test. Take now!

This was very tough indeed! Even on my DT770 Pro's, however, I could pick out more details on Sample 4. Distinguishing between the others was much more difficult, but I think I've got them in a reasonable order:

Sample 4: FLAC

Sample 2: 256kbps MP3

Sample 1: 128kbps MP3

Sample 3: 64kbps HE-AAC

Even though I am quite new to headphones, I was starting to like these sorts of tests; this was a whopper, though!

-Bizzlebin

Nice song bro.

1. 256kbps mp3

2. 128kbps mp3

3. Lossless

4. 64kbps aac

Really tough. I didn't use a ABX comparator, just listened to them through about 5 times. The 4th I definitely zeroed out as the lowest quality, but between the second and third it was pretty much a toss-up for me

1: 128

2: Lossless

3: 256

4: 64

Using my laptop and Sennheiser HD600, I thought they all sounded the same !  

Quote:

Originally Posted by Audio-Omega 

Using my laptop and Sennheiser HD600, I thought they all sounded the same !  

For the most part yes (I'm using the same headphone) but one has looser bass and one has slightly sharper cymbals (not necessarily more accurate though; it sounds a little tinny, not sharper as in more body or reverb to it). Everything else is probably far enough from the extremes that they aren't affected enough by the compression algorithm (ie, most compression works by reducing the freqs at the extremes, since the initial assumption was that people don't hear them well enough).

Just for the heck of it I did somewhat of a blind test with my own audio and compressed/converted the FLAC into MP3, mixed them and the originals in the same folder, then put them in a playlist set to random so I wouldn't know which one is which. Aside from similar observations as above wherever these can be observed the vocals tend to sound thinner, and so are the rest of the percussion. Put the same files in a MicroSD and used it with my mobile phone and a Westone2 - I had to strain to hear any difference but given the file sizes and even if I got a higher capacity microSD, even if the gap was larger I'd much rather cram more music into it than larger files of the same songs.

EDIT : Just saw the actual bitrates are already posted, and the track with looser bass from my listening was the FLAC. Not surprised though since it's supposedly lossless, the bass must have been recorded that way and/or the headphones were adding to it. Never observed it with my own audio though but for the most part they either don't have bass or have fast bass, so they probably didn't have it recorded like that. What surprised me was that the cymbals I thought were most likely 64kbps was actually 246kbps; maybe it would have lacked body but it would have takes out enough of it to not sound too sharp?

One sample had tinny treble and the other had fuller bass but I couldn't be certain.  I felt that the differences were minute.  

@ProtegeManiac: I also pinned the FLAC as poorer quality, but since I'm going to reference some tell-tale signs which can be used to more easily differentiate some of the samples, I'll go ahead and put it in a spoiler box:

Based on my experiences, then, I think this test could be improved by having a fifth sample, also FLAC, with which to compare the other samples to. The control sample would be an undisguised FLAC and the other four could be the same types as were already used. This way, our differing perceptions can be neutralized better; it won't matter what I think sounds better or more pleasing, but what I think sounds closest to the control sample. If this was non-electronic music, I think the having a control would still be quite helpful (especially for those who don't have very much experience going to live performances and hearing many instruments played in person) but maybe not imperative.

One part of the test that surprised me, though, was my mixing up the lower-quality MP3 and the HE-AAC samples. As that sound which I liked (which turned out to be the compression artifact) was actually less pronounced on the 64 kbps HE-AAC than the 128 kbps MP3, I wonder if that will translate to less artifacts in other music. It uses, after all, a newer compression scheme and was designed in order to be better than MP3 compression. Comparing them just once in a very specific type of music isn't enough for me to make a broader judgment on them, but I am intrigued and would like to listen more into the differences between the two formats.

So, to conclude, this was an extremely interesting test and I am disappointed more people didn't participate. However, aQiss, to use this test of a very carefully selected piece of simpler electronic music with no clear control and no other standard of reference and say we don't "really need lossless audio formats" is totally unfair. If you wanted to design a test that was very difficult, then yes, you did a top-notch job. But seeing that myself and some others were able to distinguish pretty accurately between most of the samples (even though we disagreed as to which ones sounded subjectively better and thus about which bitrate was which, having no standard of comparison), this hardly makes the point that the samples were indistinguishable.

-Bizzlebin

Do you think it's much harder to differentiate electronic music than orchestra ?

@Bizzlebin,

Yes I agree that without knowing the track intimately the best we can do really is pick out what "sounds best" vs "what sounds like the original (lossless)." Plus, was that track fully synthesized? I'd like to try a similar test using a recording of acoustic and even electronically amplified instruments, but instead of guessing the bitrates based on multiple choice, just what each thinks is "best" depending on the equipment used. Let's face it, this will play a part in what people hear - I'm not delusional in thinking that maybe it's my headphones that have more bass, or adds to the excessive bass, of the test track. Kudos to the aQiss for the inspiration for a test like this; I'll take a few nights this week to select a couple of tracks to convert to lossy and post for another round of tests. Also, I actually won't be surprised if some people end up picking the lossy files as "better" since our equipment aren't absolutely colorless, which is why I won't post the bitrates of the tracks so people won't be distracted by that.

@Audio-Omega,

From my own experience, yes. As per my post above, the body/weight of lower freq percussion instruments are the first things I notice if I listen to a lossy copy of a track I'm familiar with, followed by the vocals. Like I said, I'll work on another one, and post here when it's up so there's another way to find the link, and I hope the few participants here will take that test too.

Could bit rate be measured via headphones ?

Quote:

Originally Posted by Audio-Omega 

Could bit rate be measured via headphones ?

No, of course not; what comes out of the headphone is analog. That will be kind of like scanning a printout from a Nikon D3 35mm camera and guessing how many megapixels it has - probably not even a microscope can give a close enough answer. However you can still see which photo may have been downscaled, or printed out a large enough size that you begin to see pixellation (and conclude, for example, that the camera that took it wasn't a medium format). You can however measure frequency response if you have, say, a USB microphone, but you won't be able to begin guessing bitrates if you don't have the same data off the lossless copy since not all songs will register the same (ie, some will have louder lows, softer highs, etc depending on the instruments and the recording). A specially designed frequency sweep might be better for that though.

Quote:

Originally Posted by Audio-Omega 

I was meant to ask if the differences we heard could be measured via headphones and plotted on a graph. Each bit rate would show a different frequency response in speculation anyway.

Put that way, theoretically, it should be possible.