Tricky blind audio test. Take now! - Page 2

I just tried this test out of curiosity.

I can tell differences between all four files but it gets complicated to assign some of the bitrates and file formats because some of the differences I hear are actually a bit confusing and make me wonder if all the tracks were taken from different sources or if they're all from the same source and downconverted from there...

The only posts, of this thread, I've read so far are the first one and the last one.

I haven't read the answers disclosure yet, will do only after posting this.

My system: PC - foobar2000 - Yulong D100 - German Maestro GMP 400 (modified)

Ranking the files by sound quality:

Sample_1 > Sample_3 > Sample_4 > Sample_2

This is what my ears tell me, however it gets difficult to assign the 128Kpbs mp3 and HE-AAC file formats...

Anyway here go my guesses:

Sample_1 is Flac. I'm pretty sure of this one as it sounds cleaner, more articulate, better imaging, more saturated tonal colors, smoother and more extended treble without any harshness or congestion, more transparent.

Sample_2 is 64 Kpbs HE-AAC. This one sounds definitelly the worse, and it sounds very different from the Sample_3 and 4 which makes me think that it might have been taken from a different source or that it was encoded with a different method and this is the reason why I think that it is in HE-AAC and not 128Kpbs mp3. (I have no reference and don't know how a HE-AAC file compares with a mp3 for sound quality)

Sample_3 is 256Kpbs mp3. This one and Sample_4 sound very alike in the way that both seem to have a very slightly prominent and muffled low frequncies performance which makes me think they're both mp3s, this one sounds better than Sample_4 but not as good as Sample_1.

Sample_4 is 128Kpbs mp3. Most prominent low frequencies, muffled and second worst performance at frequency extremes after Sample_2, kinda forward sounding . Just like Sample_3 this one sounds warmer than Sample_1 and 2.

So let's see if I got it right.

EDIT: LOL, you guys might not believe it but I didn't really read the last post completelly (i didn't mention it) so my guess for the HE-AAC was right HaHa!

EDIT2: Well seems like I failed the Flac and 256Kpbs, lol... it makes sense though... Flac always preserves low frequency better than mp3 so the slightly muffled impression I got from Sample_3 is from the song itself. Sample_1 always sounded clearer and crisper probably because of the more recessed bass... not sure why the 256Kpbs sounds less warm than Flac and 128Kpbs, though... are these songs from the same source? Strange that nobody could tell HE-AAC... it was pretty clear to me...

I see that some of the takers also found the same differences between tracks I did.

Interesting test but this is not conclusive enough to say that Flac is not worth it.

Actually some of the impressions here show that Flac does make a difference.

It would be important to know if all this tracks are from the same source and for one to know the song really well to reach the right answer.

Interesting song by the way.

Is now the right time to point out that this is not an ABX test? 

ABX does not make the listener guess what he is listening to, only prove (or fail to prove) that he can hear a difference. 

This is a 4-way guess. 

I don't have the golden ear, but this is what I get.

Using my HP Pavilion dv3000 -> Creative X-Fi 5.1 USB -> Bravo Audio V2 -> stock T50RP

1 -> 256

2 -> acc

3 -> flac

4 -> 128

Seriously hard because not instrumental song...

But there wasn't THAT much difference though from them all. But maybe next time make an instrumental or vocal test

Oh, and can you teach me how you make those files? I want to try on my other tracks too

*add : wow didn't know i was right, and i was only doing it without really listening it, lastly just picked randomly LOL!

Thanks for the info. Here's an idea. Why don't you make some more tests. Especially in different categories like in bass, mids, highs, accuracy, etc. Not only this will show that if different file format affect the sound but also can train the ears.

on my ie8s through my laptop i could tell that 3 was flac and 2 was 64 but the other two sounded the same to me.

Sample 3 clearly sounds better to me, but I can't really see why?

Anyway, there are more vivid and important differences in sound quality than bitrate and codec, such as DAC and amplifier.  I am happy without lossless.

Originally Posted by nikongod 

Is now the right time to point out that this is not an ABX test? 

 

ABX does not make the listener guess what he is listening to, only prove (or fail to prove) that he can hear a difference. 

 

This is a 4-way guess. 

So?  ABX can have it's flaws too, like an illusion of sameness via rapid switching, I actually like this format, ABX is overrated.

I didn't listen to the whole song but the bells at the start sound more natural in Sample 3.

Thanks I will put sample 2 and sample 3 on my MP3 / WAV player to test people when they act high and mighty about l0ssl3ss haha.

Quote:

Originally Posted by kiteki 

So?  ABX can have it's flaws too, like an illusion of sameness via rapid switching, I actually like this format, ABX is overrated.

 

There is no requirement for rapid switching in an ABX test. 

The listener can listen to A for weeks if he chooses, followed by B for a few more weeks, and finally "X" for about a year if that makes him comfortable. 

There is a time limit for reasons of practicality, but since we are doing this in our homes one could easily spend an hour or 2 on one ABx pair. so much for speed. 

The problem with a multi-way-guessing test is that WHEN people guess the identity of whatever is being tested incorrectly despite being able to hear obvious differences nay-sayers WILL cite this error as proof that (whatever low sample rate or crappy compression scheme they think is good enough) is adequate. Its not. 

A multi-way guessing test also requires previous knowledge of the characteristics of the stuff under test. Several posters above cite previous knowldege as teh only thing that got them the correct ID, despite being able to hear obvious differences. 

ABX proves that things are different or the same with no previous knowledge of the "stuff" under test. Where 99.995% of the people on the street (a truly random test) would fail this guessing test, I'd wager that a significant percentage would pass an ABx test between 2 randomly selected of your 4 samples. 

Multi-way-guess tests fail for far too many reasons. 

I think most people on the street would fail this in an ABX too though.

When you find someone on the street which guesses correctly, you need to take them to the side and keep testing them, then you have evidence.  In a lot of the AES papers they compile everything into a single result, which makes no sense.

It's like proving no one can break the system at a casino, by looking at the statistics and saying "hey everyone is losing money".

Quote:

Originally Posted by kiteki 

I think most people on the street would fail this in an ABX too though.

 

You just need to pitch it correctly:

Tell them you are trying to test their hearing ability for statistical purposes. Do that without a single mention of audiophilia or anything to do with sound recording/reproduction or anything of that sort. You MUST pitch the test up so that the test-taker thinks that they are whats being tested, not the compression scheme in this case. I'd bet you will get very strong data pointing towards people being able to correctly identify X. At least 80% success should not be hard. 

And in all of that you are telling the truth - you are ultimately testing whether they can hear something or not for statistical purposes. Human nature forcing all of us to try to outdo other people will get you really strong results, even from a large sample group. 

If you are not so constrained by morality (I have no reason to doubt your morality, just saying) you could always pitch it like this:

I'm testing your hearing to see if you qualify to take another test at a later date. Only the people with the most accurate and discerning hearing will be selected. Im going to play 2 pieces of music, and then one of them again. Mark your scan-tron card with whether I replayed the first (A) or second (B).... Again, no mention of audiophile BS that nobody cares about. If you have some money behind this project, pay them $10 for the first test. If you have no money behind the project say that there is $30 for people who get selected for the second test. 

If you pitch it as a test of some new fangled stereo gear or recording technique you would get less than 25% success, if your even that lucky. When the test taker doesn't care about the test they WILL fail it. 

The key to success is asking the right questions. Or in this case pitching your test correctly. 

This does not require finding (selecting) a "good" person to test (which is BS, I agree), it simply requires that you pitch your test correctly to the average person. Ooh, it also requires that the person have an hour or so to take it.

Once you have that person committed for an hour its very easy to run an ABx (x=a/b), Adx (x=A/d), etc test. mix it up a bit. you could easily run 18 tests (all possible combinations 3 times) in an hour, with an infinite number of listeners plugged into headphones...