Head-Fi.org › Forums › Equipment Forums › Sound Science › Double blind test 128Kbps vs lossless? I'll be amazed if you can tell much difference
New Posts  All Forums:Forum Nav:

Double blind test 128Kbps vs lossless? I'll be amazed if you can tell much difference - Page 5

post #61 of 257

Lol how am I supposed to find out if track 2 was lossless or not, thus I would be correct. PM me which track was lossy and which was lossless? lol!

post #62 of 257

This sample is fine for a listening test.  There is a difference in how the two files sound.  With this sample it is more about listening to the overall sound rather than zeroing in on a specific artifact on a cymbal hit.  This type of sample is actually more relevant to normal listening than the type of listening tests that encourage you to zero in on a single artifact around something like a cymbal hit.

 

What I catch is that the MP3 version sounds like it is recorded in a dead room.  Less room effect and echo and reverberation in the high frequencies.  The lossless version sound more like it is in a real room with real room acoustics.

 

I started doing a ABX test in Foobar.  That way I wouldn't know which I was listening to and the test would be blind.  I didn't do enough tests though to actually be significant.  That many tests is a lot of work on the ears.  Each test involves listening to the sample about six or so times and I'd listen to 30 to 60 seconds worth of music each time.  Listen to A, B, X, Y, and a few additional to confirm.  After that my ears are tired and I need a break.  So getting a full statistically valid test would take some time and lots of ear freshening.  ABX tests are not easy listening.

 

One interesting thing I did find though is that on the tests where I heard the MP3 version first I had a harder time.  It was like the MP3 version was deadening my ears.  I'd switch to listening to the other file and wonder if it was actually better or worse than the first.  If I heard the lossless version first the difference seemed easier to immediately pick up.  Not anything statistically significant in that observation.  Just an observation.

 

I was listening using my SR325is.  The Grado emphasizes the highs which makes the higher frequency room reverberations easier to hear.

 

I should try to do enough samples to get a statistically valid test and a log file.  I was able to get 4/4 but that took a long time with ear relief breaks and even then by that time my ears were getting dead to the sound difference between the files.

post #63 of 257
Quote:
Originally Posted by nick_charles View Post





There is nothing stopping someone from creating their own test tracks of beloved music. You rip a CD track to wav encode a copy of it as mp3 then decode that copy back to wav you then have two wav files the same size which you can compare.

I absolutely agree, which is why I have done so in the past. However, such was not the case in this particular test.
 


Edited by Spelaeus - 7/6/10 at 4:54pm
post #64 of 257

Denon D7000 with Isone pro Crossfeed - the rest as per my signature.

 

Sample two has a lot of missing high frequency energy - apparent immediately and with the first switchover.  Not only is the HF hacked off, overall tonality is a shade lower and the the strings have an etched quality to it - as in not as smooth or full bodied (analogue sounding) as the first sample.

 

I suspected with the first switchover and confirmed it with the second.  Sample 2 is the MP3* and is not debatable as far as I'm concerned.

 

 

* edited - I stated lossless when I meant MP3


Edited by SP Wild - 7/6/10 at 9:03pm
post #65 of 257

I'm sorry guys, I must change my mind!!!

 

My initial decision was made in 3 rotations - and I didn't even finish the tracks.  Doing that way was so flawed as the moment my ears adjusted to the busy sections and it flips back to the begining, the initial lower tone strings at the start fooled me immediatey hence my above remarks.  Which were in fact true, when comparing a busy section to the beginning.

 

I had an extended listening, this time with the monitor off and a remote in hand - where I'd just keep flipping tracks - so in effect I had no Idea which track was playing.  When I did that all the differences dissappeared.  I honestly had no Idea - thats when I knew damn well I had fallen victim to placebo.

post #66 of 257

I've spent nearly two freakin hours on this test, blind.  Everytime I think a pattern is building up in that I am getting it with two in row matching guesses...the next guess would be a different track.  No matter how hard I concentrated the results were no more accurate than a coin toss.

 

 

Now to try it with the K701s with no DSPs.  If they don't pick it up because they are a treble emphasis can, and I keep thinking the difference is in the upper mid overtones, I will wave the white flag.  I just wish there are transients in this track as mid / high frequency transients is what I rely on with these sort of challenges.

post #67 of 257
Quote:
Originally Posted by SP Wild View Post

I'm sorry guys, I must change my mind!!!


Yep, that's normal. I finally took the test since I downloaded the file to rehost it anyway. Started off 1 for 3, then swore I heard a difference at about 11 seconds that clued me in. Extra space and more impact from the notes. Went to 4 for 6 like that, then got it wrong. So I found another spot somewhere near the end and went to 7 for 10, then got two wrong in a row again before I gave up. I expected any differences to be subtle because I've gotten almost exactly 50% right in V0 vs. lossless tests of my own. This is a good 128kbps, that's for sure. Must be a new encoder, my old files sound noticeably worse.

 

Two hours, though 

 

Actually, I gave myself an idea. Maybe I should track down an old encoder version, and encode one of my files with both that and the newest LAME at 128kbps. It's pretty well established that encoders have improved considerably, but I don't know if there's been a direct comparison here.

post #68 of 257

I definitely have to say I don't hear the night and day differences re: full body etc that some have suggested and perhaps I'm getting on with age and unable to detect certain sounds, who knows? But yeah, I did the test having some of the results so I feel that I ended up going into it a tad biased and like SP Wild, every time I thought I got it, I lost it and then I thought to myself, why do I even bother. I really don't like having to analyse music. To me it's either I like it or I don't or I notice that it is poorly produced. That's my take. I might take the test another day (ensuring that I eliminate any placebo effects) when I don't have a confounding headache though. Like I said in some posts in other threads, I'm not a musician, I'm just here to enjoy my music

 

post #69 of 257


 

Quote:
Originally Posted by Head Injury View Post


Yep, that's normal. I finally took the test since I downloaded the file to rehost it anyway. Started off 1 for 3, then swore I heard a difference at about 11 seconds that clued me in. Extra space and more impact from the notes. Went to 4 for 6 like that, then got it wrong. So I found another spot somewhere near the end and went to 7 for 10, then got two wrong in a row again before I gave up. I expected any differences to be subtle because I've gotten almost exactly 50% right in V0 vs. lossless tests of my own. This is a good 128kbps, that's for sure. Must be a new encoder, my old files sound noticeably worse.

 

Two hours, though 

 

Actually, I gave myself an idea. Maybe I should track down an old encoder version, and encode one of my files with both that and the newest LAME at 128kbps. It's pretty well established that encoders have improved considerably, but I don't know if there's been a direct comparison here.


I think the placebo of having bad experiences with 128 MP3 (late 90s) made me not really take the test seriously in the beginning.  Two hours alright, my ears paid the price...I tried the K701 briefly, but I couldn't stand it anymore...I MIGHT try again with fresh ears, but for now, the white flag is up...and anyone can quote me on this...My opinions need to be taken with a grain of salt because that SP Wild can't even DBT 128 and lossless and claims silver cables makes a difference!

post #70 of 257
Quote:
Originally Posted by cravenz View Post

I definitely have to say I don't hear the night and day differences re: full body etc that some have suggested and perhaps I'm getting on with age and unable to detect certain sounds, who knows? But yeah, I did the test having some of the results so I feel that I ended up going into it a tad biased and like SP Wild, every time I thought I got it, I lost it and then I thought to myself, why do I even bother. I really don't like having to analyse music. To me it's either I like it or I don't or I notice that it is poorly produced. That's my take. I might take the test another day (ensuring that I eliminate any placebo effects) when I don't have a confounding headache though. Like I said in some posts in other threads, I'm not a musician, I'm just here to enjoy my music

 

Most definitely NOT night and day, but if you concentrate hard enough for about 10 seconds of the song then quickly switch to the other bitrate song, you should be able to hear quite a large difference. Of course this is head-fi and like with everything, we exaggerate. The difference actually isn't that big. But in Head-Fi terms, it is 
 

post #71 of 257

Between the two, I hear more in the upper tones in the second. Granted, these particular recordings are pretty close. I don't listen to much classical music, but in recordings like this one, there are simply too many instruments to pick out individuals to begin with, so fine detail is already difficult to discern, regardless of encoding.

 

In a not-so-fair comparison, I have "A Love Supreme" synced and being fed simultaneously to my amp via a 192kbps MP3 via a ND-S1 connected to a DacMagic, and a SA-CD in my NS975V. Between those two, the difference is immense and instantly noticeable, especially on the cymbals. (though the ND-S1 is quieter, so I've had to adjust the volumes, but the clarity difference is still noticeable at all volumes)

 

I've compared lossless songs through the ND-S1 and DacMagic and through the CD player's analog outs, as well as analog outs on the NS975 vs digital coaxial out through the DacMagic and couldn't tell much of a difference (beyond somewhat better instrument separation on the DacMagic), so the DAC on the NS975 is pretty close to the DacMagic in terms of capabilities, given the same source. Then again, I should mention that don't have magic ears.

post #72 of 257
Quote:
Originally Posted by stang View Post



Most definitely NOT night and day, but if you concentrate hard enough for about 10 seconds of the song then quickly switch to the other bitrate song, you should be able to hear quite a large difference. Of course this is head-fi and like with everything, we exaggerate. The difference actually isn't that big. But in Head-Fi terms, it is 
 


thanks for clarifying your stand and about exaggeration which not many people are often aware of re: Head-fi exaggeration. It's just hard to interpret when people here make claims that there is a "huge difference", maybe in Head-fi terms, but I didn't even know we had Head-fi language in the first place

 

It's a pity not everyone realises this and many new Head-fiers walk in thinking that there will be big differences in store when they switch equipment, but really it isn't and I think this is what some threads have picked up. But shan't derail.

 

Anyhow, I'm up in the air about the differences. I'll have another go later tonight or something.

post #73 of 257

Well, it's complicated. We all have different values and different sensitivities. A slight change in bass response might be huge for one person, but not for someone else. And the context varies. Someone saying that the T1 has exaggerated bass (I don't endorse this pov personally, but let's if) and someone saying that the XB700 has exaggerated bass are not talking about anything near the same thing. One reason why objective data is useful. 

post #74 of 257
Quote:
Originally Posted by Ypoknons View Post

Well, it's complicated. We all have different values and different sensitivities. A slight change in bass response might be huge for one person, but not for someone else. And the context varies. Someone saying that the T1 has exaggerated bass (I don't endorse this pov personally, but let's if) and someone saying that the XB700 has exaggerated bass are not talking about anything near the same thing. One reason why objective data is useful. 


agreed, it's all very subjective :)

post #75 of 257

First, I have to say that the encoder for the 128k file is surprisingly good. Things have certainly improved very considerably since the warbly sound we're used to with files compressed that much.  

 

I loaded both tracks into iTunes + Amarra and played them through my Stax rig.  The first file that I heard (I wont say which, as I'll PM the OP instead Edit: See edit below as well) was distinctly compressed, with the instruments lacking the complex harmonics that I usually expect with violins, sounding instead flat and artificial. There are a great many classical recordings that don't capture this well, and this is one of them, as when I heard the other track, I could pick these things, but only because I was paying attention.

 

To be sure, I duplicated the files in the playlist a few times and hit the random button without looking.  I could consistently determine which I was listening to every time, soon within the first few seconds of the track starting.  

 

That all being said, I have been a critical music listener for a couple of decades; I have a high-end rig and I'm familiar with musical instruments, so the combination of all those things made it easier.  Again though, it's remarkable just how good the encoding was if that's what a 128k file sounds like now and I highly doubt I would have picked it as easily out of my iPhone or iPod, for example.

 

Edit: I just ran both files through an audio program to confirm that I was indeed correct, the file I picked as being compressed having a hard cut-off at 16 kHz.

New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Sound Science
Head-Fi.org › Forums › Equipment Forums › Sound Science › Double blind test 128Kbps vs lossless? I'll be amazed if you can tell much difference