Head-Fi.org › Forums › Equipment Forums › Sound Science › My Lossy vs Lossless ABX Experiment
New Posts  All Forums:Forum Nav:

My Lossy vs Lossless ABX Experiment - Page 3

post #31 of 42
Vel the only way any compression format works well is in a VBR mode. CBR is only appropriate for restricted bandwidth streaming etc, you are not doing anything like a fair test. Your point re 320 is N/A because you weren't using LAME.

You're comparing lossless to a compromised version of MP3, not what MP3 can (and does for any knowledgeable user) offer.

The iTunes encoder is not as good as LAME. Go debate it on HydrogenAudio if you need convincing.
post #32 of 42
Thread Starter 
Quote:
Originally Posted by badmonkey View Post
Vel the only way any compression format works well is in a VBR mode. CBR is only appropriate for restricted bandwidth streaming etc, you are not doing anything like a fair test. Your point re 320 is N/A because you weren't using LAME.

You're comparing lossless to a compromised version of MP3, not what MP3 can (and does for any knowledgeable user) offer.

The iTunes encoder is not as good as LAME. Go debate it on HydrogenAudio if you need convincing.
Well, as I said, I don't know much about VBR so can't discuss it in depth. From what I gather it is set to a specific bit rate that is the average and varies around that. How much it varies and by what method I don't know. What I do know is that a lot of mp3's are encoded with a set bit rate. We do not discuss where music comes from here so I won't, but I'm sure you understand what I mean.

Are you saying that if I redid this test with LAME encoded MP3's I would not tell a single difference at all at 320 (to a statistically significant degree of course)?

I'll try to explain the point of this experiment for me. Ultimately, I set out with the knowledge that no matter my results I would still use and upgrade my music to ALAC. My music library consists of music from various sources, at various bit rates and encoded using various encoders. When listening to one of these songs compared to a lossless version I ripped, I usually get the feeling the lossless version is better. I was simply exploring is there any proof to this feeling (taking into account the lossy version was probably not encoded in optimal conditions). In this way, I wasn't really trying to say 'yes, that's it, mp3 beats lossless always in every way possible, end of discussion', I was just doing a personal preference test, as it were.

I do take your point about LAME though. If it really is indistinguishable, that's great. That means it provides a good alternative to lossless and saves space. Personally, I have plenty of space and realize there are other benefits to lossless besides quality so will not be using lossy when upgrading. That said, I will probably never upgrade my 320 kbps music as the difference is hardly noticeable as it is, I mainly need to focus on upgrading 128 kbps

If you tell me 320 kbps LAME is indistinguishable, I will find a way to encode the two samples I could tell at 320 kbps and go into windows and repeat the test for them just at 320 kbps. If I find out they ARE distinguishable, I will be quite pissed off at you though
post #33 of 42
Quote:
Originally Posted by Vel View Post
Well, as I said, I don't know much about VBR so can't discuss it in depth. From what I gather it is set to a specific bit rate that is the average and varies around that. How much it varies and by what method I don't know. What I do know is that a lot of mp3's are encoded with a set bit rate. We do not discuss where music comes from here so I won't, but I'm sure you understand what I mean.
If you're getting CBR mp3s from "other" places, why does it matter whether you can tell the difference or not. Not like you have the CD to do a lossless archive of.

On the other hand, with the actual source, you can decide whether to use VBR mp3 or FLAC/ALAC, which is the more important matter. I use both, FLAC for archives, lame V0 VBR for portables. Storage is dirt cheap nowadays. I actually have all my music saved in 3 different hard drives.
post #34 of 42
Thread Starter 
Quote:
Originally Posted by skitlets View Post
If you're getting CBR mp3s from "other" places, why does it matter whether you can tell the difference or not. Not like you have the CD to do a lossless archive of.

On the other hand, with the actual source, you can decide whether to use VBR mp3 or FLAC/ALAC, which is the more important matter. I use both, FLAC for archives, lame V0 VBR for portables. Storage is dirt cheap nowadays. I actually have all my music saved in 3 different hard drives.
Well, I do have some CD's to rip and I also have choice as to the quality of my music so hence the test. I'm talking about actually having a full library (iTunes in my case) of lossless audio. I have my iTunes folder with the music in it, a separate external hard drive to back up that same music in another place and a Time Capsule to backup the entire hard drive of my iMac (which includes the iTunes folder). So I guess I'm the same as you with 3 copies on 3 drives

I do wish iTunes had the same "automatically convert to 128kbps aac" feature for iPhone as it does for my Shuffle though. But really, the only thing I listen to on the go is podcasts so portability doesn't worry me.
post #35 of 42
Quote:
Originally Posted by Vel View Post
Well, as I said, I don't know much about VBR so can't discuss it in depth. From what I gather it is set to a specific bit rate that is the average and varies around that. How much it varies and by what method I don't know. What I do know is that a lot of mp3's are encoded with a set bit rate. We do not discuss where music comes from here so I won't, but I'm sure you understand what I mean.

Are you saying that if I redid this test with LAME encoded MP3's I would not tell a single difference at all at 320 (to a statistically significant degree of course)?

I'll try to explain the point of this experiment for me. Ultimately, I set out with the knowledge that no matter my results I would still use and upgrade my music to ALAC. My music library consists of music from various sources, at various bit rates and encoded using various encoders. When listening to one of these songs compared to a lossless version I ripped, I usually get the feeling the lossless version is better. I was simply exploring is there any proof to this feeling (taking into account the lossy version was probably not encoded in optimal conditions). In this way, I wasn't really trying to say 'yes, that's it, mp3 beats lossless always in every way possible, end of discussion', I was just doing a personal preference test, as it were.

I do take your point about LAME though. If it really is indistinguishable, that's great. That means it provides a good alternative to lossless and saves space. Personally, I have plenty of space and realize there are other benefits to lossless besides quality so will not be using lossy when upgrading. That said, I will probably never upgrade my 320 kbps music as the difference is hardly noticeable as it is, I mainly need to focus on upgrading 128 kbps

If you tell me 320 kbps LAME is indistinguishable, I will find a way to encode the two samples I could tell at 320 kbps and go into windows and repeat the test for them just at 320 kbps. If I find out they ARE distinguishable, I will be quite pissed off at you though
LAME is transparent at -v4 or anything better for most people, whatever their claims. For your problem samples and yourself, you may well find that it is not, in which case that is useful information and I would suggest you post your test on the Hydrogen Audio listening tests forum.

The point is to "compare lossless with MP3", i.e. to give MP3 the best chance, you should be using LAME -v0, otherwise the test as you set it out is not valid: it's only a test of iTunes, which has a reputation for being a crappy encoder.

-v0 or any ABR mode cannot, as I think you understand, average 320; only peak at it if the codec thinks it's needed. For this reason 320 CBR is without exception a waste of space.

Get the latest version of LAME, use Foobar to encode, it will only take you a little while to do the ABX!
post #36 of 42
Sigh again.
post #37 of 42
Thread Starter 
Quote:
Originally Posted by MrOutside View Post
Sigh again.
You ok there?
post #38 of 42
Thread Starter 
Quote:
Originally Posted by badmonkey View Post
LAME is transparent at -v4 or anything better for most people, whatever their claims. For your problem samples and yourself, you may well find that it is not, in which case that is useful information and I would suggest you post your test on the Hydrogen Audio listening tests forum.

The point is to "compare lossless with MP3", i.e. to give MP3 the best chance, you should be using LAME -v0, otherwise the test as you set it out is not valid: it's only a test of iTunes, which has a reputation for being a crappy encoder.

-v0 or any ABR mode cannot, as I think you understand, average 320; only peak at it if the codec thinks it's needed. For this reason 320 CBR is without exception a waste of space.

Get the latest version of LAME, use Foobar to encode, it will only take you a little while to do the ABX!
Alright, I will do the ABX with LAME some time.

I understand what you mean with regard to it not being the best test of MP3. That was not really what I was trying to do as such. It wasn't something where I was deciding if I should store music in 320 or lossless, I had already made up my mind beforehand. I was seeing if I could make out differences using a "real world" encoder (i.e. one that people would likely use for ripping mp3's that I may have). It was more informal in that way than a strictly 'this is the best I can get' encoder. Also as a note, I don't have much time at the moment, and I already did quite a bit of research as it was, so I didn't really think about the encoder as being a factor to be honest. I suppose if I was to do a stricter test I would have to research all the MP3 encoders out there and figure out which is best at which bit rate, etc... I simply don't have the time.

Yep, I understand that averaging at 320 is impossible for a VBR. What I meant is that I don't know if having a, say, 128 kbps VBR means it will go all the way up to 320? Perhaps you can enlighten me? If that was the case, wouldn't it have to go down equally as much in order to be average at 128? Or am I getting something wrong? I have added this topic to a list of possible things I can research and blog about so thanks for bringing that to my attention.

With regard to your previous comment that it's not a fair test without VBR... Could you please explain a bit more? As I see it, I was trying to discern between various constant bit rates to see at which exact (well, sort of exact...) point I stopped telling the difference. If I had used VBR 96 128 192 and they all encoded tough parts at 320 (not sure if it works like this), then what's the point? Wouldn't it just be confusing and undermine the point of the trial?
post #39 of 42
If it's not VBR then you are artificially restricting the encoder from doing its job on the tough bits, and forcing it to waste bits on easy sections, but the resulting file size will be the same. LAME for example is developed and tuned around its -vX settings, nobody cares about CBR. Ultimately 95% of real world applications care only about the total file size, it's only narrow bandwidth streaming that CBR matters for.

The typical bitrate ranges found in LAME encodes is on the HA wiki entry:
LAME - Hydrogenaudio Knowledgebase
However those are just averages. Yes it will peak at 320, just less often for lower quality settings.

The consensus is that -v4 or better is transparent on the majority of music. -v0 is transparent on almost all music, with the exception of particular problem samples which represent a tiny tiny tiny fraction of all the music out there.

I'm not defending MP3. In fact I use AAC now for compressed audio, my portable devices both support it, and my impression is that it's better at lower bitrates. I'd be interested to see you do some AAC vs MP3 testing! ABX test an "average" piece of music and determine which codec goes to the smallest file-size before losing transparency...?
post #40 of 42
With the price of HDD these days, I wonder why this is even an issue/debate. I'd rather spend $70 for 1000gb and be free of any worries from distortion. 90% of time, there may not be an issue. But unless it's 100% of the time, I am sticking with lossless format.
post #41 of 42
People use compressed formats on portable devices with limited storage. Get used to it.
post #42 of 42
These days, even the portables have pretty high storage space. This trend will continue that way, not downward. Get used to it.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Sound Science
Head-Fi.org › Forums › Equipment Forums › Sound Science › My Lossy vs Lossless ABX Experiment