[Earwax]

I've tried to ABX Flac vs. compressed on my PC and have failed once the bit rate is high enough (160 - 180) but I have limited patience and get listening fatigue easily trying such tests. I'm still interested in knowing for myself if there IS a difference.

So here's the test I propose. 2 identical Sansa clip players. Each loaded with latest firmware, formated fresh. My favorite album ripped to Flac and then re-encoded using the FHG encoder in dbpoweramp to encode to 320kbps MP3. I copied flacs to one player and mp3s to the other. I did not scrutinize the player in detail looking for small marks or scratches, I've already lost track of which is which. Tags display the same on each player.

I'll carry both players around and after a little while, I'll mark one of them in some way and then try to make a judgement of which sounds "better". I don't have any set timeframe for this comparison, it may take days or weeks depending on what I notice or don't notice.

What do you think?

 

[Ham Sandwich]

Quote:

Originally Posted by Earwax /img/forum/go_quote.gif What do you think?

You've got a 50% chance of being right (or wrong) from that one trial. You can't conclude much scientifically from that.

 

[Earwax]

Quote:

Originally Posted by Ham Sandwich /img/forum/go_quote.gif You've got a 50% chance of being right (or wrong) from that one trial. You can't conclude much scientifically from that.

Maybe I should instead start with a "medium" bit rate. If I can ID the flac correctly, do another trial with a higher bit rate, if I cannot, move to a lower bitrate.

I don't expect to devise a test that would satisfy anyone else, just to demonstrate something to myself.

 

[ILikeMusic]

As mentioned you'll need to do a number of trials to eliminate the chances of just guessing correctly. This is why such tests are usually done in many short intervals in succession but if you have the time you can take as long as you like, however you'll need 15 or 20 tests to get any kind of significant results.

And I think you're probably right about trying some very low bitrates first to get the hang of it... starting with 320 is likely to be very frustrating as it's very unlikely you'll be able to tell the difference. In fact, with a good encoder don't be surprised if you can't tell a difference at a considerably lower bitrate. You may fund the results humbling (DAMHIK) but at least you'll know the truth.

 

[Earwax]

I know I can ABX CBR128 from Flac on the PC. Anything better is questionable, but I also think the Sansa is a cleaner source than the PC.

I may just start with lame -V3 and see how it goes.

 

[ILikeMusic]

Quote:

Originally Posted by Earwax /img/forum/go_quote.gif II may just start with lame -V3 and see how it goes.

Sounds good. I think you may find even that very challenging in a true 'blind' environment.

 

[CodeToad]

If I had to guess, I would say more than likely you will "train" yourself to hear the difference in this experiment. That's probably bad for your wallet going forward. Ignorance may very well be bliss.

 

[Peyotero]

Quote:

Originally Posted by Earwax /img/forum/go_quote.gif I've tried to ABX Flac vs. compressed on my PC and have failed once the bit rate is high enough (160 - 180) but I have limited patience and get listening fatigue easily trying such tests. I'm still interested in knowing for myself if there IS a difference. So here's the test I propose. 2 identical Sansa clip players. Each loaded with latest firmware, formated fresh. My favorite album ripped to Flac and then re-encoded using the FHG encoder in dbpoweramp to encode to 320kbps MP3. I copied flacs to one player and mp3s to the other. I did not scrutinize the player in detail looking for small marks or scratches, I've already lost track of which is which. Tags display the same on each player. I'll carry both players around and after a little while, I'll mark one of them in some way and then try to make a judgement of which sounds "better". I don't have any set timeframe for this comparison, it may take days or weeks depending on what I notice or don't notice. What do you think?

1. The question is not weather there is a difference (because there is for a fact) but rather is the difference is audible to you, and if so, is it worth your time?

2. You'll have a real hard time to notice anything at all out of a portable while on the move. Usually with such ABX tests one has to be real focused. Sometimes to a point where your brain starts aching and all you wanna do is to go sleep after the tests

. heh yeah some brains get stressed easily.

 

[Ham Sandwich]

Quote:

Originally Posted by Earwax /img/forum/go_quote.gif Maybe I should instead start with a "medium" bit rate. If I can ID the flac correctly, do another trial with a higher bit rate, if I cannot, move to a lower bitrate. I don't expect to devise a test that would satisfy anyone else, just to demonstrate something to myself.

I found some problem samples and used those to get an idea of what MP3 artifacts like pre-echo sound like. I used lower bit rates around 128 to make it easier. I searched HydrogenAudio and other sites for the samples. krmathis recently posted links to several test samples. At higher bit rates (-V1 or -V0) I could no longer hear the artifacts that I was able to hear at the lower bit rate.

Of course there is also more to the listening experience than listening to short problem samples in an ABX test. When you listen to an entire song you might pick up on some things that you might not catch in short test samples because in ABX tests you are listening more critically for specific things rather than listening to the whole picture.

 

[krmathis]

I don't think its fair.
Ok, it depends what you want to achieve by running the test... Cause:
* As mentioned it will only give you 50% chance of being right/wrong (basically if you guessing or not). Unless you reset and perform the test multiple (ideally 10-20 times) that is.
* A portable rig is not ideal for hearing small nuances. Cause as you raise the bitrate of the lossy files there will be harder and harder to hear audible differences.

It does not hurt to try, but don't conclude from a single run only.

 

[Earwax]

Quote:

Originally Posted by krmathis /img/forum/go_quote.gif I don't think its fair. Ok, it depends what you want to achieve by running the test... Cause: * As mentioned it will only give you 50% chance of being right/wrong (basically if you guessing or not). Unless you reset and perform the test multiple (ideally 10-20 times) that is. * A portable rig is not ideal for hearing small nuances. Cause as you raise the bitrate of the lossy files there will be harder and harder to hear audible differences. It does not hurt to try, but don't conclude from a single run only.

I'm not listening critically "on the go". I am using the Clip at home with different headphones, so far, without an amp. KSC-75, Shure SCL4, Sony CD780. And with my "new" (partially rebuilt) vintage speakers. I have no better source that can play compressed files, the Clip is no slouch.

The album I am using is Innovators by Sam Cardon I love it, one of the few things I can listed to over and over and over. And it has enough complexity and diversity to be good test material.

The results so far have surprised me. Lame -V3 was very easy to distinguish. The bass passages were congested and muddy. So much so that I suspected a faulty player and swapped the two players and switched which one had the mp3 and which the flacs. No difference. I did not do enough trials to satisfy any scientific critera, but moved on to -V2.

-V2 is better, but also distinguishable. The difference is in the 3D imaging, instrument separation and soundstage. 2 for 2 on identification and I'm now moving on to -V1.

 

[ILikeMusic]

Quote:

I did not do enough trials to satisfy any scientific critera,

I guess I'm missing something but why go through the effort to try to set up a proper objective test (as in your original post) and then invalidate the results by just doing a single run and moving on?

.

 

[mbd2884]

If you guess correctly it will be purely by chance.

There will always be golden ears parading around claiming to hear a difference

Reality, your ears as a human being aren't good enough to hear the difference. Maybe if we were a different species of animal that are dependent on their hearing more than their eye sight, then we would notice a difference.

 

[krmathis]

Quote:

Originally Posted by ILikeMusic /img/forum/go_quote.gif I guess I'm missing something but why go through the effort to try to set up a proper objective test (as in your original post) and then invalidate the results by just doing a single run and moving on? .

Exactly!
One need to run the blind test a certain number of times, to eliminate placebo and guessing.

To each their own of course!

 

[Earwax]

Quote:

Originally Posted by ILikeMusic /img/forum/go_quote.gif I guess I'm missing something but why go through the effort to try to set up a proper objective test (as in your original post) and then invalidate the results by just doing a single run and moving on? .

1. I'm doing this primarily for my own knowledge.

2. The -V3 was so obvious it wasn't worth spending any more time on, it's not an interesting test case. If I can distinguish higher bit rates, why waste time on the lower ones?

Even though I said it's easy to distinguish, it's not instananeous, it takes listening back and forth over several tracks.