Would this be a fair test? MP3 vs. Flac
May 14, 2009 at 11:23 PM Post #16 of 51
Quote:

Originally Posted by mbd2884 /img/forum/go_quote.gif
If you guess correctly it will be purely by chance.

There will always be golden ears parading around claiming to hear a difference

Reality, your ears as a human being aren't good enough to hear the difference. Maybe if we were a different species of animal that are dependent on their hearing more than their eye sight, then we would notice a difference.




Please be more specific. Aren't good enough to hear what difference? what bit rate and encoding?
 
May 14, 2009 at 11:31 PM Post #17 of 51
Quote:

Originally Posted by krmathis /img/forum/go_quote.gif
Exactly!
One need to run the blind test a certain number of times, to eliminate placebo and guessing.

To each their own of course!
tongue_smile.gif




What placebo? My prior expectation was that identifing -v2 versus flac would be impossible.

Could I have been guesssing? Well, I know I wasn't. I also know I wasn't using any physical cue to determine which player was which, but who's going to believe me on that point? I really don't expect to affect any "true believer's" thinking in either direction. When I get to a more interesting/challenging comparison maybe I'll do more trials. Or maybe not. I also want to test ogg vorbis eventually...
 
May 14, 2009 at 11:39 PM Post #18 of 51
Well, if not placebo then guessing. If -V3 or -V2 are really so obvious then it should take little time to do a couple of comparisons to cinch it. But then again if if -V3 or -V2 are really so obvious then that's very unusual... you are a helluvala lot better than most...
 
May 16, 2009 at 2:36 PM Post #19 of 51
Quote:

Originally Posted by ILikeMusic /img/forum/go_quote.gif
Well, if not placebo then guessing. If -V3 or -V2 are really so obvious then it should take little time to do a couple of comparisons to cinch it. But then again if if -V3 or -V2 are really so obvious then that's very unusual... you are a helluvala lot better than most...


Just to be clear, I did 2 trials at -v3, 2 at -v2. Now, so far, 1 at -v1.
Each round has taken significant time. If I were to do a quick ABX of a 15 second clip I don't think I could tell the difference. I've been listening to several tracks at a time, each track about 4 minutes long, then switching. I'm not finding artifacts in anything better than v3, it's the soundstage and instrument separation that's different and that takes time (for me) to pick up on the nuances.

Each step up in bit rate is definately getting harder to hear differences. -v3 was disappointing, but -V2 is very good encoding, it's just not perfect. -v1 is even better.

I don't think my hearing is all that good, certainly my high frequency (>16000) hearing has gone. Some of it is equipment, you can agrue all day long about what's good and what isn't, but it seems undeniable to me that today's common gear is at levels than would have been considered "audiophile" 5 to 10 years ago.
 
May 16, 2009 at 4:27 PM Post #20 of 51
Quote:

Originally Posted by Earwax /img/forum/go_quote.gif
... but it seems undeniable to me that today's common gear is at levels than would have been considered "audiophile" 5 to 10 years ago.


No doubt there, most would cringe if they had to go back to what they were using ten years ago, even if it was state-of-the-art at the time. But then again encoder technology has been improving right along with the hardware over the years.

Interesting that you seem to be detecting a 'knee' in the SQ curve as you go below -V2, and and flattening out above. This is also what the LAME wiki would seem to imply. Probably the foundation of the common recommendation that -V2 represents the best compromise between quality and filesize. Let us know what you hear at -V0.

.
 
May 18, 2009 at 4:23 PM Post #21 of 51
I disagree. The best speaker I found that I could afford is a vintage about 25 years old. And the Kenwood Amp is quite good also. I have no problems using it with my speakers, sounds great. I'd like to upgrade sometime, but right now I hear no reasons why I need to.

I think it's hilarious how people think Audio technology has progressed. It hasn't. It's been pretty stagnant for years now. What DVD audio is an evolution? Nope, just another gimmick to sell to people who want to spend more money on something, doesn't sound any better.

I would agree though that Lame MP3 has come a long way since MP3 first came out. But that's about it. For Audio equipment, using gear even 50 years ago would be just fine (Speakers/Amplifiers). And one of the current loved D/A the PCM1704UK is quite dated in comparison to the latest, but it's still coveted as one of the best. I believe the PCM1704UK was first released over a decade ago and it's still used in the best DACs available. New is not necessarily better.

Even from the recording studio, technology has not progressed at all to where the music is better. The best Beethoven recordings I own are the Karajan series from 1967. That's over 40 years ago and it's better than the letter Barenboim 2005 recordings. I swear it is, much better. And even for Rock, the early recordings of Clapton are just fantastic in comparison to the latest rock albums I have.

Headphone Amps? Headphones and the Headphone Amplifiers I think are incredible overpriced. For just simplistic equipment so expensive. I'm willing to spend the money because that's the only way to get the high quality equipment.

C'mon I love my Compass. But when I look inside and read about the design, it's not anything mind blowing or innovative.

Tube Amps? Tubes were invented in 1906 I believe. The circuit design used today are still based on designs from the 1950s. Point to Point wiring? Ancient? What is new about Tube Amplifiers? Nothing at all, just more gimmick talk to get people to spend more money. Tube amp 10 years ago, 20 years ago, who cares. As long as it used a solid design and good workmanship, it should be just as good as anything made today.

Audio Cables? Yeah wire technology has improved with the UPOCC method. But does it matter when people can't hear the difference between a coat hanger and a Monster cable? Not really, just more gimmicks. Granted I do love my UPOCC headphone cable and my aftermarket power cables. But I don't think they make or break the system in any way, just something nice to have.

I have a relative who spent the most money could buy from a Bang & Olufsen system for her house, it's about 3 years old now. It was the very best they had to offer at the time. I've spent plenty of time now listening to her system. I much prefer my cheap vintage JBL and Kenwood system over hers, the difference is immense. I just love it so much more. There you go, the best money can buy with the latest innovations and technology sounds like dog **** compared to the 25 year old JBL system.

Audio technology has not progressed in 10 years, not in 20 years, hell not almost in 50 years.

Quote:

Originally Posted by ILikeMusic /img/forum/go_quote.gif
No doubt there, most would cringe if they had to go back to what they were using ten years ago, even if it was state-of-the-art at the time. But then again encoder technology has been improving right along with the hardware over the years.

Interesting that you seem to be detecting a 'knee' in the SQ curve as you go below -V2, and and flattening out above. This is also what the LAME wiki would seem to imply. Probably the foundation of the common recommendation that -V2 represents the best compromise between quality and filesize. Let us know what you hear at -V0.
.



 
May 18, 2009 at 4:42 PM Post #22 of 51
I don't necessarily disagree with you and the improvements I was referring to was primarily in the source... it's hard to argue that the 20-22 kHz +/- 0.1 dB, 90+ dB SNR, .001% THD and zero wow/flutter of today isn't vastly superior to gramophone or consumer magnetic tape technology (although no doubt someone will argue it.
biggrin.gif
) And to a lesser extent transducers have improved... not in all cases of course, but I none of the headphones I used in decades past compare to what's available today, and the IEM technology of years gone by (such as it was) isn't even close. Amps... well, not much improvement there because those were pretty damn good 30 years ago.

But I wasn't trying to say that new is always better, and I fully agree with your comments related to the art of the recording process itself...that aspect hasn't necessarily changed for the better over the years. But that aspect was not what I was talking about.

.
 
May 18, 2009 at 4:47 PM Post #23 of 51
IEM technology is ancient and was developed for use by the military over 50 years ago.

Nothing in audio I see is new or innnovative. CDs were introduced in the late 70s, early 80s. Look at some of the most praised companies here at Head-Fi. Grado and Shure for example. Neither company is capable of research and development like what it takes in most other "electronics" industry to compete today. They build old technology with no improvements. The staple of their audio sales for decades were phonograph cartridges. I mean c'mon....

Does it matter if the new IEM's are capable of higher frequencies than before? Nope, our ears can't hear those frequencies. Plus the most profitable use for IEM technology right now is in hearing aids. for people who can't hear. Quite ironic, IMO.
 
May 19, 2009 at 4:48 AM Post #24 of 51
I think this is an excellent experiment, but it needs some more statistical power to be conclusive. Here's what I would do if I had 2 players:

1. (Get somebody else to do this step) Load the music into the players. Since you seem to be able to tell V3 easily, I suggest using V2 VBR as the lossy format to start with. Wrap the players in socks and shuffle them in a closed box. Label one of the players A and the other one B. Note down which player has the lossless.
2. Listen to music on them for 2 weeks. At the end of the 2 weeks, write down which player sounds better. If you more often instinctively reached for one player over the other, write that down.
3. Repeat from step #2.

Over a few months you should get a reliable set of data.
 
May 19, 2009 at 2:53 PM Post #25 of 51
If you can encode at V0, why not test with V0 and solve the issue if you need FLAC or just good V0?
 
May 19, 2009 at 4:08 PM Post #26 of 51
When I rip a CD, I encode to MP3 using lame 3.97 (final) with settings -V0 --vbr-new -q0 -ms -b32 --lowpass 20.5, and to FLAC using FLAC 1.13 at level 5. Using my PC I can very rarely notice a difference. However, when I use my Cowon D2, I regularly notice a difference. FLAC files with the D2 sound 'fuller' and has marginally better detail. Does anyone else experience this with their Cowon daps?
 
May 25, 2009 at 12:04 AM Post #27 of 51
Quote:

Originally Posted by mbd2884 /img/forum/go_quote.gif
If you can encode at V0, why not test with V0 and solve the issue if you need FLAC or just good V0?


So far, I can't tell -v1 from Flac. I got frustrated last week with trying and put the whole thing aside. Came back to it yesterday and, at least with the headphones I've been using up to now, I can't tell lame vbr -v1 and flac apart.
 
May 26, 2009 at 11:47 AM Post #29 of 51
Quote:

Originally Posted by Earwax /img/forum/go_quote.gif
So far, I can't tell -v1 from Flac. I got frustrated last week with trying and put the whole thing aside. Came back to it yesterday and, at least with the headphones I've been using up to now, I can't tell lame vbr -v1 and flac apart.


I gave up at -V2 myself. Good on you for testing, now you really know where transparency is for you. After doing this test what do you think of all the claims of people being able to easily resolve 320 kbps. and lossless ('I don't need to test, it's obvious')?
wink.gif
 
May 26, 2009 at 8:18 PM Post #30 of 51
Considering that HDD space is so cheap these days, the added flexibility of having something in lossless (transcoding without using the CD), I vouch for FLAC when it comes to compression. It does take up considerably more space, but I have an archive of all my CDs and I don't really have to worry about SQ issues. Given space constraints like on a DAP, I'd have to hear this myself.

Pretty interesting thread though, and some nice test feedback.

~Thomas
 

Users who are viewing this thread

Back
Top