Multiformat 128kbps public Listening Test.. Atrac3=last place.

[austonia]

Results here
http://www.rjamorim.com/test/multifo...8/results.html

further discussion
http://www.hydrogenaudio.org/index.php?showtopic=21904&

"Vorbis aoTuV is tied to Musepack at first place, Lame MP3 is tied to iTunes AAC at second place, WMA Standard is in third place and Atrac3 gets last place."

Sorry, MD guys heh heh

 

[ampgalore]

Have you actually heard the atrac3 format Austonia? Sometimes I really feel that you are a on holy crusade against MD's.

 

[blessingx]

Wow, WMA standard beat something. Nice to see independent tuning continuing on Vorbis and LAME still holding its own.

Quote:

Originally Posted by ampgalore Have you actually heard the atrac3 format Austonia? Sometimes I really feel that you are a on holy crusade against MD's.

Ahhh... it's a double blind test and not Austonias by the way.

 

[Eagle_Driver]

I've found the problem with ATRAC3 performance was due to the Sony OpenMG software-based alogarithm of compressing files. It just plain sucks compared to some other formats.

Also, the top two-performing codecs are supported only by very few portable players (OGG by many of the iRiver players, but I don't know which portable players support MPC). Thus, I would be restricted to playing OGG and MPC files on the PC (computer), since I would lose Redbook AudioCD and MP3 support if I installed the OGG firmware for my iRiver SlimX player.

Furthermore, my newest Sony player also supports ATRAC3 and MP3. But I didn't buy it for the ATRAC3 support, but rather for a cheap CD-based portable that I could bring to and from work everyday, playing LAME MP3s and Redbook CDs.

 

[austonia]

Quote:

Originally Posted by ampgalore Have you actually heard the atrac3 format Austonia? Sometimes I really feel that you are a on holy crusade against MD's.

i am. it's my mission in life. I lie awake at night, planning.. plotting.. new ways to reign havok on you MD zealots... yes, the day of reconning is coming! Muahaha!

Well, not really. But hey, some people still cling to AM radio, records.. and MiniDisc. _shrug_ No sweat off my back, but it's fun to debate.

 

[Big D]

Quote:

Originally Posted by blessingx Wow, WMA standard beat something. Nice to see independent tuning continuing on Vorbis and LAME still holding its own. Ahhh... it's a double blind test and not Austonias by the way.

There's something fishy here. Looking at the confidence interval spreads are identical. The lines should reflect the spread around each mean figure. If some of the results are wider of the mean then the spread line should be longer, yet for all the codecs they are the same.

The chances of that are thousands to one.

Before anyone tries to jump on me I have to know quite a bit about statistics to interpret medical research papers so I'm not grossly incompetent.

Anyone have any suggestions?

EDIT: I've seen the link and they are proper CIs. Maybe because of the nature of the data it is going to be very closely packed around the mean.

 

[DigDub]

lol. if one would care about sound quality, no one would encode at that kind of bitrates. those bitrates are meant for low storage space portable devices which are used in low-powered devices in presumably noisy environments, which isn't exactly 'headphile' conditions. and i doubt one could hear the difference in bitrates under those kind of conditons.

 

[austonia]

Quote:

Originally Posted by DigDub lol. if one would care about sound quality, no one would encode at that kind of bitrates. those bitrates are meant for low storage space portable devices which are used in low-powered devices in presumably noisy environments, which isn't exactly 'headphile' conditions. and i doubt one could hear the difference in bitrates under those kind of conditons.

yeah, very good point. I guess they are just comparing the technologies for what they can with such kbit limitation. I'm not that worried about small file sizes myself. my VBR MP3's average 256kbit, so they are pretty big.

 

[DigDub]

Quote:

Originally Posted by austonia yeah, very good point. I guess they are just comparing the technologies for what they can with such kbit limitation. I'm not that worried about small file sizes myself. my VBR MP3's average 256kbit, so they are pretty big.

yep. and the regular atrac (since ver 4.5) was virtually indistinguishable from cd, which was proven from many blind tests by audiophiles in the past. sony had a chance to make atrac work because it is a very decent codec, but they blew it with all those proprietary and licensing stuff.

 

[austonia]

Quote:

Originally Posted by DigDub but they blew it with all those proprietary and licensing stuff.

absolutely. I'd give it a fair shake myself except for all the limitations they put on it. just ridiculous.

 

[Eagle_Driver]

Quote:

Originally Posted by austonia absolutely. I'd give it a fair shake myself except for all the limitations they put on it. just ridiculous.

Agree here also. I still stand by my statement that Sony blew it with the software compression alogarithm that they used on OpenMG/SonicStage. They could have made it at least as good as the hardware chips used in real-time recording, only to put in all sorts of restrictions and an alogarithm that ultimately butchers the audio that is being compressed.

 

[DigDub]

Quote:

Originally Posted by Eagle_Driver Agree here also. I still stand by my statement that Sony blew it with the software compression alogarithm that they used on OpenMG/SonicStage. They could have made it at least as good as the hardware chips used in real-time recording, only to put in all sorts of restrictions and an alogarithm that ultimately butchers the audio that is being compressed.

yea. the on-chip atrac encoding for regular atrac is superb. excuse me while i go listen to my sharp dr7.

 

[blessingx]

Quote:

Originally Posted by DigDub lol. if one would care about sound quality, no one would encode at that kind of bitrates. those bitrates are meant for low storage space portable devices which are used in low-powered devices in presumably noisy environments, which isn't exactly 'headphile' conditions. and i doubt one could hear the difference in bitrates under those kind of conditons.

The bitrate has been brought up several times, and previously it's been mentioned going higher tends to blur the differences, when most codecs approach transparency. The biggest suspect for all claims/tests are the equipment used is never mentioned. Although the problems should be spread across all the codecs, highs/lows/etc. come into play.

 

[Eagle_Driver]

Quote:

Originally Posted by blessingx The bitrate has been brought up several times, and previously it's been mentioned going higher tends to blur the differences, when most codecs approach transparency. The biggest suspect for all claims/tests are the equipment used is never mentioned. Although the problems should be spread across all the codecs, highs/lows/etc. come into play.

Actually, in that test referenced above, the very same (unnamed) computer system was used for the listening tests. The tests had to be done on a computer, since different portable players vary widely in the op-amps used (and thus sound signature and sound quality). In addition, portable players that support every one of those codecs all at the same time simply don't exist; at most, a given portable player will support only a few of those codecs.

 

[blessingx]

Quote:

Originally Posted by Eagle_Driver Actually, in that test referenced above, the very same (unnamed) computer system was used for the listening tests. The tests had to be done on a computer, since different portable players vary widely in the op-amps used (and thus sound signature and sound quality). In addition, portable players that support every one of those codecs all at the same time simply don't exist; at most, a given portable player will support only a few of those codecs.

Unless I missed something (in earlier invitation and recent results), anyone could participate. Any system/soundcard/headphone/speakers. Only requirement was a Windows computer or one Java compatible. It's been that way with all tests at Hydrogen. That's one thing when looking at 'transparency' claims, and to a lesser extent comparison tests, that should be considered. In narrowing down all variables, it's the one big blind spot. Still useful info though of course, and thanks to Roberto for all the hard work once again.