[dprimary]

The level difference between the WAV and AAC is the AAC is .03 dB lower on the left channel and .05 dB in the right channel.

 

[sonitus mirus]

 
The level difference between the WAV and AAC is the AAC is .03 dB lower on the left channel and .05 dB in the right channel

Have you attempted to repeat the ABX test?  8 of 10 is not typically considered to be statistically definitive, even for a totally objective tester that never claimed to be able to hear an audible difference.  Can the samples be analyzed by others?  Can you pass a similar ABX test using files encoded buy someone else?  
 
You have evidence that you can tell a difference.  Let's explore this.  

 

[dprimary]

I've done it a few times recently. Some music I can't tell at all. I've only recently start paying attention to it again. I went to a network digital audio seminar a few months ago covering a range of topics. It made me think about many things in digital I have not thought about in over a decade. A fair amount AB testing was involved and just re-sparked my interest. I am at the point tend to look at audio equipment as tools, so you could say I"m looking for a better set of screwdrivers.
 
I could somehow let others analyze the samples the WAV file is 100meg the AAC is 25meg. I could easily cut down the WAV file. I'm not sure if an AAC file be edited cleanly, I never had a reason to edit one. 
I could see if anyone else at the office can pick out the difference as well.
 
The AAC was created from the WAV file. No changes were made other then the encoding. They do not null, what is left is pretty far down, I think that is to be expected. Of course encoding to a FLAC or ALAC completely nulls.
 
I think I should be able to pass an ABX test with files that some else encoded.
 
What I'm really interested in testing is the possibility that differences between lossy and lossless encodings are more audible on large sound systems. I've noticed over the 5 or so years when I'm involved in the testing of a large system and a well encoded lossy recording is playing through it, often another engineer will ask to switch to something not compressed. It has happened too many times to dismiss it. It is not like large sound systems are the pinnacle of fidelity, though really well done ones can be surprising.

 

[bigshot]

If you are hearing differences in sharp drum beats, they shouldn't null all the way down in volume. If you are actually hearing those differences, there should be fairly loud impacts left behind. It sounds like something else is creeping into the test somewhere.

 

[bigshot]

By the way, if you are interested, I would be happy to send you a FLAC or ALAC file that contains ten samples... Fraunhofer MP3. LAME MP3, and AAC at 192, 256 and 320, along with a lossless sample. They are shuffled at random. If you would like to see if you can figure out which is which, let me know and I will send you a download link.

 

[dprimary]

The part I'm listening to is a soft shuffle. I don't hear the difference when the drums are hit hard. In the null test you can make out the loud drum beats. I would say what is left in the null is about 50 dB down. A Flac to Wav was a complete null as far as I could tell listing to it. 
 
Yes send me the link. 

 

[miceblue]

I made a few new audio tests if anyone is interested. I decided to use a track that can be looped well in case people need to spend > 1-minute listening to the track and getting use to it.

The original 24/96 source file is from:
http://www.hdtracks.com/open-your-ears
For $9, why not? That's less than an album from iTunes.

256 VBR AAC vs 256 VBR MP3: https://dl.dropboxusercontent.com/u/2816447/256VBRAACvs256VBRMP3.zip (59 MB)
24/96 vs 256 VBR MP3: https://dl.dropboxusercontent.com/u/2816447/2496vs256VBRMP3.zip (55 MB)
24/96 vs 16/44.1: https://dl.dropboxusercontent.com/u/2816447/2496vs1644.zip (62 MB)
24/96 vs 16/96: https://dl.dropboxusercontent.com/u/2816447/2496vs1696.zip (59 MB)
24/96 vs 24/48: https://dl.dropboxusercontent.com/u/2816447/2496vs2448.zip (69 MB)

All zip folders contain A and B files to compare (both files have the same bit depth and sampling rate so your DAC won't make weird click sounds when switching between different sampling rates), as well as another zip folder containing the answers with the original converted files and spectrograms for each audio file. All of the A/B files except for the ones in the first link have ReplayGain tags if they're needed, but all of them were within 0.1 dB, so you don't really need to use them.

I myself can't pass the ABX test between the original 24/96 and converted 256 VBR LAME MP3 files.

 

[sonitus mirus]

Nice work on the test file setups.  I downloaded and tested the 24/96 vs 256 VBR MP3 files and almost immediately discovered that I could not even hear a difference between file A and B to set a precedent for comparison.  I honestly tried to hear any difference, as can be seen by the time I spent before beginning the test.  That time was me attempting to hear some difference between the two files.  
 
Though I knew it was a hopeless endeavor doomed to fail, I still posted my Foobar ABX logs to make Greenears happy.  

 
 
foo_abx 1.3.4 report
foobar2000 v1.2.9
2015/02/21 19:08:46
File A: 2496vs256MP3\A.flac
File B: 2496vs256MP3\B.flac
19:08:46 : Test started.
19:14:13 : 01/01  50.0%
19:14:41 : 01/02  75.0%
19:17:29 : 02/03  50.0%
19:19:04 : 02/04  68.8%
19:20:30 : 03/05  50.0%
19:20:49 : 04/06  34.4%
19:21:04 : 04/07  50.0%
19:21:20 : 04/08  63.7%
19:21:49 : 04/09  74.6%
19:22:16 : 04/10  82.8%
19:26:34 : Test finished.
 ---------- 
Total: 4/10 (82.8%)

 

[TheTrace]

High sampling rates, high bit rates and lossless make absolutely no difference, and none of these thing are guarantees of good sound. I just finished going through a bit pile of HD Tracks, MFSL, fancy audiophile vinyl, SACD, Blu-Ray Audio and DVD-A... I heard "audiophile" releases that sounded worse than the plain vanilla CD. I heard a few with better mastering. But an awful lot of them sounded exactly the same as the regular CD. I took the best sounding ones down to AAC 256 VBR and the lossy sounded exactly like the HD. I finally came to the conclusion that you can't tell anything about the quality of the sound by the format it is delivered on.

Next I am trying out multichannel. I have higher hopes for that.

I have to agree with this after spending a great amount of time trying to identify audible differences.

As someone else mentioned here (don't feel like scouring the pages searching for one post), I also do notice a difference at times between AAC 256 VBR and FLAC 44.1/16 At certain points in more complicated songs during casual listening even. Though those moments are scarce, otherwise everything appears audibly transparent. I personally use AAC 256 for the likes of my iPhone 5s via iTunes bit rate converter and my archive of ALAC for my Pono.

As for the Pono itself, I believe the reception of the device itself is a bit harsh. Their marketing point is suspect whether it's due to intended deception or shocking lack of general audio knowledge. As for the performance of the device itself as a Digital Audio Player, it beats anything I've used sound quality output wise; iPhones and iPods, nothing "hi-res." If the marketing was focused on the device performance rather than inaccurate info about formats, they'd be pretty successful.

As you can probably infer from this post, I own a Pono, enjoy it thoroughly, do not utilize their online store.

I acquire hi-Rez from the likes of HDTracks, needle-drops when the retail CD master counterparts are brick-walled, poor mastering, etc. and dither to 44.1/16 or 48/16 accordingly. Who knows? If future releases from the Pono store have legitimate superior masters to that of modern retail CDs, I may become a gradual customer. That's the magic word though, "if."

 

[U2Bono269]

I bought a copy of Neil Young's Harvest from PonoMusic. I bought the 24/192 version, just because I figured I might as well. I guessed that because this is Neil Young's project, his recordings were probably a good bet to have been mastered well.
 
I compared it to the master from my CD...I don't know which version it was, but I'm pretty sure it's from the 80s because my dad gave it to me a few years ago.
 
To my ear, the PonoMusic version is superior. Very superior.
 
And before anyone jumps on me, I don't attribute this to the 24bits. It's the mastering. It's very well done.
 
I have heard that the PonoPlayer itself is very well made with high quality components, and I'm sure it sounds amazing. I'd love to hear what this album sounds like on the PonoPlayer itself.

 

[bigshot]

As someone else mentioned here (don't feel like scouring the pages searching for one post), I also do notice a difference at times between AAC 256 VBR and FLAC 44.1/16 At certain points in more complicated songs during casual listening even. Though those moments are scarce, otherwise everything appears audibly transparent.

 
My entire music library, with over a year and a half of music is encoded at AAC 256 VBR. I did a LOT of testing to arrive at that, because I really wanted to be sure that the sound was as good as lossless before I started ripping tens of thousands of CDs. I listen every day, and only twice have I found artifacting... and both times I pulled the original CD to double check. The first time, it turned out to be a frequency response spike at 4kHz, not an encoding error, and I corrected that in a jiffy with my equalizer. The other time it was a nasty digital splat sound. Clearly an artifact. I went back to the original CD and the glitch was on there too. It had been mastered into the recording. So every time I've discovered an artifact in AAC 256 VBR, it's always turned out not to be a problem with AAC.

 

[TheTrace]

My entire music library, with over a year and a half of music is encoded at AAC 256 VBR. I did a LOT of testing to arrive at that, because I really wanted to be sure that the sound was as good as lossless before I started ripping tens of thousands of CDs. I listen every day, and only twice have I found artifacting... and both times I pulled the original CD to double check. The first time, it turned out to be a frequency response spike at 4kHz, not an encoding error, and I corrected that in a jiffy with my equalizer. The other time it was a nasty digital splat sound. Clearly an artifact. I went back to the original CD and the glitch was on there too. It had been mastered into the recording. So every time I've discovered an artifact in AAC 256 VBR, it's always turned out not to be a problem with AAC.

I see, I'd probably convert my library to AAC for my Pono too, I've grown lazy overtime through audio experimentation haha. Perhaps when the next time I'm pressed for space.

What's your stance on DSD though? That's relatively a new area for me and I see a lot of analog comparisons in regards to sound.

 

[Steve Eddy]

What's your stance on DSD though? That's relatively a new area for me and I see a lot of analog comparisons in regards to sound.

I think the "analog comparisons" are largely psychological resulting from "explanations" not having the grossly over-simplified "stair step" graphic that's so often used to explain PCM. But there's nothing of DSD that's any more "analog" than PCM. It's digital. It's sampled. It's quantized. In fact the quantization error is huge, because it's single bit, and requires a huge amount of negative feedback and noise shaping to hide it.

Here's the Lip****z/Vanderkooy on the subject. If you don't want to plow through it, just go down to the conclusions section.

Another thing to think about. Sony (along with Philips) owned the patents on Compact Disc and they were soon to expire. Sort of like drug companies coming out with questionably "new & improved" drugs to replace the drugs whose patents were about to expire.

se

 

[Greenears]

  Nice work on the test file setups.  I downloaded and tested the 24/96 vs 256 VBR MP3 files and almost immediately discovered that I could not even hear a difference between file A and B to set a precedent for comparison.  I honestly tried to hear any difference, as can be seen by the time I spent before beginning the test.  That time was me attempting to hear some difference between the two files.  
 
Though I knew it was a hopeless endeavor doomed to fail, I still posted my Foobar ABX logs to make Greenears happy.  

 
 
foo_abx 1.3.4 report
foobar2000 v1.2.9
2015/02/21 19:08:46
....

 
Massive grinning ear-to-ear

 

[OddE]

  I bought a copy of Neil Young's Harvest from PonoMusic. I bought the 24/192 version, just because I figured I might as well. I guessed that because this is Neil Young's project, his recordings were probably a good bet to have been mastered well.
 
I compared it to the master from my CD...I don't know which version it was, but I'm pretty sure it's from the 80s because my dad gave it to me a few years ago.
 
To my ear, the PonoMusic version is superior. Very superior.

 
-I've just done pretty much the same thing - except I chose 'After The Gold Rush' instead, it being one of my definite NY favourites.
 
While it does sound good (as you'd expect!), the really surprising bit is that the Foobar2k Dynamic Range plugin claims that the effective DR on the 24/192 version is slightly lower than that of my ancient CD rip. Not enough for me to notice a marked difference during initial listening - but surprisingly enough that I will see if I can find some other way of evaluating the DR. Hm, perhaps some Sox incantation can help me out.
 
It will be interesting to do a couple of ABX tests after work to see whether I can really tell the two versions apart; first impression is that it should be easy, as the noise floor on the 24/192 is significantly lower - perhaps I should add noise to the 24/192 at the same level as on my CD rip, so that I cannot use that rather simple cue to tell the versions apart.
 
This is going to be fun.