[CoiL]

In the morning, with still sleepy brain/ears:
 
 
Will try to to long test tomorrow maybe.

 

[shstux]

 
Did you expect them to make it an easy test to reproduce?

 
Well, I've just put up the FAAC 320kbps version.
(faac -b 320 x.wav x.aac)
 
http://abx.digitalfeed.net/faac.320.html
 
I think I need to create an index page on the site so I don't need to keep posting specific URLs and getting mod-checked =)
 
(Tragedy is that with my equipment and my hearing, I perform terribly on ABX tests: I have to substitute the 'lossy' files for 'tone of the same duration' to test my code)

 

[shstux]

  In the morning, with still sleepy brain/ears:
 
 
Will try to to long test tomorrow maybe.

 
Cool =)
You can see there at the bottom the problem with Bonferroni correction for the per-track comparisons: when doing 5 trials, even if you get all 5 right it isn't enough to be 'statistically significant' with the more stringent alpha.
 
If you do 10 or 20 trials, you actually get results in between each track (i.e. after a block of trials) which gives you results for that individual track. Because the total number of trials is fixed in advance, this shouldn't affect the validity of the overall result. (The per-track results are still shown in the final results anyway).
 
 
For anyone interested, I have created a 'list' page (original url + /list.html) so you can see the available tests there. Currently that is only opus 128, faac 320, and the original Tidal samples. The Opus 128 and FAAC 320 are encoded from the lossless tidal samples (to which they are compared).
 
 
Edit: To those who were commenting on sync issues. There were *big* sync issues (among other things). The James Blake song had a (typical for these samples) 45ms sync mismatch for the start of the track, and the Dixie Chicks was the worst with a full 100ms sync error. One of the tracks (The Killers? Can't remember) also had a mismatched fade-out.
Anyway, I've corrected those errors and re-uploaded the tracks so now if you go and do the test again you'll notice there is no clicking when switching between tracks.
It is still the original samples, but with the sync issues fixed and that fadeout weirdness corrected.

 

[stv014]

  When I saw the Tidal listening test, I was really annoyed by the poor reporting of results (i.e. the lack of stats).
I went and built an 'alternative' test which uses an ABX-style format to test *exactly the same samples*.

 
It is probably best to simply download the files, and use the foobar ABX plugin, or similar software. Here are the download links:

http://test.tidalhifi.com/songs/killers_30.flac
http://test.tidalhifi.com/songs/killers_30_lossy.flac
http://test.tidalhifi.com/songs/blake_30.flac
http://test.tidalhifi.com/songs/blake_30_lossy.flac
http://test.tidalhifi.com/songs/daftpunk_30.flac
http://test.tidalhifi.com/songs/daftpunk_30_lossy.flac
http://test.tidalhifi.com/songs/eagles_30.flac
http://test.tidalhifi.com/songs/eagles_30_lossy.flac
http://test.tidalhifi.com/songs/dixiechicks_30.flac
http://test.tidalhifi.com/songs/dixiechicks_30_lossy.flac

 

[stv014]

  No null because of the sync issue, but plot spectrum shows a slight increase in bass and treble in the lossless version of the Killers song. The difference in bass is about 1 dB at 60 Hz and goes up to 3 dB at 20 Hz, likewise the treble difference is about 1 dB from 10-16 kHz. I don't know if this is the result of the codec itself, or some EQ on Tidal's part. Hopefully someone with better testing software will find something more concrete.

 
Indeed, I have noticed similar treble and bass boosts on blake_30.flac vs. blake_30_lossy.flac. It looks more like EQ (although the curve may not be the same for all files) than encoding artifacts to me. The difference is enough to be relatively easily detected in an ABX test.
 
Edit: this is the frequency response I have extracted from blake_30_lossy.flac (relative to blake_30.flac), it matches the description of "Compressed music weakens for instance the pressure in the bass, details in cymbals and gives the sound less headroom.", and looks like parametric shelving EQ, so it could very well be intentional:
    (click to view full size version)

 

[CoiL]

So, do I understand correctly that this test is intentionally "produced" for PR purposes so that ppl could detect differences more easily?

 

[Hudson]

  So, do I understand correctly that this test is intentionally "produced" for PR purposes so that ppl could detect differences more easily?

 
I've downloaded the files above and I can identify between their lossy and non-lossy files using the foobar ABX plugin. 19/20 correctly so far.
 
I transcoded their non-lossy files using dBpoweramp using Lame mp3 192kbps CBR, and I fail to tell the difference by ABX, no better than guessing.
 
To my (decidedly average) ears it's far too easy to identify between their lossy and non-lossy files. Something seems wrong to me.
 
(Edit typo)

 

[limpidglitch]

   
Indeed, I have noticed similar treble and bass boosts on blake_30.flac vs. blake_30_lossy.flac. It looks more like EQ (although the curve may not be the same for all files) than encoding artifacts to me. The difference is enough to be relatively easily detected in an ABX test.
 
Edit: this is the frequency response I have extracted from blake_30_lossy.flac (relative to blake_30.flac), it matches the description of "Compressed music weakens for instance the pressure in the bass, details in cymbals and gives the sound less headroom.", and looks like parametric shelving EQ, so it could very well be intentional:
    (click to view full size version)

 
That explains why I couldn't for the life of me, despite oversampling, level adjustments and careful alignments, get the difference to creep down to expected levels.
Dirty trick.

 

[shstux]

Judging by the terrible job they did making the samples the same even in terms of sync and duration, I'm not surprised there are other problems too.
 
Using an EQ on the track does mesh with them specifically saying:
"

Compressed music weakens for instance the pressure in the bass, details in cymbals and gives the sound less headroom."​

If they weren't confident that people would be able to detect it (and frankly with 320kbps AAC, I'm confident no 'normal' user would detect it) then it would make sense that they may want to engineer a difference.​

 
@CoiL you might want to take the test again now that I've fixed the stupid sync issues. It doesn't click any more when switching between tracks (because the times actually match) so it is much easier to listen for actual differences.
(Because the Tidal guys also commented on the encoding quality of Spotify and iTunes, I've also generated ABX tests for those encoding qualities. They're all on the list of tests page, i.e. /list.html)

 

[Stereodude]

  I transcoded their lossy files using dBpoweramp using Lame mp3 192kbps CBR, and I fail to tell the difference by ABX, no better than guessing.

 
Did you mean lossless? 

 

[shstux]

   
It is probably best to simply download the files, and use the foobar ABX plugin, or similar software. Here are the download links:
...

 
Yeah, I wanted people to be able to try to do an ABX themselves without needing to download an application, etc, which is why I created the web-based app. It has also meant I've been able to use 'corrected' versions of their tracks: i.e. they are actually in sync, and there isn't a weird mismatched fade-out. Because I'm using flac files as the source, then transcoding them to raw audio data in the browser and playing the raw audio, I don't expect that they would sound any poorer through the browser than through Foobar. However, it wouldn't be the first time my expectations were violated if it did.
 
Also, I use a Mac.
 
For those who want to actually test the synced versions of the tracks using Foobar:
http://abx.digitalfeed.net/tracks/corrected/blake_30.flac
http://abx.digitalfeed.net/tracks/corrected/blake_30_lossy.flac
http://abx.digitalfeed.net/tracks/corrected/daftpunk_30.flac
http://abx.digitalfeed.net/tracks/corrected/daftpunk_30_lossy.flac
http://abx.digitalfeed.net/tracks/corrected/dixiechicks_30.flac
http://abx.digitalfeed.net/tracks/corrected/dixiechicks_30_lossy.flac
http://abx.digitalfeed.net/tracks/corrected/eagles_30.flac
http://abx.digitalfeed.net/tracks/corrected/eagles_30_lossy.flac
http://abx.digitalfeed.net/tracks/corrected/killers_30.flac
http://abx.digitalfeed.net/tracks/corrected/killers_30_lossy.flac

 

[Hudson]

Yeah sorry, typo. I transcoded the lossless flac!

 

[sonitus mirus]

When I first tried this test last week I was at work with my Mr. Speaker Mad Dogs and I only gave it a quick, halfhearted attempt.  With the distractions and low volume level, I didn't hear any obvious differences and only guessed 3/5 correctly.  I tried this at home tonight and I was able to successfully guess all 5 tracks 3 consecutive times. (15/15)  I normally would never have the patience to do this on a whim, but the test was so simple that I didn't even have to listen to a full pass of the samples for me to make my guess.  I was just listening for a slight increase in bass.  Really, nothing more was needed.
 
Chromebook>USB out to NuForce Icon HDP>RCA out to Schiit Asgard>Lawton-tuned Denon D5000 Headphones
 
I have the Eagles "Hotel California" on CD, and I have used this track in the past to see if I could hear a difference between a FLAC file and a Lame vbr -0 mp3 file in an ABX with no success.
 
I suspect that differences were intentional created, as I have never been able to successfully ABX between lossless and AAC 256 or Lame vbr -0, much less lossless and AAC 320.

 

[stv014]

I suspect that differences were intentional created, as I have never been able to successfully ABX between lossless and AAC 256 or Lame vbr -0, much less lossless and AAC 320.

 
It was already discussed above that the lossy files apparently have an "n" shaped EQ curve applied (see post #20), that makes them sound worse. This could very well have been intentional, as no competent encoder would do something like that.

 

[limpidglitch]

Just for the heck of it, and a bit as an excuse to sharpen my R-chops, I did a little analysis myself.
I did not make difference files, but rather captured the frequency response of three files: the lossless version, Tidal's lossy version, and a lossy version I made myself from the supplied lossless file. I then subtracted and compared these spectrums, giving this result:
 

 
Stuff. (no audio files, seems to have erased those)