Successful ABX testing to hear the difference between Redbook Audio vs upsampled to 192/24

[nick_charles]

I had another go this morning A vs B - very carefully using a small segment
 

 
 
Not quite significant but I think I know what it might just be. The difference I was close to detecting was volume levels when I correctly detected a difference it seemed to be a small difference in volume.
 
I'll take the samples and run them through Audacity/Excel and see if this is borne out or if I was just lucky at guessing....
 
 
 
Well it is small but there is a definite and consistent (same direction) level difference between A and B - above 22K the difference is about 63db 

 
The difference starts vanishingly small 
 

Frequency (Hz)	A Level (dB)	B Level (dB)	diff
23.4375	-25.431767	-25.390003	-0.04176

187.5

-17.609245

-17.599188

-0.01006

539.0625

-30.870625

-30.864891

-0.00573

1007.8125

-28.59691

-28.591702

-0.00521

2039.0625

-37.202091

-37.195126

-0.00697

3000

-36.747459

-36.737373

-0.01009

4007.8125

-42.068069

-42.0527

-0.01537

5015.625

-43.111671

-43.089882

-0.02179

6000

-46.391438

-46.361816

-0.02962

7007.8125

-47.42057

-47.381645

-0.03892

 
 
 
but it rises steadily above 7K 

Frequency (Hz)

A Level (dB)

B Level (dB)

diff

8015.625

-48.8223

-48.772469

-0.04983

9000

-49.206306

-49.144566

-0.06174

10007.8125

-48.627392

-48.552158

-0.07523

11015.625

-49.543945

-49.453907

-0.09004

12000

-52.505165

-52.398869

-0.1063

13007.8125

-53.120102

-52.995827

-0.12428

14015.625

-53.288063

-53.144871

-0.14319

15000

-56.275017

-56.111607

-0.16341

16007.8125

-62.024593

-61.838955

-0.18564

17015.625

-62.653885

-62.44482

-0.20907

18000

-65.765213

-65.531807

-0.23341

19007.8125

-66.440056

-66.180664

-0.25939

20015.625

-68.474182

-68.187218

-0.28696

 
 
and so on...For me everything above 17K is a foreign country and the OP says his hearing tops out about 18K. In some cases hearing a difference of .1db is not unheard of. It would still be pretty remarkable and hearing a difference in a specific high range of .2db to .23db would be extremely impressive - maybe I detected a difference but not consistently maybe I was just lucky but not quite lucky enough
 
 
I might retry later - even failing it was incredibly hard work - to me both sounded exactly the same when just listening  but I do not claim golden ears - fun though !

 

[sonitus mirus]

I believe I speak for many of us lurkers in thanking you for taking the time to test and for providing some very detailed analysis.  I understand volume differences better than more obscure instrument separation properties between the two files.  I can't hear much over 14K now, so I wasn't going to bother trying to ABX this set, as I knew it would have been useless.

 

[nick_charles]

Quote:

I believe I speak for many of us lurkers in thanking you for taking the time to test and for providing some very detailed analysis.  I understand volume differences better than more obscure instrument separation properties between the two files.  I can't hear much over 14K now, so I wasn't going to bother trying to ABX this set, as I knew it would have been useless.

 
I used to have (far too) many CD players. Each new one seemed to be different from the last. Each one however was at a different level of output. when I sampled the analog outs the loudest was 0.6db louder across the board and DBTing samples derived from it from the others was trivial. Using audacity to adjust the levels to within 0.1db made the samples indistinguishable. I did once experiment to plot the smallest difference I could detect but I forget what I concluded I did also try the same with low pass filters, although I could hear above 16K I had trouble detecting a filter starting at 13K, an 11K filter was no problem - all the usual disclaimers apply. 
 
I suppose the broader question to ask is what audible parameters can we rationally expect upsampling/resampling from a competent red book source file to change. If done properly nothing much in the audible range should change. Noise at either rate should be inaudible, channel separation should not change, channel levels should stay equivalent, distortion should not become audible or drop notably, group delay should be unaltered but in any case only gross differences would be audible. So what does that leave ? it is hard to see how upsampling would affect elements inherent in the recording such as perceived soundstage 

 

[stv014]

Quote:

Well it is small but there is a definite and consistent (same direction) level difference between A and B - above 22K the difference is about 63db 

 
What is odd is that there is a ~0.006 dB difference at 2000 Hz, even though 1500-2000 Hz was level matched with higher than that accuracy using 0 dBFS sine waves. Maybe it is because of non-linear distortion, although the 0 dBFS tones had less than 0.001% THD, while the level difference in the music is about 0.07%.

 

[nick_charles]

Quote:

 
What is odd is that there is a ~0.006 dB difference at 2000 Hz, even though 1500-2000 Hz was level matched with higher than that accuracy using 0 dBFS sine waves. Maybe it is because of non-linear distortion, although the 0 dBFS tones had less than 0.001% THD, while the level difference in the music is about 0.07%.

 
 Dunno - perhaps Audacity is not doing such a great job 
 
 
 

 
 
these were the setting i used to do the spectrum analysis - i'll happily defer to Audacity experts and  redo it using other options if there is a better set of options 

 

[stv014]

As far as I can tell, the difference really seems to be there (I did not use Audacity). It should not be audible by itself, though.

 

[UltMusicSnob]

Quote:

I believe I speak for many of us lurkers in thanking you for taking the time to test and for providing some very detailed analysis.  I understand volume differences better than more obscure instrument separation properties between the two files.  I can't hear much over 14K now, so I wasn't going to bother trying to ABX this set, as I knew it would have been useless.

Ditto--many thanks for the analyses!
Very quick and dirty charting here:
I didn't take the time to properly label the points, but it runs col by col from L to R in increasing freq as specified above. And of course these are neg db, so distance from the top is more difference. I notice that the least differences are concentrated in the most sensitive hearing ranges. The last four data points are ranges I don't hear, unless it's subliminal or something.

 

[nick_charles]

Quote:

As far as I can tell, the difference really seems to be there (I did not use Audacity). It should not be audible by itself, though.

 
 
If we can trust these differences then it depends on whose research you cite as to whether it could be audible or not. Measured JND Thresholds tested over the years vary from over 1 db to about 0.1db depending on how the tests were run , the signals used and so on.
 
I agree it is right on the edge of credibility and even then I did actually fail (in my research circles there is this weasel phrase marginally significant) - but if the (audible to ULTy) difference is in the stimuli I am at a loss to think what else it could be other than volume?

 

[UltMusicSnob]

Quote:

 
 
If we can trust these differences then it depends on whose research you cite as to whether it could be audible or not. Measured JND Thresholds tested over the years vary from over 1 db to about 0.1db depending on how the tests were run , the signals used and so on.
 
I agree it is right on the edge of credibility and even then I did actually fail (in my research circles there is this weasel phrase marginally significant) - but if the (audible to ULTy) difference is in the stimuli I am at a loss to think what else it could be other than volume?

There is a potential connection between frequencies and directionality--if high freq's are greater in one file, then conceivably directional cues could be related to the increased signal energy in those bands. That could give rise to a subjective impression about soundstage detail.
And of course, it's an interesting problem--why the differences at higher frequencies? Where did those differences come from?

 

[stv014]

Quote:

If we can trust these differences then it depends on whose research you cite as to whether it could be audible or not.

 
I meant it is the 0.006 dB difference in the midrange that should not be audible.
 
By the way, here is a quick FR graph I have created (the low frequency range may be inaccurate, but it mostly matches the Audacity numbers); it may look familiar from stereophile.com

:

Here is also D vs. B - hearing a difference here is surprising:
   

 

[stv014]

Quote:

Originally Posted by nick_charles /img/forum/go_quote.gif
 
Not quite significant but I think I know what it might just be. The difference I was close to detecting was volume levels when I correctly detected a difference it seemed to be a small difference in volume.
 
I'll take the samples and run them through Audacity/Excel and see if this is borne out or if I was just lucky at guessing....
 
Well it is small but there is a definite and consistent (same direction) level difference between A and B - above 22K the difference is about 63db 

 
This can be tested easily, just amplify A by a small amount, something like a few hundredths of a dB (so that the boost in the midrange is not audible yet, but the roll-off in the top octave becomes less significant).

 

[stv014]

Well, here is what the files in upsample_test.zip are:
 

Spoiler: Warning%3A%20Spoiler!

- A.flac: recorded from the line output of a Xonar Essence STX sound card, at -1 dB volume (to avoid clipping), and using the default fast roll-off DAC filter; the PCM1792A DAC also uses a linear phase filter, just like B.flac. The slight high frequency roll-off is from analog lowpass filtering.
- B.flac: the "reference" file, created with "resample.exe -k 0.000002923047 -r 192000 -f 3 -ff 0.5 -fw 0 -fl -2400 -g 0.93 Test_File_Foobar_Redbook.wav B.wav". It basically tries to simulate Test_File_Foobar_192_24.wav, but with the volume reduced by a factor of 0.93 to prevent clipping
- C.flac: recorded from a Xonar D1 sound card at full volume, with the slow roll-off DAC filter (shorter impulse response, lower group delay, more ultrasonic imaging)
- D.flac: another upsampled file that tries to approximate (very roughly) the fast roll-off filter of the Xonar D1 (CS4398 DAC); it was created with the command "resample.exe -k 0.000002923047 -r 192000 -f 3 -ff -0.503061 -fw -80 -fl -3300 -g 0.93 Test_File_Foobar_Redbook.wav D.wav". So, it is a minimum phase filter, but the CS4398 filter does not seem to be entirely minimum phase, as it does have a small amount of pre-ringing. Also, the frequency response could not be accurately reproduced with a windowed sinc FIR filter.
- E.flac: recorded from a Xonar D1 sound card at full volume, with the default fast roll-off DAC filter; it is basically the same as a previously already tested file, just re-recorded, and it has slightly reduced volume (by a factor of 0.93) to match the level with the other files while avoiding clipping
- F.flac: it has identical PCM audio data to B.flac, it was just encoded with slightly different FLAC settings not to make it obvious that the files are in fact the same

 
In addition to the small level differences mentioned by nick_charles, another potential issue is that the foo_abx plugin does not smoothly fade in and out the sound when switching or starting/stopping files. So, if there are any phase/delay differences (that are not large enough to be audible by themselves) between the files, it could result in different clicking artifacts when starting or stopping the sound, and those could be picked up as clues (maybe even sub-consciously). This is only a random theory, though, and perhaps in practice it is not easy to find and exploit such glitches, even with hundreds of trials.

 

[UltMusicSnob]

Here are some analyses of the pairwise files. The following are static spectrum analyses of difference files on each pair.
I extended the range out to 96 kHz, because these are 192/24 wav's, and for the chance to see any artifacts that pop up, but I seriously doubt any energy up there made any difference in playback, unless my headphones have secret ultrasonic capabilities I didn't know about.
 
Difference file A B:

 
Difference C B:

 
Difference D B:

 
Difference E B:

 
I see a uniform difference in 20 Hz to ~ 300 Hz in the test files that were recorded from the Xonar sound card. That difference is entirely missing from the D file, which was strictly a resample exercise. Those resample settings ended up with all of the difference energy in audible high freq's. The D B difference was also greater in 10k - 20k than any other comparison. C appears to have a difference peaked right at the 22.5 kHz level, symmetrical around it. C is also the only one with the extra difference energy ~40 kHz, so maybe these are results of the slow roll-off instead of fast filter?
 
Just to check the descriptions above:
A.flac is playback of B.flac recorded from the Xonar Line Out?
C.flac is also playback of B.flac, recorded from the Xonar Line Out, but with a different DAC filter setting?
D.flac is straight resample, like B.flac, but with slightly different settings for upsampling
E.flac is also playback of B.flace, recorded from Xonar Line Out, but with volume adjusted before playback, instead of at playback like A.flac.

 

[nick_charles]

^ I'm not sure i understand this. when I plotted the differences between A and B they were smallest at low frequencies and greatest at higher frequencies. You graph shows an opposite trend with the difference at low frequencies being about 0.4% dropping to about 0.0016% at 20K - are we using the same A and B ? I used the files linked from post #85
 
What tool did you use to plot the differences - is it free ?

 

[UltMusicSnob]

In Soundforge10 I opened both files, inverted one, and then mixed to the other. A, C, D, and E were flipped, and each one was mixed to B to create difference files for each pair.
 
The difference files that resulted were saved, and then analyzed using a VST plugin, Voxengo SPAN, free version. The plugin was opened in SoundForge 10.

I opened the Static analysis window, opened the db range to -110 db, left the max db at -18, extended the max freq range to 96 kHz, set slope to 0 (default Slope was 4.5). A lot of users recommend slope of 3, which does change the result somewhat, but bass difference is still there and represents largest differences in this comparison.
 
All the comparisons (and all my listening tests) were done using .wav files decompressed from the original FLAC's you posted.
 
Difference A B file with tool set to display Slope 3 instead of 0