Head-Fi.org › Forums › Equipment Forums › Sound Science › Successful ABX testing to hear the difference between Redbook Audio vs upsampled to 192/24
New Posts  All Forums:Forum Nav:

Successful ABX testing to hear the difference between Redbook Audio vs upsampled to 192/24 - Page 2

post #16 of 136
Quote:
Originally Posted by UltMusicSnob View Post

 

Can you describe what you would expect a lowpass filter effect to sound like, as a differential?

 

I cannot because I do not hear a 20+ kHz lowpass filter. But high frequency response could affect how spatial information is perceived. Then again, it is just a random theory in case the difference is not spurious (which I still suspect it is), and it does assume "golden" ears. What do you think is causing it ? It must be either in the file data, or in the way it is played back (in which case it will vary by system).

post #17 of 136
Thread Starter 
Quote:
Originally Posted by stv014 View Post

 

I mean the time between clicking "Play", and when you actually hear the sound. I did a quick test now running foobar2000 with WINE (on Linux) using the WASAPI plugin in "push" mode, and indeed I hear a delay difference which I could easily ABX with 100% success.


How long is the delay? I'm running Win7, but I didn't have the WASAPI plugin (just now picked it up, thanks for the tip).

I've previously tried deliberately going for delay time as a cue, but no success--the way I was running it (foobar2000 plus ABX utility only) I couldn't find a delay--I've never had a 100% success.

 

OK, just now installed WASAPI and loaded it up.

I think I hear what you're talking about in the delay (haven't actually been able to get a positive result, though), but in any case I don't use continuous playback to A/B.

I set "Start" and "End" times, and uncheck the "keep playback position" box.

This allows me to directly compare one distinct musical segment in two versions, not different parts of the clip as we pass through the entire track while A/B-ing.

So I'll play, say, 'A' multiple times, stopping every time. Then I take a breath, and play 'B', and stop.

It's possible that there could also be a delay as well at the beginning of every playback restart, except that it sounds instantaneous to me--distinctly different from the continuous-playback skip that occurs when A/B-ing with the "Keep playback position" box checked.

 

In any case, now I can go back and try with WASAPI for the several comparisons, many thanks for the tip.  smile.gif

post #18 of 136
Thread Starter 
Quote:
Originally Posted by stv014 View Post

 

I cannot because I do not hear a 20+ kHz lowpass filter. But high frequency response could affect how spatial information is perceived. Then again, it is just a random theory in case the difference is not spurious (which I still suspect it is), and it does assume "golden" ears. What do you think is causing it ? It must be either in the file data, or in the way it is played back (in which case it will vary by system).


According to what I know about PCM coding and information theory in general, it seems to me the difference *must* lay in what happens between the bits in the .wav files and what hits the Line Out (after that the signal chain is physically identical), i.e., something is different in the way that the data provided is used. I suspect an outcome specifically of the D/A conversion which affects temporal resolution. I've gone as far as looking at time-smearing artifacts and phase-shifting artifacts which could "blur" the spatial image in 44.1 playback, but less so in 192 playback, since that is my subjective experience. Directional information is conveyed more accurately by high frequencies, so that could play a role in spatial imaging. I have 52-year-old ears, so I'm far from confident about the role of near-22-kHz in these outcomes.

post #19 of 136
Thread Starter 

The following replicates the "Groove Tube" test, but this time foobar2000 was using the WASAPI plugin to provide a bitstream known to be unaltered by Windows-- http://wiki.hydrogenaudio.org/index.php?title=Foobar2000:Components_0.9/foo_out_wasapi

 

foo_abx 1.3.4 report
foobar2000 v1.2.8
2013/08/14 14:56:52

File A: C:\Users\jhughes1\Desktop\Test_File_Foobar_Redbook.wav
File B: C:\Users\jhughes1\Desktop\Test_File_Foobar_192_24.wav

14:56:52 : Test started.
14:57:31 : 01/01  50.0%
14:57:41 : 02/02  25.0%
14:57:53 : 02/03  50.0%
14:58:04 : 03/04  31.3%
14:58:15 : 04/05  18.8%
14:58:29 : 05/06  10.9%
14:58:54 : 06/07  6.3%
14:59:07 : 07/08  3.5%
14:59:21 : 08/09  2.0%
14:59:56 : 09/10  1.1%
15:00:34 : 09/11  3.3%
15:00:49 : 09/12  7.3%
15:01:06 : 09/13  13.3%
15:01:46 : 10/14  9.0%
15:02:04 : 11/15  5.9%
15:02:34 : 12/16  3.8%
15:03:22 : 13/17  2.5%
15:03:41 : 14/18  1.5%
15:04:23 : 14/19  3.2%
15:04:52 : 14/20  5.8%
15:05:37 : 15/21  3.9%
15:06:26 : 16/22  2.6%
15:07:16 : 17/23  1.7%
15:07:53 : Test finished.

 ----------
Total: 17/23 (1.7%)

post #20 of 136

If I've read your description correctly, you're comparing a 16/44.1 file with a 24/192 version. The problem with doing this is that you are not just comparing the files, you are comparing the performance of the playback chain at 2 different bit depth/rate combinations. If you want to compare just the effects of down/upsampling, you need to make both files the same depth/bitrate.

I would suggest starting with a known 24/192 file, downconverting it to 16/44.1 and upconverting it back to 24/192, then comparing it with the original.


Edited by Don Hills - 8/14/13 at 3:47pm
post #21 of 136
Thread Starter 
Quote:
Originally Posted by Don Hills View Post

If I've read your description correctly, you're comparing a 16/44.1 file with a 24/192 version. The problem with doing this is that you are not just comparing the files, you are comparing the performance of the playback chain at 2 different bit depth/rate combinations. If you want to compare just the effects of down/upsampling, you need to make both files the same depth/bitrate.

I would suggest starting with a known 24/192 file, downconverting it to 16/44.1 and upconverting it back to 24/192, then comparing it with the original.


I hadn't thought of that experiment, but it sounds interesting, I will try it. The original impulse was discovering the foobar2000 ABX tool in the first place by a skeptic who appeared to believe that users who tried it on their upsampled files would discover that they could not hear a difference, and were thus merely experiencing a placebo effect when listening non-blind. I accepted the challenge, grateful for the tool which I had not heard of before, and have since repeatedly passed the test under an increasingly variety of circumstances.

 

I think it's likely that indeed the cause of the differences I hear does indeed result from the "performance of the playback chain at 2 different bit depth/rate combinations", and that is indeed my actual listening interest. I had been upsampling for a short time before (just since purchasing SoundForge 10), and felt that the upsampled playbacks, playback chain and all, did sound better subjectively. This series of ABX tests demonstrates that I am hearing a difference that I can identify in blind testing--no placebo effect here.

 

The scientific question of isolating the pure effects of 192/24 vs Redbook, if that pure isolation is even possible, is interesting, but is not my goal. I want better sound out of the CD's I own, and I can get it by upsampling and playing back through my equipment. Specifically, I hear better spatial detail. I hypothesize that my listening experience (and true, confirmed ability to discern) may be the result of improved temporal resolution during the D/A conversion processes specifically, but I don't have near the circumstances to do anything remotely like proving it rigorously. I need at least one more listener who can tell the files apart, for one thing--right now I'm a sample n of 1, and unless more ears turn up capable of passing ABX under *some* circumstances, playback chain and all, there's never going to be an opportunity of drawing a conclusion about anything except my own personal listening experiences, at best as a single case.

 

I *have* passed ABX repeatedly on more than one system, and I intend to get to as many different ones as I can. Perhaps they are all the same, but in that case I would simply be able to rest assured that the difference I hear IS robust across different kinds of equipment and different playback chains.

post #22 of 136
Thread Starter 

The following is the result of a "cold" listen of 44.1_16 vs 192_24, meaning no training, no warmup, and no use of the A and B buttons in the foobar2000 ABX utility.

The foobar2000 player is running the WASAPI plugin, this is through the RME Babyface and DT 770 Pro's.

I've developed a sense that I definitely know what I'm listening for--soundstage size and depth--so I selected the files, opened the tool, and went straight to X and Y buttons to choose, with no listening to A and B whatsoever.

 

If there is an audible delay difference caused by switching, this procedure should negate it. The only operation in this test was to go directly back and forth between X and Y, so moving from one version to another of the same sample rate was not possible. No time-based switching cue is available, because A and B are not used at all. The sample rate switches every single time, period.

The track is 'Catch My Breath' by Yasutaka Nakata's group capsule, from their album "Sugarless Girl"--I recommend it as a fantastic piece of EDM, btw, along with the whole rest of the album.

foo_abx 1.3.4 report
foobar2000 v1.2.8
2013/08/14 20:46:43

File A: C:\Users\KiarkAudio\Documents\capsule\Sugarless Girl\Catch my breath.wav
File B: C:\Users\KiarkAudio\Documents\capsule\Sugarless Girl\Catch my breath_192_24.wav

20:46:43 : Test started.
20:48:39 : 01/01 50.0%
20:49:05 : 02/02 25.0%
20:49:14 : 03/03 12.5%
20:49:31 : 04/04 6.3%
20:50:04 : 05/05 3.1%
20:50:13 : 06/06 1.6%
20:50:22 : 07/07 0.8%
20:50:31 : 08/08 0.4%
20:50:52 : 09/09 0.2%
20:51:09 : 10/10 0.1%
20:51:12 : Test finished.

----------
Total: 10/10 (0.1%)

I realize there's an element of taking my word for it on various aspects of these tests, but if you want to see me do it, a video screen capture of the entire test from loadup to conclusion is available at the link below. It apparently doesn't play inside the zippy webpage itself, but the mp4 file is only about 10MB.

Zippyshare.com - Foobar_ABX_capsule.mp4

 

I used about the first 25 seconds of this track to make my determination, I'd be happy to ship anyone the clips if they want to try it.

post #23 of 136

As stv014 and Don Hills have mentioned, I'd also be more interested in a comparison with the playback system at equal sampling rate (well, bit depth too, I suppose, to be safe).

 

Some poorer outputs and systems—particularly older ones doing on-the-fly resampling—may not handle 44.1 kHz that well. If there is really something better about your system(s) at the higher sampling rate, you could just get foobar to do it in real time while playing music. The built-in SRC is pretty good; the SoX plugin is even better (generally shouldn't be at audible levels, but anyway). That beats having to convert and store larger files.

 

By the way, what with respect to pretty much all audio DAC implementations being oversampling and of course filtering and so on, the steps involved, etc., I'm not so convinced by hypotheses of "improved temporal resolution" here. Of course it depends on the details of what's done.

post #24 of 136
Thread Starter 
Quote:
Originally Posted by mikeaj View Post

As stv014 and Don Hills have mentioned, I'd also be more interested in a comparison with the playback system at equal sampling rate (well, bit depth too, I suppose, to be safe).

 

Some poorer outputs and systems—particularly older ones doing on-the-fly resampling—may not handle 44.1 kHz that well. If there is really something better about your system(s) at the higher sampling rate, you could just get foobar to do it in real time while playing music. The built-in SRC is pretty good; the SoX plugin is even better (generally shouldn't be at audible levels, but anyway). That beats having to convert and store larger files.

 

By the way, what with respect to pretty much all audio DAC implementations being oversampling and of course filtering and so on, the steps involved, etc., I'm not so convinced by hypotheses of "improved temporal resolution" here. Of course it depends on the details of what's done.


I'm going to try Don Hills' recommendation above, is this what you mean as well? -- "starting with a known 24/192 file, downconverting it to 16/44.1 and upconverting it back to 24/192, then comparing it with the original."  

If not, I'm open to all testing suggestions (that I can set up).

I have a brand-new RME Babyface, so whatever the flaws of "older ones" are, I hope this equipment is beyond them.

I didn't know foobar would upsample on the fly, nor about the SoX plugin, thanks for the tip!

I take the point about large files, but I have the time and the space, and so far this is the only process I've got that gives me the improvement I hear--will try foobar realtime alternatives of course.

 

I don't consider "temporal resolution" even close to proven, and I have no way to advance the state of actual scientific knowledge myself--so that's just one guess about why I get a result on spatial details, but nothing on tonal balances.

post #25 of 136
Quote:
Originally Posted by UltMusicSnob View Post ... The scientific question of isolating the pure effects of 192/24 vs Redbook, if that pure isolation is even possible, is interesting, but is not my goal. I want better sound out of the CD's I own, and I can get it by upsampling and playing back through my equipment. ...

 

If your tests show that works for you with your equipment, that's OK. There are sound technical reasons why a DAC might sound better at 192 KHz versus 44.1 KHz. There are also sound technical reasons why it might sound better at 96 KHz than 192 KHz, so you should try that too.

post #26 of 136
Quote:
Originally Posted by UltMusicSnob View Post

If not, I'm open to all testing suggestions (that I can set up).

 

 

Ok, here is a simple test you can do.  Put both versions of the file as the only 2 in your foobar playlist, then use the 'Playback / Random' button (play symbol with the question mark next to it) to start playing a file.  Without looking at the file name on the screen, see if you can accurately pick which version you are hearing, 192 or 44.1.  This is just something you can test for yourself.  You know there is a difference between the 2 files, and you assume one is better than the other, but this will provide you with the confidence that you truly do prefer the 192 version over the 44.1, as you won't know which one is playing to compare as you would with the ABX test.

post #27 of 136
Thread Starter 

I've been kind of suspecting that 96 ought to be noticeable, but also thinking about 88.2, since it's a simple integer multiple. Not expecting to be able to distinguish 96 from 192, but if the filters for 96 interact with the waveform very differently than for 192, it might be possible. Just dawned on me I also ought to look at both mono files instead of stereo, since spatial imaging is my cue.


Edited by UltMusicSnob - 8/15/13 at 8:02pm
post #28 of 136
Thread Starter 
Quote:
Originally Posted by sonitus mirus View Post

 

Ok, here is a simple test you can do.  Put both versions of the file as the only 2 in your foobar playlist, then use the 'Playback / Random' button (play symbol with the question mark next to it) to start playing a file.  Without looking at the file name on the screen, see if you can accurately pick which version you are hearing, 192 or 44.1.  This is just something you can test for yourself.  You know there is a difference between the 2 files, and you assume one is better than the other, but this will provide you with the confidence that you truly do prefer the 192 version over the 44.1, as you won't know which one is playing to compare as you would with the ABX test.

Thanks for the suggestion!

 

This extends my #22 above, looks like. Because I never touched A or B, I never had reference knowledge of which one was playing--the X and Y are always chosen randomly by foobar, without revealing it to the listener. The goal in #22 above was similar: correctly identify without ever comparing to a known audible sample. If I understand your procedure, then no comparison of X to Y will be available either--instead each file will be identified strictly on its own sound characteristics and my knowledge of what to expect.

I do have to correct something here: I can prove I can hear there is a difference between the two files, and I report that one is subjectively better than the other. I don't assume it. That's why I started testing it in the first place, because I experienced an improvement, and wanted to see if I could find the real improvement in blind testing. I can.

OK, I just went and did this one. Keeping a record was difficult, since I don't have the ABX plugin to make the log and keep count. I solved it by moving the foobar window to the left so that the track indicator was off the screen and I couldn't see it. I used screen capture video to prove the procedure was followed. Once I thought I knew what I had, I used a Notepad window to type my decision where it could be seen in the video, and then checked the foobar indicator; finally, I kept the running tally in the Notepad sheet, then hit the Random button again for the next round.

I scored 4 out of 5 in a short test, which is calculated as an 18.8% chance I was just guessing.

Anyway, if there are doubters, it's viewable at http://www69.zippyshare.com/v/63192623/file.html

It's a clumsy procedure, but it does log the results in a provable way.

I'll visit this one again when I've got more time, but no one should doubt at this point that I can tell the difference between the two files, nor that I do in fact prefer the 192_24 playback.

post #29 of 136

I do not know if this is relevant but when i ran the samples through Audacity I found some clipping in the 192/24 sample but not in the redbook sample. I tried to DBT them in FooBar but my DAC does not cope with 192 well it blips a lot making it quite obvious which is which - I also analyzed the volume over the audible range it was not amazingly precise as I had to use 4x the number of buckets for the 192 sample and edit both to just under 20K the average level for the red book was -52.164 and the average level for the 192 sample was - 51.724 again I have no way of knowing how good an estimate this is but even a loudness difference across the board of 0.44 db would be extremely hard to hear. Doing it by range 0 - 1000hz, 0 - 5000Hz and 5000-10000Hz was pretty consistent with a difference of about 0.3 to 0.4db. Obviously above 20k all bets are off - do you have hearing above 20K - that would be a possible variable.

 

has anybody else replicated your DBTS ?


Edited by nick_charles - 8/16/13 at 9:27am
post #30 of 136
Thread Starter 

Wow, thanks for the analysis--what tools did you use?

I don't think I have hearing over 20 kHz--when I run my own tests or use the various files online it disappears for sure around 18kHz at best.

No one has replicated my results (sorry, what's DBTS?--Double Blind Test Series?), at least that I know of.

If you looked through the thread, I've passed the ABX under quite a few conditions now, and on two completely different platforms.


Edited by UltMusicSnob - 8/16/13 at 3:40pm
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Sound Science
Head-Fi.org › Forums › Equipment Forums › Sound Science › Successful ABX testing to hear the difference between Redbook Audio vs upsampled to 192/24