[castleofargh]

  This one the volume wasn't so pumped up - I would say I listened at a little higher than medium volume.  I could listen to the whole piece at that.  
 
It's very subtle and maybe just one note.  I thought maybe there was a slightly different texture on the first note but very difficult to place it with A or B so I gave up on that and focused on the 6th.
 
I seem to do better with short passages. I've also tried just to sit back and "feel" the whole experience without focusing on specific passages but not seen much success.
 
Are others posting more convincing logs for 16 vs 24 (down to 1%?)

but it's all about how loud you go. the difference is the noise floor and should be only the noise floor, so as long as you can go loud enough to make the noise floor audible on the 16bit you'll win. there really is nothing else to this test I think. if you were listening at 90db, the noise floor would be below 1db loud in a room probably saturated with noises around 30db or more+the music itself.
so you either go too loud on a silent passage just so you can win, and we're back to what we were talking about. or else that would suggest another problem not directly related to the track being in 16bit.
that's why loudness is all but a trivial matter.

 

[Greenears]

   
Oh the extended Kirk commentary wasn't about you, just the general problem with the ABX MMO. Do you have a link for that Linn track?

Oh, the commentary is definitely about me.  I'm Kirk.  Since those with superior audio intellect already know the outcome of the KM test, the only way I can get a different outcome is to cheat by reprogramming the ABX tester or similar.
 
http://download.linnrecords.com/test/flac/test192.aspx
 
FYI I also tried the shorter Linn http://download.linnrecords.com/test/flac/recit24bit.aspx  but I couldn't find anything audible to latch onto and my scores were 60-100%.  I gave up after a couple short tries at it.  If you want my logs I can dig them up.

 

[RRod]

Originally Posted by Greenears /img/forum/go_quote.gif
 
Do you think I'm guessing?

 
All depends on your acceptance criterion. Basically, at the beginning of the test you have to specify two of three things:
.the total number of trials
.the false positive rate
.a false negative rate for a given outcome
 
If you chose 75 trials and a 10% false positive rate, then yes you can say "I reject the notion that I am guessing." But if you set a 5% false positive rate, then you can't reject. It is breaking the rules to select your rate after you see results. Me personally, I require 19/25 for myself, since I want <=1% false positives and get fatigued with many more tests, plus it has decent power for picking up true results when the actual probability of hearing a difference is >80%.
 

  Oh, the commentary is definitely about me.  I'm Kirk.  Since those with superior audio intellect already know the outcome of the KM test, the only way I can get a different outcome is to cheat by reprogramming the ABX tester or similar.
 
http://download.linnrecords.com/test/flac/test192.aspx
 
FYI I also tried the shorter Linn http://download.linnrecords.com/test/flac/recit24bit.aspx  but I couldn't find anything audible to latch onto and my scores were 60-100%.  I gave up after a couple short tries at it.  If you want my logs I can dig them up.

 
Oh the first Kirk post was about you definitely, but in the sense that it's fine to people to try to figure out how to beat the system. Still, though, at some point you have to get it set up all right. I'll try the Linn recording myself when I get home.

 

[bigshot]

  Since those with superior audio intellect already know the outcome of the KM test, the only way I can get a different outcome is to cheat by reprogramming the ABX tester or similar.

 
It might not be cheating. You might just be not setting up the test right, or you may be unconsciously allowing bias to alter your results.

 

[limpidglitch]

  Oh, the commentary is definitely about me.  I'm Kirk.  Since those with superior audio intellect already know the outcome of the KM test, the only way I can get a different outcome is to cheat by reprogramming the ABX tester or similar.
 
http://download.linnrecords.com/test/flac/test192.aspx
 
FYI I also tried the shorter Linn http://download.linnrecords.com/test/flac/recit24bit.aspx  but I couldn't find anything audible to latch onto and my scores were 60-100%.  I gave up after a couple short tries at it.  If you want my logs I can dig them up.

 
I gave it a go, doing it like this:
 
sox test192.flac test9624.wav rate -v 96k #I don't think my soundcard supports 192k output.
sox test9624.wav -b 16 -t wav - -D | sox -t wav - -b 24 test9616.wav
 
Result was 6/10.
Only noise I could hear was the background noise of the recording itself, and I could hear no distortion or other artefacts.
 
I then made a diff file and checked the stats; everything looked normal.
I know my own physiological limitations well enough to say that the differences are indeed inaudible.
 

	Left	Right
Peak Amplitude:	-95,87 dB	-95,87 dB
True Peak Amplitude:	-90,72 dBTP	-90,67 dBTP
Maximum Sample Value:	135	135
Minimum Sample Value:	-134	-134
Possibly Clipped Samples:	0	0
Total RMS Amplitude:	-101,26 dB	-101,25 dB
Maximum RMS Amplitude:	-100,82 dB	-100,88 dB
Minimum RMS Amplitude:	-132,53 dB	-132,43 dB
Average RMS Amplitude:	-102,15 dB	-102,15 dB
DC Offset:	0	0
Measured Bit Depth:	24	24
Dynamic Range:	31,71 dB	31,55 dB
Dynamic Range Used:	31,30 dB	31,20 dB
Loudness:	-101,32 dB	-101,28 dB
Perceived Loudness:	-∞ dB	-∞ dB
ITU-R BS.1770-2 Loudness: (selection too short)
0dB = FS Square Wave
Using RMS Window of 50,00 ms
Account for DC = true

 

[Greenears]

@limp: Thanks for running that.  I really appreciate it.  Real runs and results I'm a fan of - whatever the outcome or shortcomings.  Inuendo about my Starfleet cadet history .... not so much. 
 
To be clear for everyone, referenced by you and several other posts, I am not hearing any noise floor.  There was speculation about cranking the volume to the level where I could hear noise.  I have not done that, and no matter what volume I tried I did not hear noise or any other non-music artifact in any of the 12 clips I have, in either 16 or 24.
 
The way I got the result on the above Linn test192 was listening for a certain quality I heard on the  6th note played by the clarinet.  It is difficult to describe, but it is a certain roughness on the front end of that held note, just the first little bit, in the 24b version.  The 16b sounds a little thinner or purer. I'm not sure whether it is the reed or actually distortion within the instrument as that note is played louder.  I also tried listening to the reed "shhh" on the first note, and the sounds of the key pads closing which I can hear quite easily in both the 16 and 24.  Unfortunately I could not zone in on any difference and that probably accounts for about 10 trials.
 
Give it a try.  Maybe you have better equipment. 
 
So the question is am I guessing?  Well I don't really know.  I feel I can recognize that detail, but sometimes it is very hard to pick up.  Then I start to hear it well, and I get a streak of successes, often with quick decisions.  Then it eludes me for a bit.  I saw one poster recommend taking a break every 5th trial.  Good idea but I havnt had the time yet.
 
@RRod:  Honestly I don't have a set criteria.  I'm open for input.  Many posters suggest at least 5% or better 1%.  If I saw any run of >10 trials with 1% result I would call that absolutely not guessing.  I did hit 5% a couple of times with lots of trials so I'm really on the fence.  Note my better runs correspond to passages and pieces where I've heard something specific.  I have other runs on different pieces that look very different, starting at 60% and heading straight to 100% and stayin gin the range.  I can post those logs if someone wants.  The two passages I have some success start at 50 and head to 10 and then stay in the 30-40 range.   I consider any run valid as long as you are trying really really hard to win on every trial. 
 
My current leaning is that chances are high that I am hearing something but it is really subtle.  But my mind may change with more testing.  I did not expect this.  I thought if I heard something once I learned what to look for I would get a 9/10 run.  But that never happened. 
 
And no I am not acting like Kirk.  Kirk reprogrammed because he couldn't face failure.  I can accept failure but I still really really want to know the truth.  I am very curious.

 

[bigshot]

The difference between 16 bit and 24 bit, all things being equal, is the depth of the noise floor. There is no difference in resolution up in the audible range. If you are hearing differences in tone on a clarinet note, it's probably just unconscious expectation bias at work. I'm sure if you did enough tests, it would come out as random chance.

 

[Krutsch]

^^ What about sample rate? I know he is knocking down the bit-depth from 24 to 16, and leaving the sample rate at 96 kHz, but what if the opposite were true: downsampling to 44.1 (or 48) from 96, but leaving the bit-depth alone for the sake of argument. Would it be reasonable to find a passage, section or notes where these subtleties might be audible?
 
I'm sincerely curious, not trolling...

 

[bigshot]

Only if you can hear frequencies humans can't hear.


It's pretty safe to say that if you hear a difference between redbook and high bit/sampling rate, it's the difference in the way your equipment is playing the file that makes the difference, not the sound in the recording.

 

[RRod]

 
@limp: Thanks for running that.  I really appreciate it.  Real runs and results I'm a fan of - whatever the outcome or shortcomings.  Inuendo about my Starfleet cadet history .... not so much. 
 
To be clear for everyone, referenced by you and several other posts, I am not hearing any noise floor.  There was speculation about cranking the volume to the level where I could hear noise.  I have not done that, and no matter what volume I tried I did not hear noise or any other non-music artifact in any of the 12 clips I have, in either 16 or 24.
 
The way I got the result on the above Linn test192 was listening for a certain quality I heard on the  6th note played by the clarinet.  It is difficult to describe, but it is a certain roughness on the front end of that held note, just the first little bit, in the 24b version.  The 16b sounds a little thinner or purer. I'm not sure whether it is the reed or actually distortion within the instrument as that note is played louder.  I also tried listening to the reed "shhh" on the first note, and the sounds of the key pads closing which I can hear quite easily in both the 16 and 24.  Unfortunately I could not zone in on any difference and that probably accounts for about 10 trials.
 
Give it a try.  Maybe you have better equipment. 
 
So the question is am I guessing?  Well I don't really know.  I feel I can recognize that detail, but sometimes it is very hard to pick up.  Then I start to hear it well, and I get a streak of successes, often with quick decisions.  Then it eludes me for a bit.  I saw one poster recommend taking a break every 5th trial.  Good idea but I havnt had the time yet.
 
@RRod:  Honestly I don't have a set criteria.  I'm open for input.  Many posters suggest at least 5% or better 1%.  If I saw any run of >10 trials with 1% result I would call that absolutely not guessing.  I did hit 5% a couple of times with lots of trials so I'm really on the fence.  Note my better runs correspond to passages and pieces where I've heard something specific.  I have other runs on different pieces that look very different, starting at 60% and heading straight to 100% and stayin gin the range.  I can post those logs if someone wants.  The two passages I have some success start at 50 and head to 10 and then stay in the 30-40 range.   I consider any run valid as long as you are trying really really hard to win on every trial. 
 
My current leaning is that chances are high that I am hearing something but it is really subtle.  But my mind may change with more testing.  I did not expect this.  I thought if I heard something once I learned what to look for I would get a 9/10 run.  But that never happened. 
 
And no I am not acting like Kirk.  Kirk reprogrammed because he couldn't face failure.  I can accept failure but I still really really want to know the truth.  I am very curious.

 
Really didn't mean to get you so riled up with the Star Trek ref. Anyway, here are my test results:

 
Here is the spectrogram of the difference between my two files:

 
Using Sox, I did a convert to 16-bit with shaped dither and re-upsampled to 24-bit. No normalization was needed as the peak is only at -6.74.

 

[Greenears]

  The difference between 16 bit and 24 bit, all things being equal, is the depth of the noise floor. There is no difference in resolution up in the audible range. If you are hearing differences in tone on a clarinet note, it's probably just unconscious expectation bias at work. I'm sure if you did enough tests, it would come out as random chance.

75 is a lot of trials.  It's hard to say I need to do more trials.
 
I'm sure you are familiar with the fact that quantization noise is a nonlinear process that cannot be mathematically described with differential equations. The partial analysis shows that at some specific frequencies (harmonics of Fs) the noise energy can get concentrated.  I'm sure you are familiar with the details but http://qtwork.tudelft.nl/~schouten/linkload/adc-tutorial.pdf is a reasonable overview that is repeated in all the textbooks at length, for those that aren't.  Describing Q-noise as something close to Gaussian noise floor works for most of the time, but are there could be exceptions.
 
So if I am hearing something I believe it is most likely subtle phase shifts.  I say phase and not amplitude since I believe that we are not as sensitive to amplitude changes.  Think of playing a major third on a piano.  If you hit the 2 keys with equal pressure compared to 1 key twice as hard as the other, both still sound like a major third and may actually be very difficult to distinguish at all.  But if the piano tuner turns the key even a little on one of the strings, the color of the chord changes immediately, even very subtle shifts can be picked up right away.  We are wired to detect harmonies and dissonances, I don't know why.  
 
It the effect is psychological I am well aware I can fool myself, but I can't fool the machine.  The log is there, that's the test run.  Is it consistent with guessing?  I don't think it's that easy to dismiss.

 

[limpidglitch]

 
@limp: Thanks for running that.  I really appreciate it.  Real runs and results I'm a fan of - whatever the outcome or shortcomings.  Inuendo about my Starfleet cadet history .... not so much. 
 
To be clear for everyone, referenced by you and several other posts, I am not hearing any noise floor.  There was speculation about cranking the volume to the level where I could hear noise.  I have not done that, and no matter what volume I tried I did not hear noise or any other non-music artifact in any of the 12 clips I have, in either 16 or 24.
 
The way I got the result on the above Linn test192 was listening for a certain quality I heard on the  6th note played by the clarinet.  It is difficult to describe, but it is a certain roughness on the front end of that held note, just the first little bit, in the 24b version.  The 16b sounds a little thinner or purer. I'm not sure whether it is the reed or actually distortion within the instrument as that note is played louder.  I also tried listening to the reed "shhh" on the first note, and the sounds of the key pads closing which I can hear quite easily in both the 16 and 24.  Unfortunately I could not zone in on any difference and that probably accounts for about 10 trials.
 
Give it a try.  Maybe you have better equipment. 
 
So the question is am I guessing?  Well I don't really know.  I feel I can recognize that detail, but sometimes it is very hard to pick up.  Then I start to hear it well, and I get a streak of successes, often with quick decisions.  Then it eludes me for a bit.  I saw one poster recommend taking a break every 5th trial.  Good idea but I havnt had the time yet.
 
@RRod:  Honestly I don't have a set criteria.  I'm open for input.  Many posters suggest at least 5% or better 1%.  If I saw any run of >10 trials with 1% result I would call that absolutely not guessing.  I did hit 5% a couple of times with lots of trials so I'm really on the fence.  Note my better runs correspond to passages and pieces where I've heard something specific.  I have other runs on different pieces that look very different, starting at 60% and heading straight to 100% and stayin gin the range.  I can post those logs if someone wants.  The two passages I have some success start at 50 and head to 10 and then stay in the 30-40 range.   I consider any run valid as long as you are trying really really hard to win on every trial. 
 
My current leaning is that chances are high that I am hearing something but it is really subtle.  But my mind may change with more testing.  I did not expect this.  I thought if I heard something once I learned what to look for I would get a 9/10 run.  But that never happened. 
 
And no I am not acting like Kirk.  Kirk reprogrammed because he couldn't face failure.  I can accept failure but I still really really want to know the truth.  I am very curious.

 
I haven't seen much Star Trek, so I've got absolutely no idea what that Kirk stuff is about. My Trek trivia knowledge begins and ends with the popular knowledge that being a red is a risky life, because so many of them die. But in fact the opposite is true, their mortality rate is the lowest of all the groups on board.
 
When listening to that sample I concentrated on the very beginning, thinking that if there was a difference it would be in the form of a slight change in the noise profile. But while the track is pretty quiet, it isn't dead quiet, and what little difference there might have been probably got swamped.
 
I usually take a two-tier approach to these tests. First 10 trials, p-value=0.05. If I pass with good margin (say 1%) I call it a day, but if it's a close call I do 20 or 30 new trials (in two or three sittings), with a stricter criterion.

 

[RRod]

  75 is a lot of trials.  It's hard to say I need to do more trials.
 
I'm sure you are familiar with the fact that quantization noise is a nonlinear process that cannot be mathematically described with differential equations. The partial analysis shows that at some specific frequencies (harmonics of Fs) the noise energy can get concentrated.  I'm sure you are familiar with the details but http://qtwork.tudelft.nl/~schouten/linkload/adc-tutorial.pdf is a reasonable overview that is repeated in all the textbooks at length, for those that aren't.  Describing Q-noise as something close to Gaussian noise floor works for most of the time, but are there could be exceptions.
 
So if I am hearing something I believe it is most likely subtle phase shifts.  I say phase and not amplitude since I believe that we are not as sensitive to amplitude changes.  Think of playing a major third on a piano.  If you hit the 2 keys with equal pressure compared to 1 key twice as hard as the other, both still sound like a major third and may actually be very difficult to distinguish at all.  But if the piano tuner turns the key even a little on one of the strings, the color of the chord changes immediately, even very subtle shifts can be picked up right away.  We are wired to detect harmonies and dissonances, I don't know why.  
 
It the effect is psychological I am well aware I can fool myself, but I can't fool the machine.  The log is there, that's the test run.  Is it consistent with guessing?  I don't think it's that easy to dismiss.

 
The tuner is listening for periodic variations in amplitude caused by cancellation between the untuned notes, with period defined by the difference (beat) frequency. And yes, your log is consistent with guessing if you're getting a p-value of 8%.

 

[Greenears]

   
Really didn't mean to get you so riled up with the Star Trek ref. Anyway, here are my test results:
......
Using Sox, I did a convert to 16-bit with shaped dither and re-upsampled to 24-bit. No normalization was needed as the peak is only at -6.74.

Consider me duly riled.  It wasn't just you.  Think about it: The whole world is on the cusp of going 24-bit crazy (well the audiophile world anyway).  Someone comes along and tries to do some honest ABX testing.  And within nanoseconds I'm accused of constructing some kind of devious Kobayashi Maru no-win test (because of course everyone _knows_ what the right answer is) and if I somehow pass I must have rigged it.  Really.  
 
Anyway back to the testing - so you got 15/25 on the Linn test192 clarinet, right? Do you think you heard anything specific?
 
I just did another run, this time with Cassandra Wilson from HD Tracks sampler.  Same Sox -b 16 not dither no upsampling.  This time however I unchecked "look at the results".  I also discovered that when you click that it doesn't tell you how many trials (?) and then I got interrupted. I definitely intended to do 10 or more.
 
The biggest change was on this I just played A and B a couple times each, then played X then Y from the start and normal med-low volume extended listening conditions.  I just closed my eyes and played about 16-24 bars from time 0 until I had the feel taking in the whole groove & rythm and not trying to focus on anything specific.  Played X once, Y once, whichever I liked better I picked.  Frankly I'm surprised how well I did.  I got interrupted at this point, and I know it is not enough trials.  But still - interesting.  Totally different method than the previous "successes".   This method I would say is much more akin to "natural" listening.  Hmmm.  
 
The ABX is not co-operating.  It would be much more convenient just to get results all over the place and conclude it's a wash and go back to fighting clingons.
 
foo_abx 1.3.4 report
foobar2000 v1.3.6
2014/12/16 17:00:38
File A: C:\Users\Public\Music\HDtracks\Various Artists HDtracks Sampler\HDtracks 2014 Sampler\08-Another Country.flac
File B: C:\Users\Public\Music\HDtracks\Various Artists HDtracks Sampler\HDtracks 2014 Sampler\cassandra16.flac
17:00:38 : Test started.
17:03:43 : 01/01  50.0%
17:04:33 : 02/02  25.0%
17:05:41 : 03/03  12.5%
17:06:34 : 04/04  6.3%
17:07:21 : 04/05  18.8%
17:08:42 : 05/06  10.9%
17:09:34 : 05/07  22.7%
17:09:43 : Test finished.
 ---------- 
Total: 5/7 (22.7%)

 

[bigshot]

I think either your equipment is introducing noise somewhere that wouldn't be there if the conversion and playback were being handled properly, or you are genuinely trying to fairly administer the test to yourself, but are unable to.