[Hybrys]

Quote:

Quote:

Great! Significance (that you reliably can tell one from the other) starts with 95%, that's for example 9 right out of 12.

So you mean 75%?

 

[GreenNight]

320cbr mp3 is enough for me

 

[JxK]

Quote:

So you mean 75%?

I think he was referring to the p value. The p value should ideally be less that 0.05, which indicates that the chance that a result occurred due to chance is less than 5%.

 

[zenpunk]

Quote:

I think he was referring to the p value. The p value should ideally be less that 0.05, which indicates that the chance that a result occurred due to chance is less than 5%.

 I am really confused now. Did he get 93 per cent right or wrong??

 out of how many tries? 

 

[Hybrys]

Quote:

 I am really confused now. Did he get 93 per cent right or wrong??

 out of how many tries? 

140 correct for FLAC out of 150 tries.  I was going to stop at 100, then 120, but the uneven numbers bothered me.  >_>

 

[xnor]

Quote:

140 correct for FLAC out of 150 tries.  I was going to stop at 100, then 120, but the uneven numbers bothered me.  >_>

Uh no more than 25 trials should be done cause the results will get skewed. (and no less than 10, to be able to achieve the confidence level)
 
(You can randomly click through an ABX test with over a hundred of trials and achieve significance...)
So 140 out of 150 doesn't say much really.
 
Hybrys was talking of 93% right out of all trials, I was talking about the confidence level of 95% (for example 9 right out of 12, or 14 out of 20 trials)

 

[falis]

deleted for being redundant.

 

[Hybrys]

foo_abx 1.3.4 report
foobar2000 v1.0.3
2010/05/23 10:52:02

File A: C:\Users\Hybrys\Music\FLAC Project\Anberlin\Lost Songs\Anberlin - 09 - Dismantle. Repair. (Acoustic).flac
File B: C:\LAME\test.wav.mp3

10:52:02 : Test started.
10:52:53 : 01/01  50.0%
10:54:01 : 02/02  25.0%
10:55:09 : 03/03  12.5%
10:56:14 : 04/04  6.3%
10:57:20 : 05/05  3.1%
10:58:25 : 06/06  1.6%
10:59:30 : 07/07  0.8%
11:03:35 : 08/08  0.4%
11:05:39 : 09/09  0.2%
11:08:47 : 09/10  1.1%
11:09:52 : 10/11  0.6%
11:10:57 : 11/12  0.3%
11:12:02 : 11/13  1.1%
11:14:08 : 12/14  0.6%
11:15:13 : 13/15  0.4%
11:17:17 : 14/16  0.2%
11:18:21 : 15/17  0.1%
11:19:26 : 16/18  0.1%
11:20:30 : 17/19  0.0%
11:21:36 : 18/20  0.0%
11:25:46 : Test finished.

 ----------
Total: 18/20 (0.0%)

 

[QRomo]

 
Quote:

Uh no more than 25 trials should be done cause the results will get skewed. (and no less than 10, to be able to achieve the confidence level)
 

 
Isn't that limit of 25 trials suggested to avoid listening fatigue?  If Hybrys spread his trials out over multiple sessions that wouldn't be an issue. Besides, I'd think listening fatigue would make it more difficult to tell the two samples apart, so would skew the results toward randomness rather than significance.
 
 

Originally Posted by xnor /img/forum/go_quote.gif
 
(You can randomly click through an ABX test with over a hundred of trials and achieve significance...)
So 140 out of 150 doesn't say much really.
 
Hybrys was talking of 93% right out of all trials, I was talking about the confidence level of 95% (for example 9 right out of 12, or 14 out of 20 trials)
 

 
This doesn't make sense to me.  Especially "140 out of 150 doesn't say much".  The odds of randomly guessing correctly 140 out of 150 trials are so vanishingly small that it's virtually impossible (something like 10^(-30)).  Unless there's something I'm missing, I'd say that pretty definitively shows Hybrys can hear a difference.
 
I'm also not sure how more trials can make it easier to achieve a significant result randomly.  Sure, the percentage of correct guesses required to achieve a 95% confidence goes down as the number of trials goes up, but the requirement is still that the total number of correct guesses must have a < 5% chance to occur randomly.

 

[xnor]

Quote:

Total: 18/20 (0.0%)

Nice, trying to get my hands on that song atm. How did you encode the mp3? (encoder, params)

 

[Hybrys]

Quote:

Nice, trying to get my hands on that song atm. How did you encode the mp3? (encoder, params)

I can send you each copy, if you shoot me a PM.
 
LAME v3.97
lame -b 320 test.wav
 
That's it.  The first guitar hit sells it almost every time.  The highs aren't quite as crisp.

 

[zenpunk]

That's remain impressive! would be nice to get those two samples and check the encoding. Or could it be the cheap dac on the netbook fails to reproduce the mp3 accuratly??
I can't discern differences on my  STX (DAC) ->Meier Concerto --> Pro 900. Might have to check my hearing 

 

[xnor]

The sample I have right now has some kind of high frequency noise in the background, just at the beginning at a very low level. It sounds a bit different on the mp3. Without that it might not be so "easy" to abx.
 
But I wait for hybrys to send me his sample, so I can recheck that.

 

[Hybrys]

Quote:

The sample I have right now has some kind of high frequency noise in the background, just at the beginning at a very low level. It sounds a bit different on the mp3. Without that it might not be so "easy" to abx.
 
But I wait for hybrys to send me his sample, so I can recheck that.

I had mentioned specifically that it was from the CD, since I did originally torrent it (I'm on a netbook, no CD drive) and found that the quality wasn't perfect.  I then setup my server computer with EAC and ripped it through that.  Both the FLAC and WAV used as a source for the MP3 are both via EAC on that computer.
 
Files inc soon.  Uploading now.  I'm uploading the original FLAC, and the MP3 used (so you can test any encoder differences or irregularities.)
 
Edit: Slight delay, the filehosting service you linked me to errored out on me at the last moment.

 

[zenpunk]

Random static noise are likely to give the game away as most mp3 encoder are optimized to compress music.