|The gist of this little paragraph is that doing it right isn't easy. Something like ABX doesn't begin to approach some of these problems.
yeah, i think you're right. the other thing is, with only 12 evaluations each, it's very possible your one single person who demonstrated some ability might have been able to simply guess correctly.
but i know you have a job doing this kind of stuff... i also vaguely remember some formulas from my days in the stat classes that allows you to calculate when results are statistically significant. i dunno how many out of 12 that that person got right... but if he got all 12 right, well, then that sounds pretty overwhelming (though still possible to flip coins with the same result.) but if it's just like 9/12, well, that's only 3 above odds, which could be accountable by random chance, i would think.
anyway, i would like to discuss this with you further some time. you see, i do want to do such an experiment, but i think your results would be more credible than mine if you happen to find that cables don't make a difference. and of course, mine would be more credible if it was found that cables do. cause you know, i publicly don't believe too much, and you do.
but anyway, that's a subject for another thread in a non "dbt-free" forum.
perhaps we can discuss this further in the Member's Lounge?