Beware: Red Pill ahead! - ABX test: Asus Essence STX II soundcard ( $220 new) vs. Creative Soundblaster X-Fi Xtreme (from $5 second hand): they sound *EXACTLY THE SAME*.

Discussion in 'Sound Science' started by deafmutelame, May 7, 2018.
Page 1 of 4
2 3 4
Next
 
Last
  1. deafmutelame
    [​IMG]


    PLEASE, DO NOT BELIEVE OP


    This is the Sound Science forum: just test this for yourself with your own gear and come back to us with your data.


    Related threads:


    • What's Wrong with the Hi-Fi Scene?: A 700$ NAD Integrated Amp Is Indiscernible from a +20.000$ Mark Levinson Preamp & Dual Monaural Power Amplifier
    https://www.head-fi.org/threads/wha...-preamp-dual-monaural-power-amplifier.877362/


    • ABX test: Sound Blaster X-Fi Xtreme Soundcard vs. Fiio X1 DAP. Indiscernible from each other.
    https://www.head-fi.org/threads/abx...-x1-dap-indiscernible-from-each-other.878783/



    [​IMG]





    Today:


    [​IMG]

    Asus Essence STX II
    https://www.asus.com/us/Sound-Cards/Essence_STX_II/


    vs.



    [​IMG]


    Sound Blaster X-Fi Xtreme Audio PCI-e

    http://support.creative.com/Products/ProductDetails.aspx?prodID=15855&prodName=Sound Blaster X-Fi Xtreme Audio
    http://www.guru3d.com/articles_pages/sound_blaster_x_fi_xtreme_audio_pcie,1.html



    Ok, so I finally received my Asus Essence STX II via Amazon.

    My gear:


    [​IMG]


    Integrated Amplifier: NAD C370

    This amp was reviewed by Stereophile here:

    NAD C370 Integrated Amplifier
    Chip Stern | Jan 4, 2002
    https://www.stereophile.com/integratedamps/633/index.html

    Also featured at What Hifi:

    NAD C370 - The best stereo amplifiers of the 21st century
    https://www.whathifi.com/features/best-stereo-amplifiers-21st-century



    [​IMG]


    Speakers: B&W 602 S3

    That Was Then… B&W 602 S3 review
    https://www.whathifi.com/features/was-then-bw-602-s3-review



    My headphones:

    [​IMG]

    Etymotic Research ER4XR: $349 via Amazon
    https://www.etymotic.com/consumer/earphones/er4-new.html


    After matching loudness levels with a sine wave AND with a testing track, AND after properly syncing the track playing on the soundcard with the track playing on the X1 (within less than a second of difference between them) and after connecting my headphones to the headphone output in the NAD C370:


    But before that I wanted to make another small test.

    • I connect the Soundblaster X-Fi Xtreme Soundcard to one free slot in my motherboard AND to the CD level input in my C370 amp.
    • I connect the Asus Essence STX II Soundcard to another free slot AND to the AUX level input in my C370 amp.

    I match the loudness on both as best as I can playing a sine wave first and some testing tracks then.

    Then I can freely and easyly switch between one channel and the other, without transition or clicks or any kind of noise.

    I use the following tracks:

    Limehouse Blues - Arne Domnérus, George Ridel - Jazz at the Pawnshop
    Rebecca Pidgeon - Spanish Harlem - The Raven
    Time - Pink Floyd - Dark Side of the Moon
    Right Off - Miles Davis - A tribute to Jack Johnson
    Bruckner Symphony nº 4 - I - Bewegt, nicht zu schnell
    Chameleon - Herbie Hancock - Head Hunters
    Thinking of a Place - The War on Drugs - A Deeper Understanding
    Blue Train - John Coltrane - Blue Train
    The Sidewinder - Lee Morgan - The Best of Lee Morgan


    I ask a friend to switch between the Soundblaster X-Fi Xtreme and the Asus Essence STX II for me with my eyes closed.

    I test this with the B&W 602 S3 speakers *AND* with my Etymotic ER4XR IEMs.

    Result #1: I cannot discern whatsoever between them. Neither can him.

    This time I ask some other members of my family and the result is THE SAME: *NOBODY* of us can discern between one and the other.

    Result #2: I am going to return the Asus soundcard and I will enjoy my music with a $5 soundcard knowing that now I can LAUGH at all the so called "audiophiles" out there (and I don't mean any particular one in this forum, of course).


    As I said before:

    So... is 90% of the HI-FI market based on snake oil and in the fact that the human brain has a 3-4 seconds auditory memory at much?

    https://en.wikipedia.org/wiki/Echoic_memory


    BONUS TRACK:
    I BET NO ONE CAN DISCERN A $550 CHORD MOJO (...or any other DAC) in an ABX test from a $5 Soundblaster.


    [​IMG]



    [​IMG]
     
    Last edited: May 7, 2018
  2. pinnahertz
    While the results are not entirely unexpected, you do realize you have not done an ABX test here, right? You have several biases at work in your methodology.

    There are also several issues with the matrixhifi.com test relating to analysis. What they've done is taken potentially valid (though the data set is a bit small) statistical data and created forced binary results based on an arbitrary threshold of 90%. It's that kind of thing that gives ABX testing a bad name.
     
  3. deafmutelame
    Please tell me about them and I will repeat the test to the best of my ability. Thanks.
     
  4. pinnahertz
    Since you are doing an AB test, not ABX, you know there must be a change in what you are listening to. You've already pre-biased your expectations of "no change" by studying and quoting the results done on matrixhifi.com, so your expectation is that there will be no difference. Your expectations are underscored by how you state your conclusions. Your test is also sighted, even though you have your friend switching, because you don't have the unknown X choice. If you were performing an ABX test, your number of trials is far too small.

    When you make this statement, "I BET NO ONE CAN DISCERN A $550 CHORD MOJO (...or any other DAC) in an ABX test from a $5 Soundblaster", you're further adding bias, both to yourself and others. A true ABX test of two DACs is not trivial at all, and requires quite a complex setup with a few technical hurdles just to accomplish it at all, not the least of which is simultaneous playback through both DACs. The implication is that anyone can do it, and no differences will be found. Those are biased expectations, and simply get in the way of true research.

    Please understand, I'm not saying I would necessarily expect any particular outcome, but rather, you need to do real science before stating conclusions like that with such conviction.
     
  5. deafmutelame
    In other words, let me see if I have understood this properly (english is not my first language, sorry):

    there's no way I am going to get a "true" result for, apparently, after having studied the matrixhifi.com results I expect that there should be no difference, hence I'm already biased and there's nothing I can do about it, is that it?

    Also, unless I repeat this 100 times with 50 people each, the results are not valid neither, right?
     
    Last edited: May 7, 2018
  6. bigshot
    There is a difference between doing a test to serve your own purposes, and doing a full rigorous test for scientific publication. You decide how far you want to go, but I think you've probably proven that if a difference exists, it's probably too small to make any difference at all for the purposes of listening to music in the home.
     
    Harry Manback likes this.
  7. deafmutelame
    ^ This.
     
  8. pinnahertz
    You need to do an actual ABX test, where X=A or B, randomly assigned, and switching time between any choices is less than 20ms, ideally less than 10ms. For comparing devices, this requires an ABX comparator, like the one matrixhifi.com used.

    I have not said anything about how many trials or people are required. But usually you need at least 16-32 trials for each tester. Increasing the test music increases the required number of trials.

    Reading the matrixhifi.com site creates a bias. ABX testing controls that bias.
     
  9. ev13wt

    Well, you can test yourself. Its just much easier with a friend and a switchbox.
    If you do 10 or 20 runs, it should be fine. The idea is that you don't know what you are listening to. You could also place a really bad component into the mix, like an old onboard with real loud noises when you move the mouse or something.

    Software wise, you can do this with the foobar comparator plugin https://hydrogenaud.io/index.php/topic,107354.0.html


    For the record, I have biased myself a couple of times in the past, its really quite easy.

    And we really need to find something where you hear a real difference :)
     
    Last edited: May 7, 2018
  10. deafmutelame
    Ok, thanks, I understand now. I am using the 2 channels in my amp as a switch: as explained in the procedure above I usually sync both tracks within 1 second of each other, but there are other lucky times when I hear "absolutely" no delay between one track and the other. Seamless.

    Plus, with our echoic memory of 3-4 seconds, I presume that going down to <10 ms isn't even that shouldn't even be an issue, but I'm not an expert at that...

    So if I need a "true" ABX comparator then I'm out, for obvious reasons.

    Also, I'm afraid I'm not going to meet the "randomly assigned" requirement unless I get a Geiger counter connected to my PC...


    [​IMG]



    :sweat:


    (j/k)
     
    Last edited: May 7, 2018
  11. pinnahertz
    You cannot perform at an ABX test without an anABX comparator of some sort. With the proper kind you can do it yourself, but the OP doesn't have one.

    You cannot ABX hardware with Foobar.

    A real ABX comparator includes X randomization, and a means of reporting the identity of X when collecting the data.

    This is why I said that ABX testing of hardware devices is not trivial. It requires special switching hardware and a means of level matching. You cannot do it with a friend operating your input selector!
     
    ev13wt likes this.
  12. deafmutelame

    Let me re-quote myself explaining why the ABX switch might not be necessary:


    I am using the 2 channels in my amp as a switch: as explained in the procedure above I usually sync both tracks within 1 second of each other, but there are other lucky times when I hear "absolutely" no delay between one track and the other. Seamless.

    Plus, with our echoic memory of 3-4 seconds, I presume that going down to <10 ms isn't even that shouldn't even be an issue, but I'm not an expert at that...


    Let me requote bigshot, please:

    ^ This.
     
    Last edited: May 7, 2018
    ev13wt likes this.
  13. bigshot
    I think he's found out what he needed to find out anyway.
     
  14. deafmutelame
    Indeed: and as a result I'm going to save $220. At least.

    For I'm not going to even consider getting the Chord Mojo or ANY other DAC for the matter.

    Readers of this thread should not believe my conclusions and should just TEST this for themselves.

    For god's sake, I realized there was NO differece after switching the channels within the very first 5 times of trying this... and NO ONE I have shown this has been able to tell any difference whatsoever.

    In other words: you would expect at least some obviously noticeable difference between a $5 soundcard and a $220 one, wouldn't you?

    :hushed:
     
    Last edited: May 7, 2018
  15. pinnahertz
    Just understand that what you are doing is a sighted and biased AB test. You are free to draw your own conclusions, but they cannot be excepted as scientifically proven because of your test methodology, and your tests are nothing like it was done on that other site.

    My only expectation bias would tend to agree with your conclusions but if I were actually testing I would have to control my own biases before I could except any conclusions.
     
Page 1 of 4
2 3 4
Next
 
Last

Share This Page