The RMAA (RightMark Audio Analyzer) Source and Audio Device Measurement Thread [Overview of Measurements in Post #3, Tutorial in Post #2]

Discussion in 'Sound Science' started by hifichris, Mar 1, 2016.
  1. HiFiChris Contributor
    1. Introduction: http://www.head-fi.org/t/800208/#post_12386876
       
    2. Tutorial/Guide: http://www.head-fi.org/t/800208/#post_12386879
       
    3. Overview of all Measurements: http://www.head-fi.org/t/800208/#post_12386883
     



    All right folks, I have been planning to start this thread/topic long ago here on Head-Fi as I was an active guy on that topic in a German online audio community I was a very active and leading member in the past years, but was just too lazy and lacked the required time to write the introduction, tutorial and so on here. What I am talking about are (quite simple) audio/source device measurements that everybody can do with his/her (Windows) computer (given that the input that is used for recording has got a as flat as possible input response).


    The first post should be seen as an introduction to the topic, the second is a tutorial of how to easily measure source devices (and how to see whether one's audio interface is suitable or not) and the third an overview of measurements in this thread and on other websites.

     
    For a DAP/DAC/Amp, the most important things for me are a strictly flat unloaded frequency response (not that difficult to achieve, but very few devices still fail to reproduce it), as low as possible noise floor (hiss), low output impedance (lower than 1 Ohm is a must for me, as I am using mainly multi-driver in-ears with varying impedance response over their frequency spectrum) and no roll-off in the bass with a low impedance load (the lack of caps in the signal path is required for that and fortunately most recent devices don't fail this test) plus load stability (difficult to measure with my rig, but more about that in a second). That's an objectively (sonically) perfect audio device for me and I actually don't care too much about its price point, marketing or even my subjective perception of its sound quality, as when properly volume-matched (but not fully blind-folded), if numerous devices are about comparable in terms of the just-mentioned things, the difference in "sound quality" is extremely minute, close to inexistent in most cases (and to hear them, really hard concentration is required). And the still remaining differences can mostly be explained by things like noise floor, noisefloor shaping, THD (sometimes) and stereo crosstalk (given that the output impedance is low, of course). And still, the differences are extremely subjective under that conditions without a real blind-folded test. I'm not saying that it isn't legitimate to still speak about subjectively perceived audio quality (and I even still do it either, but it is important to note that it is my personal, highly subjective perception), even if they can be placebos based on the mood, new toy syndrome, build quality, marketing, attitude towards the product/maker, expectations or whatever (which however doesn't matter for our brain which will still translate those to perceived superiority).
    That's why when I buy/review/get my hands on an audio device like DAPs, Amps and DACs, the first thing after initial listening for a few minutes (very rarely, I even skip that part) I do is to put a Micro SD card with the RMAA test file into the device and play it back (unloaded) while it is being recorded with my soundcard, which takes just barely more than one minute. Then it can be analysed in the RMAA application (freeware for Windows). Afterwards, I perform a loaded measurement for which I am using the Ultimate Ears Triple.Fi 10, as it has been a common model for performing RMAA measurements in that German audio community (with that, there was also a good consistency between various enthusiasts of us who posted their measurements). A different multi-driver in-ear with varying impedance response that reacts critically in combination with a device that has got a high output impedance will be sufficient as well, of course.
    Having done that, I then know whether the unloaded FR is flat, I can calculate the output impedance based on the IEM's impedance response and the FR output deviation with the just measured source device or just see whether the deviation is really small (which means that the OI is very low; I also know how the FR deviation of the T.F10 looks with certain OI values like 0.1, 0.5 and 1 Ohm) and I can see whether there is a roll-off caused by caps in the signal path.
    The hiss test then just requires listening and the load stability is nothing I can really accurately measure, as my audio interface (actually just my laptop's audio input which is however perfectly flat even down to below 20 Hz, where many interfaces unfortunately show a roll-off) is somewhat too simple to even get comparative results of SNR, Crosstalk etc. within my own measurements (and most importantly, besides FR and FR deviation, measurements between different devices that are taken to record the signal shouldn't be made, as they will show results that can't be compared because each interface is not equally precise).
    The rest of perceived audio quality besides those objective things is then highly subjective and mostly irrelevant in many situations, and as I mentioned, the differences are (objectively regarded) pretty small when the comparisons are being made properly volume-matched (which cannot be done by ear) with most gear.


    My main point of this thread is just to get an overview of various source/audio devices' unloaded and loaded frequency response measurements (also to get an idea of their output impedance and behavior with critical multi-driver in-ears).
     
     
    So with that being said, I invite you guys to join the party and to post your or link others' RMAA measurement plots of audio devices (though I fear that there are not many Head-Fiers who do so, as of all users I only know that @ClieOS and @shigzeo (the latter with a super precise pro interface) are doing RMAA tests).
     
    castleofargh and getclikinagas like this.
  2. HiFiChris Contributor
    TUTORIAL:
     
     
    Requirements:

    In theory, now one could start measuring right now, but it is recommended to check whether everything that is necessary is available:

    Hardware:

    computer (running Windows), audio input (-> soundcard/interface), 3.5 mm (male) to 3.5 mm (male) cable, 3.5 mm stereo to 2x 3.5 mm stereo y-split, DAP (, headphone amplifier in case you want to amplify the signal with hardware instead of with Audacity)

    Software:

    Audacity, Right Mark Audio Analyzer (RMAA)

     
    Realisation:

    Before beginning with the actual measurements, it is recommended to test whether the used equipment (soundcard/audio-interface) records signals linearly and therefore is good for making measurements.

     
     
    Determination of the Input’s Suitableness:

    The first step would be to check whether the microphone input records the incoming signal linearly. Therefore, besides the soundcard, the 3.5 mm to 3.5 mm cable, a DAP that is already known to have a perfectly linear output (such as the FiiO X3 or SanDisk Sansa Clip Zip, but most other DAPs also output a linear signal), Audacity and RMAA are required.


    S1.png

     

    For preparation, all DSPs or other sound-shaping programs should be turned off. Using the computer’s microphone input, one should “disable all sound effects” in the Windows system settings (hardware & sound -> sound -> recording -> microphone -> enhancements) and set the pre-gain to maximum. Other DSPs (like “Dolby Home Theater”) should of course be disabled as well.

    Let’s go on with the software: in RMAA, one has to generate a test signal by clicking on “Generate WAVE” (the created file is called “Test signal (44 kHz 16-bit).wav”). This file has to be copied to a DAP that is known to have a flat output signal response.

    Once that is done, Audacity can be started, along with connecting the DAP’s output to the computer’s audio input. The DAP’s volume has to be fairly high, then one can start recording in Audacity and then play the file on the DAP.
    Once the file was completely played back, the recording can be stopped in Audacity. Then, the file can be exported and analysed in RMAA, wherefore “Analyze WAV” has to be selected.
    Ideally, a window should pop up and the yellow marked button can be clicked to see the input’s recorded frequency response:


    rmaa-1_545236.jpg

     
    In case RMAA says that the volume level was too low, the signal could either be amplified with a (linearly measuring) headphone amplifier or digitally in RMAA, then the file should be recognised.
    Ideally, the recorded frequency response should then look like this in RMAA:


    RMAAideal.jpg

     
    If there is “sizzle” or distortion, that is an indication for an incorrect measurement, wherefore it should be repeated until a proper looking curve can be seen.
    If the graph doesn’t show a flat curve (and for example has got a roll-off in the lows), it means that the input doesn’t record linearly, wherefore buying another audio-interface should be considered.

     
    In the best case, you should get a flat graph like above. Now you can measure with real headphones and in-ears as connected load. If your soundcard/interface shows some nonlinearities or a roll-off in the lows (which many do), you can either live with that (but please note that this is caused by the soundcard's input and not the tested source device in case you want to publish the plots) or search for a different soundcard/interface.
     
     
    Loaded Measurements:

    Please note that the drawing below was created for another tutorial of mine (for the Vibro Veritas), so the "Soundcard Output" should be replaced with "DAP Output".

     
    The measuring setup is pretty much identical to the unloaded one, but it requires a headphone or in-ear (best a sensitive low-impedance multi-driver with varying impedance response to check for a roll-off in the lows and effects of high output impedance) that will be plugged into the other output of the stereo to double-stereo y-split.

    S3-0.png

     
    The measuring process is as usual, but as the IEMs are connected and the volume is set to a fairly high level, the test-tones are clearly audible through the IEMs when they are laying on the table next to you. Amplifying the signal in this case is practically unavoidable (either by a linearly playing headphone amplifier in the signal path between the y-split and the audio input (and NOT between the output and the IEMs, as the signal that reaches the audio input and not the one that reaches the headphones has to be amplified) or by using the digital software amplification in Audacity, but when using the software amplification, please set the level to few dB below 0 dB FS, as RMAA very often doesn't like a fully modulated signal).
    After exporting the file, you can analyse it in RMAA, just like with the unloaded measurement.
     
    Congratulations - you can now perform simple measurements with RMAA.
     
    earfonia, tenedosian and h1f1add1cted like this.
  3. HiFiChris Contributor
    OVERVIEW:



    DAPs/Digital Sources (Battery-Powered):

    Google Asus Nexus 7 (first generation): http://www.head-fi.org/t/800208#post_12388760

    Apple MacBook Air (late 2013 model generation): http://www.head-fi.org/t/800208#post_12388837

    iBasso DX90: http://www.head-fi.org/t/800208#post_12388853

    iBasso DX80: http://www.head-fi.org/t/800208/#post_12485433

    iBasso DX200 ("AMP1" module): http://www.head-fi.org/t/800208/#post_13278998

    iBasso DX200 ("AMP2" module): https://head-fi.org/t/800208/#post-13487008

    Cowon Plenue M2: http://www.head-fi.org/t/800208/#post_12937168

    Cowon Plenue 2: https://www.head-fi.org/t/800208/#post-13475137

    Cowon Plenue D: http://www.head-fi.org/t/800208/#post_12485445

    Cowon Plenue S (by @CraftyClown): http://www.head-fi.org/t/800208/#post_13000991

    Cowon Plenue J: https://head-fi.org/threads/2.800208/page-14#post-13878637

    Questyle QP1R (by @CraftyClown): http://www.head-fi.org/t/800208/#post_12999784

    BlackBerry Q10: http://www.head-fi.org/t/800208#post_12388862

    Samsung Galaxy S7 (by @CraftyClown): http://www.head-fi.org/t/800208/#post_13001880

    Apple iPhone 4 (with InEar StageDiver SD-2 as load): http://www.head-fi.org/t/800208/15#post_12426540

    Apple iPad Mini 2 (by @hakuzen): http://www.head-fi.org/t/800208/#post_13041224

    Google Pixel XL (by @reginalb): http://www.head-fi.org/t/800208/#post_12962566

    Nexus 6, AK300 and Sony ZX1 (by @reginalb): http://www.head-fi.org/t/800208/#post_12963308, http://www.head-fi.org/t/800208/#post_12678630 and http://www.head-fi.org/t/800208/#post_12678536

    Apple iPod Nano 1G: http://www.head-fi.org/t/800208/15#post_12426566

    AGPTek C05: http://www.head-fi.org/t/800208/#post_12485411

    AGPTek C1: http://www.head-fi.org/t/800208/#post_12485419

    Benjie S5 (by @hakuzen): http://www.head-fi.org/t/800208/#post_12985545

    HiFiMan HM-602 (by @reginalb): http://www.head-fi.org/t/800208/#post_12689814

    HiFiMan MegaMini: https://www.head-fi.org/f/threads/800208/page-13#post-13559831

    Luxury & Precision L3 (single-ended): http://www.head-fi.org/t/800208/#post_12689843

    Luxury & Precision L3 Pro (single-ended): http://www.head-fi.org/t/800208/#post_12937137

    FiiO X3 (first Generation): http://www.head-fi.org/t/800208/#post_12689874

    $1.49 No-Name China MP3-Player: http://www.head-fi.org/t/800208/#post_12689904

    Hidizs AP60: http://www.head-fi.org/t/800208/#post_12937149

    Sony Xperia XA (by @danspy): https://head-fi.org/f/threads/800208/page-12#post-13532614

    Walnut V2 (by @hakuzen): http://www.head-fi.org/t/800208/#post_13107087

    xDuoo X3: http://www.head-fi.org/t/800208/#post_13441180

    xDuoo X3 (by @hakuzen): http://www.head-fi.org/t/800208/#post_13075638

    Xiaomi RedMi Note 2 (by @hakuzen): http://www.head-fi.org/t/800208/#post_13041235

    Xiaomi RedMi 4A 4G: https://head-fi.org/threads/800208/page-14#post-13879875

    Zishan DSD (by @hakuzen): http://www.head-fi.org/t/800208/#post_13107744


    DACs:

    Beyerdynamic A 200 p: http://www.head-fi.org/t/800208#post_12388780

    Chord Electronics Mojo: http://www.head-fi.org/t/800208/#post_12803183

    LH Labs Geek Out IEM 100: http://www.head-fi.org/t/800208#post_12388787

    (Likely) Still embargoed, small DAC (already posting the measurements so I don't forget about it at a later state, as I did with its predecessor):: https://head-fi.org/threads/800208/page-14#post-13899612

    M-Audio Micro DAC 24/192 (by @h1f1add1cted): http://www.head-fi.org/t/800208/#post_13160900

    HiFiMe 9018d: http://www.head-fi.org/t/800208/#post_12485460

    Shozy Lancea: http://www.head-fi.org/t/800208/#post_12605264

    Sabaj Audio D1: http://www.head-fi.org/t/800208/#post_13025483

    Schiit Fulla (by @reginalb): http://www.head-fi.org/t/800208/#post_12689382

    Stoner Acoustics UD125: https://www.head-fi.org/f/threads/800208/page-13#post-13536299


    Headphone Amplifiers and DAC-Amps (devices that can be used as a DAC but also feature an analogue input):

    iFi Audio micro iDSD: http://www.head-fi.org/t/800208#post_12391886

    FiiO A1, no load (by @danspy): https://head-fi.org/f/threads/800208/page-12#post-13525793

    FiiO Q1 MkII: https://head-fi.org/threads/800208/page-14#post-13909256

    GAD mini A1 (by @hakuzen): http://www.head-fi.org/t/800208/#post_13107386

    Leckerton UHA-6S.MKII: http://www.head-fi.org/t/800208/15#post_12426577

    iBasso D14 (by @hakuzen): http://www.head-fi.org/t/800208/#post_12995932

    Elekit TU-HP02: http://www.head-fi.org/t/800208/post_12485428

    Cayin C5 (by @hakuzen): http://www.head-fi.org/t/800208/#post_12994488

    Topping NX1a (by @hakuzen): http://www.head-fi.org/t/800208/#post_12994430

    JDS Labs C5D (by @hakuzen): https://head-fi.org/threads/800208/page-13#post-13774048


    Other:

    Avantree Clipper Pro (Bluetooth DAC/receiver for headphones), unfortunately unloaded (by @danspy): https://head-fi.org/f/threads/800208/page-12#post-13526533

    Mass Fidelity Relay (Bluetooth DAC): https://head-fi.org/f/threads/800208/page-12#post-13526659

    Nintendo 3DS: http://www.head-fi.org/t/800208#post_12388869

    Shinrico SHD5 (desktop audio player): http://www.head-fi.org/t/800208/#post_13076139

    Sony PlayStation Vita: http://www.head-fi.org/t/800208/#post_12485400

    PGA2311 Pre-Amplifier: http://www.head-fi.org/t/800208/#post_12485437

    Zoom UAC-2 Audio Interface (by @yuriv): http://www.head-fi.org/t/800208/#post_12872369

    Realtek ALC888 Soundcard (by @hakuzen): http://www.head-fi.org/t/800208/#post_13019304

    C-Media 6206 Audio Interface (by @hakuzen): http://www.head-fi.org/t/800208/#post_13016538
     
    Last edited: Dec 12, 2017 at 9:31 AM
    stalepie and thehexagonal like this.
  4. HiFiChris Contributor
    Google Nexus 7 (first generation):


    Loaded measurement with the UE Triple.Fi 10 and Sennheiser Amperior:

    fr.png
     
     
    Loaded measurement with the t.bone EP-7:

    fr.png
     
     
    Drastic FR deviations with the multi-driver in-ears and the very slight deviation with the Amperior indicate extremely high output impedance. Definitely not multi-driver friendly.
     
  5. HiFiChris Contributor
    Beyerdynamic A 200 p:

     
    Comparative measurement w/ and w/o load:
     
    fr.png
     
    OI should be somewhere between 1 and 2 Ohms. Good but could be still a bit better.
     
  6. HiFiChris Contributor
    LH Labs Geek Out IEM 100:
     
     
    Comparative measurement w/ and w/o load, also showing the digital filters:
     
    fr.png
     
    Very good! The OI should be around tiny 0.2 Ohms. Fully multi-driver friendly.
     
  7. HiFiChris Contributor
    Apple MacBook Air (late 2013 model, 256 GB SSD, 13"), UE Triple.Fi 10 as load:


    fr.png

    Should be below 0.25 Ohms. Would be an excellent source for multi-driver in-ears if there wasn't the high amount of hiss and the rough volume steps.
     
    Last edited: May 7, 2017
  8. HiFiChris Contributor
    iBasso DX90:
     
     
    Headphone output, loaded and unloaded:
     
    fr.png
     
    Perfect - just about 0.2 Ohms OI. The extremely minor roll-off at 20 Hz is unnoticeable. Basically hiss-free (hiss so low that it is only audible with extremely sensitive in-ears in a quiet room, and even then it is just barely possible to tell it apart from the blood noise), the most hiss-free low OI source I have come across. Perfectly fine-grained volume steps over the whole attenuation range. Perfect standalone machine for super sensitive multi-driver IEMs.
     
     
    Unloaded measurement of the Digital Filters:
     
    fr.png
     
  9. HiFiChris Contributor
    BlackBerry Q10:
     
     
    Comparison of loaded and unloaded FR:
     
    fr.png
     
    Not bad for a mobile phone, though the unloaded FR could be metrologically flatter in the upper range. With around 2 Ohms, the output impedance is okay.
     
  10. HiFiChris Contributor
  11. castleofargh Contributor
    a lot more people use RMAA on occasions(like let's say, me ^_^), and it had it's glory days some years ago (I sure wish the markuskraus measurements were still available somewhere with all the old DAPs measured into the tf10 [​IMG]).
    RMAA has a few annoying limitations/problems(at least the free version, IDK about the pro one), but also a lot of really cool info when done right and the ADC isn't much worst than the source.
    now for frequency response comparisons like you're doing to look for impedance or caps acting out on some IEMs, I agree that it's very fine for everybody with any soundcard.
    maybe people will want to play around with the resolution settings of the recording soundcard? as 16/44 can looks really not flat on some soundcards inputs. it's still good enough for side by side comparisons but that's about it, so guys don't hesitate to record at a different value than the one being tested on the DAP/AMP/DAC, to find what works best on your soundcard.
     
     
    ClieOS stepped up a bit with a QA400 now. /mesojealous
     
  12. Samueru Sama
    Why don't you post the whole report as an screenshot? 
     
    I use this: https://chrome.google.com/webstore/detail/full-page-screen-capture/fdpohaocaechififmbbbbbknoalclacl?utm_source=chrome-app-launcher-info-dialog
     
    Example: http://i.imgur.com/wLZcTb4.png
     
  13. HiFiChris Contributor
     
    Because except for capturing the FR and FR deviation (fortunately my soundcard has a perfectly flat input recording response without roll-off in the lows), it isn't much suitable. Very inaccurate and rough, without showing a difference in let's say IMD between two recorded devices where other sites that do RMAA measurements do. Only if there is a drastic difference between two devices, a difference is visible. It's just way too inaccurate for everything except for the FR. That's why. [​IMG] 
     
     
    Anyway, I'm going to upload some more measurements later on. Still got more than a dozen on my USB sticks, cloud storage and on Head-Fi inside of the reviews (plus many devices that I once measured bud haven't saved the files/plots).
     
  14. HiFiChris Contributor
    iFi Audio micro iDSD:


    Some basic FR measurements:


    xBass enabled and disabled (unloaded):

    [​IMG]


    Digital Filters in DAC Mode (unloaded):

    [​IMG]
    Standard ≙ Fast Roll-Off, Minimum Phase ≙ Slow Roll-Off, Bit-Perfect ≙ Non-Oversampling


    Impact of the different Gain Levels' Output Impedance on the Frequency Response (Connected Load: Triple.Fi 10):

    [​IMG]





    -> Lowest output impedance (~ 0.1 Ohms) with disabled iEMatch, around 1.4 Ohms with iEMatch set to Ultra Sensitivity and somewhere between 4 and 5 Ohms (?) with iEMatch set to High Sensitivity.
     
    Last edited: May 7, 2017

Share This Page