Chapter 2 – Anatomy of a Review – The Equipment
Feb 2, 2016 at 5:16 AM Thread Starter Post #1 of 32
Joined
Jan 30, 2011
Posts
13,315
Likes
24,298
So you’ve started reviewing your own gear, and you’re enjoying the process, and you want to take it to the next level.  What do you use, how do you use it, and why would you need to consider specialised equipment?
 
Welcome to chapter two – where I’ll outline the equipment I use, and give a brief overview of how I use it, some of the advantages and pitfalls, and most importantly why I think measuring equipment is essential in a well-balanced review.  And for my fellow reviewers – please contribute your own thoughts, tips, and advice on what you use and why.  Also feel free to disagree with anything I suggest.  We’ll treat this as a learning curve together.
 
NOTE – please excuse the photos this time – I can’t reproduce this in a proper setting (light-box), so what you’re getting is pretty amateurish compared to my usual output!
 
So let’s skip back a bit a few years, to when I first started reviewing my own gear. The first portable amp/dac I owned after joining Head-Fi was the original E7.  I’ve since amended the original review, but I can still remember waxing lyrical about the increase in clarity, sound-stage, bass definition, and all other manner of magic.  It was a great improvement over my iPod Touch by itself – fantastic!
 
Fast forward a few years, and I am now a bit older and a lot wiser.  I’d actually love to repeat the testing with the knowledge gained in the last few years, and the very first thing I would do is volume match the two items I was comparing.  And not volume matching by ear, or by iPhone app – I’d use a proper SPL meter.  The one thing I cringe at a lot of times is the mantra “trust your own ears”.  Every time I see this spoken on the forums, my immediate thoughts are:
“No problem with trusting my ears mate – it’s the brain that is a little trickier”.
 
You see – one of the biggest issues I’ve found as I’ve gained more experience is that our wonderful brains have a habit of adapting to the environment it is in, and we (humans) also seem to have some common physiological responses to external stimuli, and no matter how hard we try to override them, they still influence what we think we’re hearing.
 
Measuring the q-Jays - X3ii + E17K + splitter + attenuator
 Volume matching the Adel U6 - X3ii + E17K + splitter + attenuator
 
So – the first real tool for my reviewing was a proper calibrated SPL meter.  They aren’t expensive – but they are essential if you want to make a proper comparison.  Want to test my “theory” out?  You’ll need someone to help you, two similar sounding amps (or headphones or any piece of equipment really).  You’ll also need an SPL meter.  Firstly – match by ear and do a sighted evaluation of which sounds better.  I almost guarantee that one will be subjectively more vivid, more detailed, have better bass definition, more sound stage – all the things that we read often in a review – especially comparing a straight source with the same source adding a portable amplifier. I’d further guess that unknowingly one of the two is louder than the other – probably by as little as 1 dB, but maybe more. You see, we perceive something slightly louder as being (among other things) - more vivid, more detailed, having better bass definition, more sound stage etc.  It’s not our fault – it’s how we’re wired.  It is what makes us human.  To test it, all you need to do is volume match properly with the SPL meter (to within 0.1-0.2 dB), and then get your friend to conduct a blind test where you don’t know the source.   See if you can tell which source is which volume matched, and then get your partner to randomly increase the volume of one marginally, and see if your impressions change.
 
Splitter + attenuator
Adel U 6 mounted in a simple coupler
 
So what have we learnt? If we’re not accurately volume matching before making a comparison of any equipment, there is a high likelihood that our impressions of the advantages of a product will be heavily weighted towards the product which has more volume.
 
Why do we always read that an amplifier always makes our headphones sound better – even when the specifications of the headphone suggest that amplification won’t help?  Because usually we’ll listen to the amplified version louder – we just won’t know it.
 
So we’ve covered the why – let’s look at what I use and how.
 
My tool of choice is equivalent to this Amazon product.  To use it, I mostly use an IEM and a coupler (just a simple tube which snuggly fits the meter and the IEM).  I then play a calibrated test tone (usually 1 kHz – you can create and down load these here http://onlinetonegenerator.com/). Then it’s just a simple matter of adjusting the volume on both sources until they match exactly.
 
And this introduces the next (very cheap piece of equipment) I use for comparing IEMs.  It’s just a simple two way splitter, and a Shure volume attenuator.  When comparing IEMs, simply attach the splitter to the output of your source, and the louder of the two IEMs (usually the one with lower impedance and/or higher sensitivity) gets the attenuator in-line.  Set the volume so the two match, secure the volume control with a bit of blue tack, recheck again, and then you can fast switch between two IEMs very quickly.
 
Why do we need to do this? Well we already know why they must be volume matched, and the fast switching is simply because our echoic memory is actually pretty poor.   Anything over about 10 seconds and we’re not really able to remember the differences we’re trying to compare.  So the quicker we can compare the better.  Eventually I’d like to find a nice splitter box with integrated volume controls on each channel – that would make the whole switching exercise a lot better.
 
One final thing – when testing IEMs that might have very different signatures – what frequency do you use?  Most of the time I use the standard 1 kHz test tone.  Occasionally I’ll use a 2 kHz tone (depending on the frequency shape of the earphone).  We know that our hearing is generally most sensitive at the 1-3 kHz range, its where we perceive sound as being the loudest, and I guess this is because it is where a lot of enunciation of human speech lives.  So it makes sense for comparison to make this the central point for setting the volume.  Then we can comparatively describe differences in the rest of the signature because we have a fixed common point for both IEMs / sources.
 
But one thing I usually do now before volume matching is first measure the frequency response (if I’m comparing IEMs) because sometimes you’ll have one IEM with a dip at 1 kHz and another with a peak or boost.  So in this case if I set the volume at 1 kHz – it would skew the results at other ends of the frequency spectrum.  In cases like that – choose a frequency response point where both IEMs are consistent, without having the entire frequency domain completely out of whack.
 
This leads us to frequency measurement (and a host of other measurable information).
 
Probably the centre of my testing lab now revolves around my measurement equipment for IEMs and also for other equipment.  Unfortunately it’s not uber precise gear, but for the hobbyist it is perfect – low cost, reasonably accurate, and as long as we look at the data comparatively, enough to give us a read on differences.
 
Vibro Veritas couple and mic
You can just see the mic at the bottom of the coupler
 
So what you need here is a soundcard (input and output channels), a coupler to mimic an ear canal, and some software to measure the results and provide meaningful visual output (graphs).  As a soundcard, I use a very cheap USB micro unit from Startech which actually works a lot better than my on-board, and was recommended by the makers of the coupler. The coupler is the very affordable Vibro Veritas – which is essentially a microphone mounted at the bottom of a coupling mechanism.
 
The software is the truly excellent ARTA software suite which allows me to play selected sine waves / pink noise / white noise in set patterns though the sound card, to the earphones being tested, and then they sit in the coupler which is attached to mic input and recorded.  The ARTA software then analyses the result, and produces an absolute plethora of information about what you’re measuring.  Your job is the to interpret the output, and put the results into meaningful commentary in the review.
 
Startech sound card
Sound card to E11K to Adel U6 connected to Veritas connected to sound card
 
OK Brooko – so what do you measure and why?
 
Let’s start with the easy part – IEMs.  The obvious thing we can measure is the frequency response, but we can also output CSD plots which allow us to see decay in certain frequencies  over time, and identify any particular problem areas which could indicate things like driver ringing or unwanted resonances.  There are also screens for impulse response, burst decay, and other reports – but I don’t know enough about those tools yet to be able to utilise them.
 
[size=inherit][/size]
Recording in progress
Adel U6 frequency response
Adel U6 CSD
 
I have also set up an extremely basic contraption to measure full sized headphones – but it is very rudimentary, nowhere near accurate, and has nothing to allow for interaction with the outer ear.  What it has allowed me to do is measure comparative bass and mid-range though, which has been handy when working out what a little EQ in the lower end of the spectrum does.  The funny thing about this rudimentary set-up was that I was talking to Jude a couple of weeks ago on the phone about setting up the blog, and we were talking about measuring systems.  It led to discussion about what he was planning and what I was doing (with measuring gear), and at one stage he mentioned that some people were simply using a mic with headphones stretched across a box, and how wrong that was.  We didn’t have time to cover what I was doing – but I’m sure he’ll be amused when he sees my rig.  I basically use Veritas nestled in a pad of foam with a soft outer surface (bubble wrap carefully covered with shrink-wrap.  I then have the headphones sitting across a box, but it is measured to be the exact width as my head (with the pad included), and it delivers surprising consistent results.  What I’d love to do is get a working model of an outer ear – that would be neat to incorporate!
 
[size=inherit][/size]
Veritas and my "face pad"
Measuring HD600
This set up is exactly the width of my head
 
Still using the same sound card and ARTA I can also perform loopback tests with DACs, DAPs and amps, and this can give me really good data on frequency response (is it flat from 20-20?), and things like gain, and bass boost or tone controls.  For example, I used this on the E17K test, and it gave me some really good results - allowing me to map the tone controls, and the volume of the gain.  The beauty of this, is I can also measure filtering roll-offs, and even EQ effects on headphones.
 
I have also been able to measure THD and IMD – but I stopped measuring that recently because I realised the limiting factor was the equipment – most competently designed audio gear is either at or below the threshold of the little Startech unit I’m using.  What I really need is a better external sound card – but that is going to have to wait until much later in my reviewing “career” I think.
 
Example of a loopback test - testing the E11K
Close up of the Veritas coupler in action
 
So – I have the gear, and the reports – what are the pitfalls, or some of the issues I’ve had?
 
By far the biggest issue is the accuracy of the coupler – it gives me raw data, but it isn’t compensated to any standard.  When I compare IEMs I have to the same ones Tyll has measured though (on properly calibrated gear), my raw data isn’t a million miles away from his (see example). I was all set to try and introduce a calibration profile, and discussed this with Jude.  He gave me some really good advice – and suggested not to change anything.  His reasoning is that every measurement rig is different in some form or other.  As long as when I’m measuring IEM’s I explain the limitations of the system, and mainly use it for comparative data (i.e. comparing different IEMs), then there is no need to change what I’m doing for now.
 
Other issues include consistency – basically to get the coupler to have consistent fit all the time I need to use foam tips – and sometimes I have to switch the settings around to get a decent reading.  I’ve also often measured and measured for hours sometimes to make sure the read get is consistent.  For me it is not usable unless I can get practically the same result again the very next day.
 
Then there is also the sound card’s relatively high impedance (I think it’s about 6-7 ohms), but I’ve overcome that with introducing the E11K into the mix.
 
This is starting to get overly long now so I’ll try and shorten it a bit, and save some for another blog entry.
 
The final pieces of equipment I use are a home-made light box (please don’t laugh, I built it myself, and it works surprisingly well) coupled with a relatively cheap Canon EOS series DSLR camera. The light box is getting too small, so mark two is well overdue to be built.
 
[size=inherit][/size]
Home made light box
The tiny Fiio A1 in the light box
What you see on the reviews
 
I also use MS word to write the reviews, Paint Shop Pro to manipulate the images, and the formatting I actually do inside Head-Fi’s editor software.
 
The final piece of the puzzle is my own ears.  They are the equipment I use most, but the ones I trust the least.  I always listen first (now), and then use the measurement information to either confirm, clarify, or understand what I’m hearing.
 
So let’s call it quits this week on that note.
 
If you have questions about the gear, fire away. I’ll answer where I can, and others can chime in and help me.  If there are others who review – what equipment do you use and why?
 
For next week (or the next chapter anyway), I want to take a closer look at the basics I apply to my own reviews, what I include and why. 
 
Feb 2, 2016 at 12:48 PM Post #2 of 32

BloodyPenguin

Headphoneus Supremus
Joined
Aug 10, 2011
Posts
2,759
Likes
4,531
Wonderful post!  Thank you so much for a the valuable information. 
cool.gif
 
 
Feb 2, 2016 at 1:45 PM Post #4 of 32
Joined
Oct 6, 2013
Posts
12,740
Likes
32,567
Thumbs Up is not going to be enough, so in addition I would like to post a Big Thank You for the time and the effort you put into this Blog chapter!  Very educational!
 
Feb 2, 2016 at 2:04 PM Post #6 of 32
Joined
Mar 4, 2014
Posts
6,978
Likes
14,993
Location
the land of cheese and beer
Haha, Paul we have a lot of the same gear!

My light box came from massdrop, and I absolutely love it. It's so nice to be able to snap photos without having to worry about lighting and shadows. There's things I know I could do to up the ante for my photo game, but to he completely honest I'm not shooting a swimsuit model, I'm trying to give potential customers an idea of what to expect.

I don't want to dictate your blog, but I hope one of the next entries will cover the format and style in how we cover things.

Cheers Paul, great stuff
 
Feb 2, 2016 at 2:33 PM Post #7 of 32

h1f1add1cted

1000+ Head-Fier
Joined
Apr 23, 2014
Posts
1,033
Likes
712
Location
Germany
@Brooko
 
Great write up. But what do you think instead of using this SPL meter and other tools, to use a multimeter and play on all sources the same 1 kHz sinus 0dBFS test signal for scaling all to the same voltage, to get same loudness level for listening, I recommend a 3,5mm adapter with a small resistance like 30 ohms for better results, this works also very fine, I use this way always for me reviews.
 
 
Feb 2, 2016 at 3:22 PM Post #8 of 32
Joined
Jan 30, 2011
Posts
13,315
Likes
24,298
Thanks all - I appreciate the feedback.
 
My light box came from massdrop, and I absolutely love it. It's so nice to be able to snap photos without having to worry about lighting and shadows. There's things I know I could do to up the ante for my photo game, but to he completely honest I'm not shooting a swimsuit model, I'm trying to give potential customers an idea of what to expect.

I don't want to dictate your blog, but I hope one of the next entries will cover the format and style in how we cover things.

 
Link to the lightbox Vince?  I probably have to make a "Brooko MK2" due to the size I want (about 3 times as large as current) - but wouldn't mind seeing what is out there that others are using.
 
And the next entry will be on what I include (order, format) and why.  That's where I'll want your guys input as well.
 
  @Brooko
 
Great write up. But what do you think instead of using this SPL meter and other tools, to use a multimeter and play on all sources the same 1 kHz sinus 0dBFS test signal for scaling all to the same voltage, to get same loudness level for listening, I recommend a 3,5mm adapter with a small resistance like 30 ohms for better results, this works also very fine, I use this way always for me reviews.

 
I've actually often thought of doing that.  Confession time though - I have a multimeter, but not sure how to set up using it.  I'd be very grateful if you could post (do it here if its OK) an explanation step by step of how you do it.
 
Also - anyone know how ot measure output impedance on a DAP or amp - what would I need, how would I do it?  
 
Feb 2, 2016 at 4:34 PM Post #9 of 32

h1f1add1cted

1000+ Head-Fier
Joined
Apr 23, 2014
Posts
1,033
Likes
712
Location
Germany
  I've actually often thought of doing that.  Confession time though - I have a multimeter, but not sure how to set up using it.  I'd be very grateful if you could post (do it here if its OK) an explanation step by step of how you do it.
 
Also - anyone know how ot measure output impedance on a DAP or amp - what would I need, how would I do it?  

 
Hi sure, please see below, incl. how to measure output impedance of devices.
 
 
PART 1 - To measure all gear the same loudness levels, regardless with earphones or headphones are using.

An example gear A and gear B both different DAPs.

You are using for the test a i.e. 1 kHz sinus 0dBFS WAV signal on the player end.

You simply connect a 3,5mm adapter as dummy load with a small resistance like 33 ohms and you connect your multimeter on both ends (plus and minus):
 

Here I'm measure the FiiO X7for my review.
 

Here my adapters, one adapter without resistance and one with a small 33 ohms resistance.

On gear A and you play the mentioned test signal and you may have as result i.e. 100 mV. You do the same on gear 2, but now you change the volume settings so long that you reach the same 100 mV. Now both devices are the same loudness levels for comparing with the same earphones or headphones, as long you don't change for now the volume settings on both devices.
 
For "later" you could take notes on the different voltage levels and from this time you can "calculate" on different volume settings quite easy like, this example, again we using the same method but now we no "matching" the voltage now we only taking notes which output level you reach at 10% volume setting, 20%, 30% and so on. This for the other gear too. Simple example:
 
Gear A, volume setting 10% = 50 mV
Gear A, volume setting 20% = 100 mV
Gear A, volume setting 30% = 150 mV
...
Gear B, volume setting 10% = 25 mV
Gear B, volume setting 20% = 50 mV
Gear B, volume setting 30% = 75 mV
...
In this case gear A on 10% volume setting has the same loudness level as gear B on 20% volume setting.
 
If you want up to set gear A and B with the same loudness level up to +/- 0,5 dB differences, you need to measure really well i.e.

10^(0,5/20) = 1,059
like 100mV to 94,4 ... 105,9 mV
 
PART 2 - To measure the output impedance of any device.

Gear X with unknown output impedance, like a in my example a mobile phone.
 
Again you are using for the test a i.e. 1 kHz sinus 0dBFS WAV signal on the player end.
 
You simply connect a 3,5mm adapter without any dummy load and you connect your multimeter on both ends (plus and minus), like before.
 
On gear X and you play the test signal and you may have as result i.e. 0,2445 V. Now you swap the 3,5mm adapter to the one with the dummy load of 33 ohms and you don't change any volume setting on gear X and you play the test signal again you may have now a result of 0,2126 V.
 
Now its time for maths (lol I hate this speech):
 
"Voltage result without load" minus "voltage result with load" = voltage difference:
 
0,2445 V - 0,2126 V = 0,0319 V
 
Next will be "voltage result with load" divided by "the calculated voltage difference" from the step before = ratio:
 
0,2126 / 0,0319 = 6,66
 
Next will be the "used resistance load" divided by "the last calculated ratio" = output impedance:
 
33 / 6,66 = ~ 4,95 so around 5 ohms output impedance of gear X.
 
I'm still a "noob" with this stuff, but this worked for me very well, I tested devices which I know already the output impedance (from tech specs) and it was always near to 100% accurate, so It can't be wrong. But take note you need a good multimeter which is accurate enough, if not the results will be not really realistic finally.
 
Feb 2, 2016 at 5:38 PM Post #10 of 32
Joined
Jan 30, 2011
Posts
13,315
Likes
24,298
Thank you - that is brilliant.  Now just need to buy a resistor, butcher a couple of plugs and test it out :)
 
Feb 2, 2016 at 11:23 PM Post #11 of 32

castleofargh

Sound Science Forum Moderator
Joined
Jul 2, 2011
Posts
9,325
Likes
4,700
just 2 little points about measuring impedance of a source, first as mentioned, a multimeter that does well at 1khz or more is not something as obvious as it seems. the cheap stuff is made to deal with 50 or 60hz at 110 or 220v for home stuff. not to measure 0.01V or to deal with rapid frequencies.  the right stuff is actually pretty expensive (what a surprise
mad.gif
!). you can still manage with el cheapo multimeter (I sure do), but it takes some trial and errors and some known references to find out the limits and accuracy of your measures. on mine 200hz is fine, 500hz is close and 1khz seems to be a little less accurate. the joke being that a product can very well have a different impedance at 50hz and 1khz, so you can either trust your multimeter ($$$$$$) or you must have lot of stuff to test that have already been measured.
 
also remember to measure pretty close to the device, else you add whatever cable to the measurement. I have tiny little croco to croco plugs with 10cm of cable they're close to 1 ohm resistance each so including them in the reading would be a problem.
 
 here are a few stuff I measured that you or others may want to use as guinea pigs(but I wouldn't trust myself too much, given my gears, even my resistors are crap, so take those as a wild reference ^_^):
sony A15 3.34ohm (measured 4.04ohm on headphoniaks)
fiio X1 1.78ohm (measured at 2.3ohm again on headphoniaks)
O2 0.58ohm (measured 0.54 by JDS/nwavguy)
 
 
-the second point is that IMO(you guys do whatever you like), it's better to measure into 2 different loads(2 different resistors) and go for something like Z=(V1-V2)/((V2/R2)-(V1/R1)) . no need to stress the small stuff you just write your entire formula with your measured values inside this guy http://www.wolframalpha.com/ and voila!
why 2 loads instead of 1 measure loaded and 1 unloaded? well, some devices just don't deal all that well with no load(10kohm or more). usually it would still work fine, but not always. with 2 loads unless you end up with one load being too small and the source clipping on it like crazy, there is little reason for the values to too wrong.
 
Feb 3, 2016 at 8:11 PM Post #12 of 32

castleofargh

Sound Science Forum Moderator
Joined
Jul 2, 2011
Posts
9,325
Likes
4,700
 I'm having second thought about the shure attenuator thing you're using to match the levels for 2 IEMs(not that this is exact science anyway). isn't it simply adding resistance to the circuit? if so it would change the signature of many IEMs.
if you don't have yet a multimeter, you can test it with the veritas, changing the volume setting on the shure attenuator and seeing if some IEMs with wild and low impedance(most multi BA drivers) have a change in overall frequency response due to that added resistance from the attenuator.
 
Feb 3, 2016 at 9:30 PM Post #13 of 32
Joined
Jan 30, 2011
Posts
13,315
Likes
24,298
Hadn't though of that.  I'll try as soon as I get a chance (bogged with work at the moment again)
 

Users who are viewing this thread

Top