How to Compare Headphones - Factoring out loudness
Jul 4, 2015 at 10:58 AM Thread Starter Post #1 of 25

mikesale

100+ Head-Fier
Joined
Mar 15, 2014
Posts
178
Likes
38
At the last Canjam in Denver I compared a pair of HD650s on Balanced vs Unbalanced cables... there was a clearly discernible difference, but to my ears it could have been loudness, or clarity. 
 
When comparing any two sets of headphones I would like to know the actual details on "how" to setup, measure and set levels so that I'm listening at the same "volume" when comparing sets. 
 
I've done 4-6 hours of reading and research trying to get to the actual "how" without enough detail. Could be that I suck at search, could be that I'm making a mountain out of a molehill. Most efforts seem to have other goals (e.g. protecting hearing).
 
Where I am today: I have an iPhone meter app and would be willing to purchase an inexpensive SPL meter or mic for the iPhone to measure, here are the core propositions and questions:
 
Here's what I think I already know:
  1. Generate a tone (or two?) through the headphone and measure it with SPL meter with headphone #1 & #2 and note the amp volume knob location
    1. Please do correct me if I'm wrong here
 
Here's what I think I don't know how to do:
  1. Set the meter configuration (A, C, slow, fast, range, etc.)
  2. What tones to use? (associated with meter configuration)
    1. Can I just use my audition music sample and capture peak?
  3. How to move the volume back and forth accurately enough for me (basic testing shows I can tell 1db differences 100% of the time, but 0.5db about 65%)
    1. Simple visual marker like a post-it strip?
    2. Always have to measure when switching?
  4. How to align the meter/microphone to the headphone cup?
 
I'm not looking to publish a paper or spend $100s, I'm looking to compare headphones to both train my ears further and make better purchasing decisions. Ideally I'd love to have something quick and portable that I could take to local meet-ups and vendor shows.
 
Jul 4, 2015 at 11:56 AM Post #2 of 25
telling if someone's balanced amp gain spec is hiding 2x more V gain, +6 dB SPL is possible with acoustic SPL meter measurements
 
but finer matching to below 1 dB is hard
 
for amps the more accurate test is the voltage output when driving the headphone
 
 
Tyll did "publish" headphone SPL measurement and does use pro equipment - his repeatability article shows its not easy - the same headphones vary by more than the just noticeable amplitude vs frequency response every time they are replaced on your head (well at least on his dummy's head)
 
home headphone SPL measurements aren't likely to be good enough to match levels to ABX thresholds of even the same headphone from repeatability of remeasuring the same headphone if you reset them on your jig/microphone adapter
 
and the bigger difference in frequency response between different models means matching at one frequency, or even with pink noise and a weighting curve is really only "advisory"
 
a conclusion is that all different mode headphones do sound different - and there's not much hope of any agreeing that "the loudness" has been set correctly between them for the comparison
 
 
 
its still good to do some objective loudness matching than not at all - the "loudness effect" on perceived frequency response is smaller the closer you get in overall loudness
 
it is pointless to comment on 2 headphone's apparent bass/mid/hi balance if you listened at 10-20 dB different average SPL between them due their sensitivities and the amp's gain and setting
 
but a pro like Bob Katz claims he can use the perceptual changing frequency balance with loudness to tell a 1 dB level difference
 
Jul 4, 2015 at 12:35 PM Post #3 of 25
OK, not absolutely sure, but I think you're saying my goal is a lost cause, but I should still try to match loudness as best I can.
 
Hence, I return to the question: How?
 
Jul 4, 2015 at 3:53 PM Post #5 of 25
To level match for what you are doing, use a 1k sine wave and match the level as close as you can. Since you are doing a relative measurement and you are using the same equipment it should give you a good result. If you are doing this on different days I would do all the measurement again to confirm nothing has changed. The problem with headphone getting a consistent measurement. You have to get the microphone the same place each time to get a fairly consistent measurement. If you have a 1/2" measurement mic an old CD can be used as a simple measurement jig by going to the hardware store getting two o-rings that fit snugly around the mic body push one down about 1/4" put the microphone through the hole in the cd and put the second o-ring trapping the cd between the o-rings.
Hold the CD with the microphone to the headphones centered in the ear cup you should be able to get more consistent measurements.
I would measure everything you do a few times to check for consistency.  You could possibly tape your phone to a piece of cardboard or a cd if you figure out where the microphone is.
 
Jul 4, 2015 at 5:02 PM Post #6 of 25
Jul 4, 2015 at 7:44 PM Post #7 of 25
If you are always using the same headphones, you don't need a SPL meter or a mic.  Just use a voltmeter.  Modify a connector so that you can measure across the 2 terminals to one can.
 
Jul 5, 2015 at 4:51 AM Post #8 of 25
  At the last Canjam in Denver I compared a pair of HD650s on Balanced vs Unbalanced cables... there was a clearly discernible difference, but to my ears it could have been loudness, or clarity. 
 

 
 
I'll bet money this was another one of those dreadful casual audiophile evaluations that have the following inherent problems:
 
(1) Audiophile Sighted Casual Evaluations are not useful evidence because they are not tests. That is, they do not involve comparison to a fixed, reliable standard.
 
(2) Audiophile Sighted Casual Evaluations are not useful evidence because they involve excessively long switchover times, which makes them highly susceptible to false negatives because they desensitize the listeners.
 
(3) Audiophile Sighted Casual Evaluations are not useful evidence because the do not involve proper level matching, which makes them highly susceptible to false positives because people report the level mismatches as sonic differences.
 
(4) Audiophile Sighted Casual Evaluations are not useful evidence because they do not involve listening to the identical same piece of music or drama within a few milliseconds, creating false positives because people report the mismatched music as sonic differences in the equipment.
 
(5) Audiophile Sighted Casual Evaluations are not useful evidence because they constantly reveal the true identity of the UUTs to the listener, creating false positives because people report their prejudices and preconceived notions as sonic properties of the equipment
 
 
Measurements made with a SPL meter are almost as bad because SPL meters, relying on acoustical measurements lack the precision and reliability and stability that are required for this kind of test.
 
As Speedskater pointed out, a voltmeter is the tool of choice.
 
Jul 5, 2015 at 1:50 PM Post #9 of 25
   
 
I'll bet money this was another one of those dreadful casual audiophile evaluations that have the following inherent problems:
 
(1) Audiophile Sighted Casual Evaluations are not useful evidence because they are not tests. That is, they do not involve comparison to a fixed, reliable standard.
 
(2) Audiophile Sighted Casual Evaluations are not useful evidence because they involve excessively long switchover times, which makes them highly susceptible to false negatives because they desensitize the listeners.
 
(3) Audiophile Sighted Casual Evaluations are not useful evidence because the do not involve proper level matching, which makes them highly susceptible to false positives because people report the level mismatches as sonic differences.
 
(4) Audiophile Sighted Casual Evaluations are not useful evidence because they do not involve listening to the identical same piece of music or drama within a few milliseconds, creating false positives because people report the mismatched music as sonic differences in the equipment.
 
(5) Audiophile Sighted Casual Evaluations are not useful evidence because they constantly reveal the true identity of the UUTs to the listener, creating false positives because people report their prejudices and preconceived notions as sonic properties of the equipment
 
 
Measurements made with a SPL meter are almost as bad because SPL meters, relying on acoustical measurements lack the precision and reliability and stability that are required for this kind of test.
 
As Speedskater pointed out, a voltmeter is the tool of choice.


I don't disagree, and I didn't migrate to balanced cables for my 650s and I don't see myself ever spending much for a replacement cable *ever* (just too much physics in college to live with myself after a decision like that). 
 
The difficulty I'm faced with is more about preference and personal evaluation: How to do your best to factor out stupid obvious factors like volume. I've tested myself carefully and I clearly show a bias for slightly louder music. When looking to see if I want a new pair of headphones, **how** do I do what I can to factor that out? What's the best I can reasonably do in a show setting? in my home?
 
Jul 5, 2015 at 1:51 PM Post #10 of 25
  If you are always using the same headphones, you don't need a SPL meter or a mic.  Just use a voltmeter.  Modify a connector so that you can measure across the 2 terminals to one can.


See Subject -- whole point here is to compare headphones and remove volume bias from the process as much as is reasonably possible in a show/meet-up environment.
 
Jul 5, 2015 at 2:13 PM Post #11 of 25
How accurate is a voltmeter that is not RMS? I seem to remember measuring a signal level with a cheap voltmeter and it being way off in the range of 20%. Even if you have and RMS meter what frequency does it measure unto the cheaper one don't have full specifications. One of my meters goes to 1MHz and other can barely do 1KHz. The other issue is you have have access to the conductors. If you already own a good voltmeter I would use that if you are willing to open up the cable or make a break out adapter.
 
Jul 5, 2015 at 2:37 PM Post #12 of 25
 
  If you are always using the same headphones, you don't need a SPL meter or a mic.  Just use a voltmeter.  Modify a connector so that you can measure across the 2 terminals to one can.


See Subject -- whole point here is to compare headphones and remove volume bias from the process as much as is reasonably possible in a show/meet-up environment?


some of us had this talk a few days back when talking about how to do a proper review. and headphones can't be tested objectively that way because of how much difference there is in frequency response from one to another headphone. so pick whatever method is fastest and more practical, so that at least it doesn't take 2 full minutes to check the levels. it would be sad to lose all accuracy in our memory just for something that doesn't completely solve the loudness differences.
 
Jul 5, 2015 at 2:45 PM Post #13 of 25
 
some of us had this talk a few days back when talking about how to do a proper review. and headphones can't be tested objectively that way because of how much difference there is in frequency response from one to another headphone. so pick whatever method is fastest and more practical, so that at least it doesn't take 2 full minutes to check the levels. it would be sad to lose all accuracy in our memory just for something that doesn't completely solve the loudness differences.


Completely agree, full objectivity is by no means possible or even close to possible. That's not the questions, the question is how much can you do in a show or home setting that makes sense. 
 
Or are you saying nothing worthwhile is possible?
 
Jul 5, 2015 at 2:54 PM Post #14 of 25
  Thank you dprimary:
At this point I'm thinking either get the "Neewer® USB Digital Handheld Sound Noise Level Meter Tester with A and C Frequency Weighting for Musicians and Sound Audio Professionals" meter to act standalone or this mic for the iPhone. Any reason to go one way or the other (or a completely different path?)
 
Given this and I think I now understand a bit more about mic placement, what about meter settings? A or C? Fast or Slow? Range? 


I would get the sound meter but be aware it could be off by a few dB and often are. However it should be consistently off in its measurements as long as you use the same meter on all the headphones they should end up the same. Two people with two meters and you could be way off. The down side of phone app's is keeping them calibrated in absolute SPL. They can be excellent if you have a calibrator, preamps and microphone. But that is starting at about $1k. A and C curves are both the same at 1KHz fast or slow should not matter for a sine wave but slow will average and might help be more consistent. 94dB would be a standard calibration level, but it is pretty loud might freak out people running their head phones that loud. You want to be louder then the room noise maybe 80dB is a good starting point.  
 
Jul 5, 2015 at 3:10 PM Post #15 of 25
 
I would get the sound meter but be aware it could be off by a few dB and often are. However it should be consistently off in its measurements as long as you use the same meter on all the headphones they should end up the same. Two people with two meters and you could be way off. The down side of phone app's is keeping them calibrated in absolute SPL. They can be excellent if you have a calibrator, preamps and microphone. But that is starting at about $1k. A and C curves are both the same at 1KHz fast or slow should not matter for a sine wave but slow will average and might help be more consistent. 94dB would be a standard calibration level, but it is pretty loud might freak out people running their head phones that loud. You want to be louder then the room noise maybe 80dB is a good starting point.  


Thank you!
 

Users who are viewing this thread

Back
Top