Great job purrin with the study, it has value and actually your sample size was for the most part suitable. Once you have 30 participants the central limits theorem comes into play and you can assume the population was normally distributed. Just being picky here, but because you had a sample it is actually the standard error of the mean, not the standard deviation which is a population statistic, but I suspect you know that. I really liked that you tried this, and I think designing a more robust study would actually be a great thing. I would love to work on such a study for use as a thesis paper I need to do soon, but it has bearing on my career area so sadly I will have to study something else. Really interesting line of questioning so keep it up if you can.
Recent Images In This Thread
- productAKG K701 Studio Headphonestagged by System, 6/19/12
- productAudeze LCD2 Planar Magnetic Headphonestagged by System, 6/19/12
- productbeyerdynamic Tesla T1tagged by System, 6/19/12
- productGrado RS1tagged by System, 6/19/12
- categoryHeadphonestagged by System, 6/19/12
- productSennheiser HD 650 Headphonestagged by System, 6/19/12
- productSennheiser HD 800 Headphonestagged by System, 6/19/12
- productUltrasone Edition 8tagged by System, 6/19/12
Related Forum Threads
- Headphones sensitivity, impedance, required V/I/P, amplifier gain Last post on 1/25/15 at 9:06am in Sound Science
- Virgin - Needs Advice - Never had a Headphone Last post on 2/8/13 at 9:43am in Headphones (full-size)
- Do I buy Beyerdynamic T1 or Hifiman HE500 as upgrade solution to my current HD650/K701 combo? Last post on 10/26/13 at 2:27am in Headphones (full-size)
- OPEN headphones with the TIGHTEST bass! Last post on 12/30/14 at 1:17am in Introductions, Help and Recommendations
- 5-year Re-Review: Audio-Technica ATH-AD2000 Last post on 3/8/12 at 11:45pm in Headphones (full-size)
Head-Fi Buying Guide (Over-Ear Headphones) 2
Last edited: 12/5/14
- Head-Fi Buying Guide (Ultra-High-End Headphones (Summit-Fi))Last edited: 12/8/14
- Head-Fi Buying Guide (Over-Ear Headphones)Last edited: 11/28/14
- Comparisons of LCD-2 Rev. 1 and Rev. 2Last edited: 9/19/11
- Headphone ImpedanceLast edited: 8/18/12
This is my first review on head-fi. I had humble beginnings in my path as an audiophile that only officially started once I got the wonderful AKG K550 and grew more conscious of the link between...
Before I start my review, I would like to Thank RIVA Audio team for a review sample in exchange for my honest opinion. Thanks to the original Jambox, going back a few years, wireless...
http://www.raysamuelsaudio.com/ It takes a lot of confidence to name your product after the Lockheed SR-71 Blackbird. If your going to honor the legendary mach 3+ strategic reconnaissance...
Every 6 months I've had the pleasure of talking to Shinobu Karaki of Aurorasound at the Tokyo headphone festivals. As an ex-Texas Instruments employee he has come up with some interesting...
For me, the Audeze EL-8Cs satisfy a long awaited two year search for a closed headphone that I can use at work that sound just as good as my home theater setup. I enjoy them so much, that I cannot...
Do Objective Headphone Measurements Correlate to the Audiophile's Subjective Experience? - Page 5post #61 of 1906/16/12 at 11:32pm
Gear mentioned in this thread:post #62 of 1906/16/12 at 11:56pm
I think it's because Purin has put so much effort into these measurements, far beyond that of all except a very few, that people get the impression he is only interested in measurements. The Corvette is known for its LS3 motor, but that's not to say that the suspension doesn't exist.post #63 of 1906/17/12 at 12:04amQuote:Originally Posted by rhythmdevils
Which is why you are very vocal about liking two headphones which have issues showing up on your CSD plots. Because you trust your ears first, and use these graphs as a tool to further understand what you are hearing. LOL that quote actually proved the opposite point as intended...
People want things in such easy, black and white packages. I'm not sure why you are supposed to choose between graphs and ears. This whole thread is about how they relate to each other.
to be serious for a moment, i think you've misconstrued my post. i was quoting purrin in a light-hearted way to show that headphones can seemingly defy bad measurements by sounding good to the guy who is actually taking them. the implication being that purrin ultimately relies on his ears rather than his measurements to judge the merits of a headphone.
Edited by shimmer n roar - 6/17/12 at 1:39ampost #64 of 1906/17/12 at 12:07amQuote:Originally Posted by purrin
A pool of thirty headphone hobbyists were asked to rate on a five point scale seven headphones. They were not told of the purpose of this study. The headphones in the study were selected because of their popularity in order to increase the likelihood that the hobbyists were familiar with them, and hence could provide a rating. Each hobbyist was asked to rate the headphone as follows without any consideration to price:
I read through the thread so far and nowhere has it been mentioned how you conducted the study, which I think is relevant info. You said there was a pool of 30 hobbyists and that they're all active Head-Fi members, who rated 7 headphones. IMO this begs the question, how did all 30 members listen to all 7 headphones? Were any headphones shipped around from one Head-Fi member to another, or was there any listening done in a micro-meet setting? Were there any instances of people who rated a headphone but didn't currently have it on hand? Because if there were people who gave ratings without having at least a recent listen to a headphone in their own home, I'd consider their rating completely invalid.
Logistically speaking I don't see how this study was achieved without extensive time being put into it and how the feedback given from the 30-member pool has any credibility at the same time.
Also I've seen mentions of "purrin's forum". What forum is this, and can I have a link? Edit: someone sent me a link and I took a quick look at it. No offense but the results from this study now have absolutely 0% credibility to me as I saw how the study was done.
I'll add that I've personally owned 6 out of the 7 headphones as listed in post #14 (haven't heard the Ultrasone Edition 8) and I wouldn't trust myself to rate most of them, with the exception of the HD800 as I currently own one. If I was asked to rate them, I'd reject rating on the principle that I haven't recently heard some of them, not to mention that I absolutely don't believe in assigning any sort of numerical rankings to headphone equipment.
Edited by Asr - 6/17/12 at 12:29ampost #65 of 1906/17/12 at 12:15ampost #66 of 1906/17/12 at 12:47am
I think some people misunderstood what this thread was about. Purrin clearly stated in the title that the question to be discussed was if “objective headphone measurements correlate to the audiophile's subjective experience.” From my understanding he did not intend to carry out a valid scientific study but give people some input as to how measurements could help them judge the performance of different headphones. I am not under the impression that he is trying to convince people that measurements should be considered the Holy Grail of headphone appreciation. As I understand it, those measurements can be considered valuable, albeit not the only and ultimate, information and input. I do not see any elitist or smart-alecky approach.
I guess, at HF we do indeed see too many posts by people who discuss and judge on headphones and gear in unacceptably subjective ways. Sometimes it gets simply ridiculous when people enthusiastically praise or condemn something without even having listened to the product in question. Too often different people use terminology in a different way, and it is evident that while talking about the same "thing", they do not share the same language, terminology, and do not take into account the different connotations that terms like “warm”, “lush”, “neutral”,“natural” etc. entail with the individual reader/listener. So, often people take wrong conclusions based on these misunderstandings and intricacies of language, terminology and connotative diffences.
Personally, I have learned to be skeptical or too enthusiastic “reviews” , and to rely on my own ears and people/reviewers I have found I can trust. Needless to say that I have paid a lot of money for that learning process.
Edited by stratocaster - 6/17/12 at 12:50ampost #67 of 1906/17/12 at 1:07amI've heard plenty of headphones that have sounded exactly like the graph portrayed. On the other hand, I've also heard so many headphones sound nothing like the graph portrayed.
I do think that they both are important to have, more of a checks and balances type of deal. If the subjective agrees with the objective, they are most likely both true. If there is a difference, there is something causing an error somewhere. Where? We'll never know. could be anything from tip/pad Selection all the way down to positioning.
Sent from an iPod touch with TapaTalk... Autocorrect may alter the meaning of this messagepost #68 of 1906/17/12 at 1:16ampost #69 of 1906/17/12 at 1:42am
I think it's funny how the usual suspects have taken this 'study' to be something other than what the OP said and intended it to be and applied standards and criteria to it based on comments from other posters who completely missed the point.post #70 of 1906/17/12 at 2:07am
I think Purrin was pretty clear with regard to what he was attempting to look at with this study...
"The point of this study is as follows:
post #71 of 1906/17/12 at 3:17am
- Is there correlation of certain measurement patterns to headphones which on average are subjectively rated good or bad?
- If there is, could it be possible that measurements are of use?"
i'm detecting some very defensive and ill tempered reactions in this thread to intelligent, questioning posts (and i don't count mine amongst them ). the thread topic and opening post has attracted a level of scrutiny that is entirely appropriate given the subject matter, and i'd be surprised if the op wasn't expecting as much.post #72 of 1906/17/12 at 4:00amHeadphone reveal (Click to show)Quote:Originally Posted by purrin
Average Rating: 4.07 (1 = good, 5 = bad)
Standard Deviation 1.14
- glaring, sibilant upper-end. I do like other facets of it
- heard it once at a meet..walked away thinking "people actually paid nearly XX for this?? Maybe they will buy my dog poop for 100 bucks per pound as a statement piece"
- the ED8 was your sacrifice to the Headphone Gods to give you a good YY
As a side note, to illustrate why measurements are important - they keep us honest. A reviewer once said of this headphone:
The XX’s are the flattest sounding, most extended headphones I have ever heard. No headphone I know of will equal them from top to bottom for evenness of performance... I have never heard or thought I would hear headphones this extended and flat in both directions in my lifetime, but here they are.
Average Rating: 1.78 (1 = good, 5 = bad)
Standard Deviation 0.83
- i wanted to love this HP when i owned it years ago, but ultimately found it a bit dull and lifeless.
- and despite the HD650’s flaws, I still like them quite a bit. Switching to them coming from sterile sounding cans makes music fun to listen to again.
- I think it's not the best headphone in every aspect but still a very capable headphone with a pleasant sound signature.
Average Rating: 2.00 (1 = good, 5 = bad)
Standard Deviation 0.74
- very airy and clear, wide soundstage,
- It's a damn good-sounding headphone. It's not perfect, but as AA said, EQ works well to correct its flaws.
- I can appreciate it from a technical standpoint but don't really want it near me. Like a professional weightlifter.
Average Rating: 2.95 (1 = good, 5 = bad)
Standard Deviation 0.94
- love the instrument separation and bass - especially the double bass as i find most recordings or HPs bloat this instrument. requires EQing and/or careful amp matching
- I like the clarity and separation from it, and I'm actually okay with the overall FR. Don't like the weird bump in the mids though, and the tonality is definitely artificial.
- properly amped, this is a good one. I gave it to a good friend when I decided it's not for me, and he's still happy with it, and I wouldn't give bad headphones as presents
Average Rating: 3.20 (1 = good, 5 = bad)
Standard Deviation 0.89
- bad sounding headphone. Pain = bad. I think it's about as simple as that.
- The RS1 is like a beautiful slut with a disease. I know there may very well be repercussions, but I just don't care to think that far ahead. Plus I know I've been bad, and I need to be punished.
- It was my main headphone for years, so I'm quite used to its sound signature.
- I feel they impart a certain body to music, but it's kind of weightless and not planted... kind of like an elephant in zero gravity. Aside from that I found them to be pretty thoroughly "meh."
- I bought these after the usual hype from the likes of YY, ZZ, and a few others kept harping on them. I do not consider them in the same class in the Audeze and HD800. They are thoroughly middling to my ears.
- I've heard several and they were all warm, bright, and metallic. All at the same time. Kinda like when my Dremel's cut off wheel shoots a stream of aluminum particles in my face.
Average Rating: 1.65 (1 = good, 5 = bad)
Standard Deviation 0.49
- I hate to nitpick this headphone because I found it so appealing. For the price I was expecting an ortho succubus that would be everything I wanted it in headphone.
- I'd buy it for $500-$600
It warms the cockles that the HD650 is so well regarded. Where is this Ark of yours purrin? Sounds like an interesting place with sensible people.post #73 of 1906/17/12 at 9:35amThread Starter
The intent from the outset was not to have random participants, but to have participants with already developed tastes. It's been my and many other hobbyists' experience that preferences develop and change over time, but that they will eventually settle. (There are exceptions to this of course.) I specifically wanted to poll this group. This is a constraint and limitation of this informal study.
As I've said, it is what is is, and anyone else is free to conduct their own studies. I can make my measurements available for anyone who wishes them for this purpose.
Perhaps a more apt title should have been:
Do Objective Headphone Measurements Correlate to the Subjective Experience of a Targeted Selection of Experienced Head-Fi Members
@Asr: I don't think it's necessary for most other people who have been around the block to have recently listened to a headphone (and on X or Y gear) to say in general whether they like it or not on 5 point scale. This study was meant to be informal and not get down to precise attributes. The distribution of votes did follow a bell curve in most cases, and in some cases, the distribution was quite wide.post #74 of 1906/17/12 at 11:03am
Obviously preferences play a big role, the problem is the people not being able to admit they have certain preferences and speak about their preferences as they would be the absolute truth and then may even go rambling about how great these are and ultimately the other person may have a bit different preferred sound signature and gets misslead. I've discovered my preferences by now so I know what kind of sound appeals to me, if they'd let me a draw a frequency response graph of how it should look like I'd be able to do that as well but then there's obviously things like soundstage that can't be measured just like that. For example Sennheiser HD800 is very anti-ideal signature to me, it's technically impressive with the sound but it doesn't give me the optimal engaging listening experience I search, I can probably find bunch of 100~$200 that would satisfy me a lot more even if amping was up-to-par. LCD-2/3 then again would be a lot closer but still not quite ideal either.
Edited by RPGWiZaRD - 6/17/12 at 11:10ampost #75 of 1906/17/12 at 1:04pm
I find measurements interesting, but ultimately quite inconclusive. For example, there is no real agreement on how a headphone SHOULD measure in order to mimic the experience of hearing an external soundsource, so what factor should combined with the raw measurements to produce a "perceived flat" experience?
Also, since the direct acoustic load on the phone is the ear and every ear is different, the differing load will also change the raw measurements, meaning they are completely relevant only to the dummy ear. In fact , it would be interesting to model various shaped ear/canal structures and measure how the measured response of a variety of phones change with the differing load.
- Do Objective Headphone Measurements Correlate to the Audiophile's Subjective Experience?
Gear mentioned in this thread:
- Head-Fi Buying Guide (Ultra-High-End Headphones (Summit-Fi))
- › The Deals DISCUSSION Thread (READ THE FIRST POST!!!) 2 minutes ago
- › New Schiit! Ragnarok and Yggdrasil 2 minutes ago
- › Music Game IX 2 minutes ago
- › Band Name Game II 4 minutes ago
- › The Apex Teton Review and Impressions Thread 4 minutes ago
- › Chord Electronics 2Qute DAC announced 5 minutes ago
- › Three Word Story 5 minutes ago
- › Sennheiser HD 598 Impressions Thread 8 minutes ago
- › SONY NWZ-A10 Series 10 minutes ago
- › Light Harmonic Geek Out EM/1000 Impressions Thread 10 minutes ago
- › HiFiMAN HE-560 by DreamKing
- › Riva TurboX High Performance Premium Mobile Bluetooth Speaker (Black) by twister6
- › Ray Samuels Emmeline "The Black Bird" SR-71A Portable... by Stereocilia
- › Aurorasound HEADA Balanced Headphone Amplifier by Currawong
- › Audeze EL-8 Closed-Back by bearcat1986
- › Bravo Audio Tube Amplifier - V3 by YoYo JoKeR
- › thinksound rain2 Wooden In-Ear Headphone (Gunmetal Chocolate) by DJScope
- › HiFiMAN HE-500 by MrJohnny
- › Shure SE112m+-GR Sound Isolating Earphones with Remote + Microphone... by pformagg
- › MEElectronics M6 PRO Universal-Fit Noise-Isolating Musician’s... by Luckbad
- › Can I change my user name? Can I delete my...
- › Table of Earphocalypse
- › The Great FiiO DAP FAQ Wiki
- › Audio Connections between components --...
- › Head-Fi Buying Guide (Introduction)
- › Head-Fi Buying Guide (Over-Ear Headphones)
- › Head-Fi Buying Guide (Over-Ear Headphones) 2
- › Head-Fi Buying Guide (Over-Ear Headphones) 3
- › Head-Fi Buying Guide (In-Ear Headphones)
- › Head-Fi Buying Guide (In-Ear Headphones) 2