Great job purrin with the study, it has value and actually your sample size was for the most part suitable. Once you have 30 participants the central limits theorem comes into play and you can assume the population was normally distributed. Just being picky here, but because you had a sample it is actually the standard error of the mean, not the standard deviation which is a population statistic, but I suspect you know that. I really liked that you tried this, and I think designing a more robust study would actually be a great thing. I would love to work on such a study for use as a thesis paper I need to do soon, but it has bearing on my career area so sadly I will have to study something else. Really interesting line of questioning so keep it up if you can.
Recent Images In This Thread
- productAKG K701 Studio Headphonestagged by System, 6/19/12
- productAudeze LCD2 Planar Magnetic Headphonestagged by System, 6/19/12
- productBeyerdynamic Tesla T1tagged by System, 6/19/12
- productGrado RS1tagged by System, 6/19/12
- categoryHeadphonestagged by System, 6/19/12
- productSennheiser HD 650 Headphonestagged by System, 6/19/12
- productSennheiser HD 800 Headphonestagged by System, 6/19/12
- productUltrasone Edition 8tagged by System, 6/19/12
Related Forum Threads
- Virgin - Needs Advice - Never had a Headphone Last post on 2/8/13 at 9:43am in Headphones (full-size)
- Do I buy Beyerdynamic T1 or Hifiman HE500 as upgrade solution to my current HD650/K701 combo? Last post on 10/26/13 at 2:27am in Headphones (full-size)
- OPEN headphones with the TIGHTEST bass! Last post on 8/11/12 at 8:04am in Headphones (full-size)
- 5-year Re-Review: Audio-Technica ATH-AD2000 Last post on 3/8/12 at 11:45pm in Headphones (full-size)
- so i just got some K 701's.... Last post on 10/6/11 at 12:57am in Headphones (full-size)
Head-Fi Buying Guide (Over-Ear Headphones) 2
Last edited: 12/5/14
- Head-Fi Buying Guide (Ultra-High-End Headphones (Summit-Fi))Last edited: 12/8/14
- Head-Fi Buying Guide (Over-Ear Headphones)Last edited: 11/28/14
- Comparisons of LCD-2 Rev. 1 and Rev. 2Last edited: 9/19/11
- Headphone ImpedanceLast edited: 8/18/12
My setup: iBasso DX50 --> Dunu DN-1000 Packaging Yes the accessories in the box are plenty. But the cloth material used for inner box and metal case is poor. The black dust from the cloth...
It is really surprising to find this high quality CD performance in analog and digital format. When I connected it to Burson Audio Conductor, all the ignored CD library came back to life. This...
Got this earphones for the sole purpose of working out/biking. Tested out multiple sports headsets (similarly priced), from sennheiser and bose. They tend to sound better, especially than the...
Overall a great little card. Dolby headphone is amazing I preferred this card to my Sound Blaster Z, I sold that card when I needed a bit of money and bought this one to replace it for cheap and I...
I was in the market for a higher end 2.1 system when my search ended up at the Edifier website, I was impressed with the specs and decided to delve further, found a set at a nice price and pulled...
Do Objective Headphone Measurements Correlate to the Audiophile's Subjective Experience? - Page 5post #61 of 1936/16/12 at 11:32pm
Gear mentioned in this thread:post #62 of 1936/16/12 at 11:56pm
I think it's because Purin has put so much effort into these measurements, far beyond that of all except a very few, that people get the impression he is only interested in measurements. The Corvette is known for its LS3 motor, but that's not to say that the suspension doesn't exist.post #63 of 1936/17/12 at 12:04amQuote:Originally Posted by rhythmdevils
Which is why you are very vocal about liking two headphones which have issues showing up on your CSD plots. Because you trust your ears first, and use these graphs as a tool to further understand what you are hearing. LOL that quote actually proved the opposite point as intended...
People want things in such easy, black and white packages. I'm not sure why you are supposed to choose between graphs and ears. This whole thread is about how they relate to each other.
to be serious for a moment, i think you've misconstrued my post. i was quoting purrin in a light-hearted way to show that headphones can seemingly defy bad measurements by sounding good to the guy who is actually taking them. the implication being that purrin ultimately relies on his ears rather than his measurements to judge the merits of a headphone.
Edited by shimmer n roar - 6/17/12 at 1:39ampost #64 of 1936/17/12 at 12:07amQuote:Originally Posted by purrin
A pool of thirty headphone hobbyists were asked to rate on a five point scale seven headphones. They were not told of the purpose of this study. The headphones in the study were selected because of their popularity in order to increase the likelihood that the hobbyists were familiar with them, and hence could provide a rating. Each hobbyist was asked to rate the headphone as follows without any consideration to price:
I read through the thread so far and nowhere has it been mentioned how you conducted the study, which I think is relevant info. You said there was a pool of 30 hobbyists and that they're all active Head-Fi members, who rated 7 headphones. IMO this begs the question, how did all 30 members listen to all 7 headphones? Were any headphones shipped around from one Head-Fi member to another, or was there any listening done in a micro-meet setting? Were there any instances of people who rated a headphone but didn't currently have it on hand? Because if there were people who gave ratings without having at least a recent listen to a headphone in their own home, I'd consider their rating completely invalid.
Logistically speaking I don't see how this study was achieved without extensive time being put into it and how the feedback given from the 30-member pool has any credibility at the same time.
Also I've seen mentions of "purrin's forum". What forum is this, and can I have a link? Edit: someone sent me a link and I took a quick look at it. No offense but the results from this study now have absolutely 0% credibility to me as I saw how the study was done.
I'll add that I've personally owned 6 out of the 7 headphones as listed in post #14 (haven't heard the Ultrasone Edition 8) and I wouldn't trust myself to rate most of them, with the exception of the HD800 as I currently own one. If I was asked to rate them, I'd reject rating on the principle that I haven't recently heard some of them, not to mention that I absolutely don't believe in assigning any sort of numerical rankings to headphone equipment.
Edited by Asr - 6/17/12 at 12:29ampost #65 of 1936/17/12 at 12:15ampost #66 of 1936/17/12 at 12:47am
I think some people misunderstood what this thread was about. Purrin clearly stated in the title that the question to be discussed was if “objective headphone measurements correlate to the audiophile's subjective experience.” From my understanding he did not intend to carry out a valid scientific study but give people some input as to how measurements could help them judge the performance of different headphones. I am not under the impression that he is trying to convince people that measurements should be considered the Holy Grail of headphone appreciation. As I understand it, those measurements can be considered valuable, albeit not the only and ultimate, information and input. I do not see any elitist or smart-alecky approach.
I guess, at HF we do indeed see too many posts by people who discuss and judge on headphones and gear in unacceptably subjective ways. Sometimes it gets simply ridiculous when people enthusiastically praise or condemn something without even having listened to the product in question. Too often different people use terminology in a different way, and it is evident that while talking about the same "thing", they do not share the same language, terminology, and do not take into account the different connotations that terms like “warm”, “lush”, “neutral”,“natural” etc. entail with the individual reader/listener. So, often people take wrong conclusions based on these misunderstandings and intricacies of language, terminology and connotative diffences.
Personally, I have learned to be skeptical or too enthusiastic “reviews” , and to rely on my own ears and people/reviewers I have found I can trust. Needless to say that I have paid a lot of money for that learning process.
Edited by stratocaster - 6/17/12 at 12:50ampost #67 of 1936/17/12 at 1:07amI've heard plenty of headphones that have sounded exactly like the graph portrayed. On the other hand, I've also heard so many headphones sound nothing like the graph portrayed.
I do think that they both are important to have, more of a checks and balances type of deal. If the subjective agrees with the objective, they are most likely both true. If there is a difference, there is something causing an error somewhere. Where? We'll never know. could be anything from tip/pad Selection all the way down to positioning.
Sent from an iPod touch with TapaTalk... Autocorrect may alter the meaning of this messagepost #68 of 1936/17/12 at 1:16ampost #69 of 1936/17/12 at 1:42am
I think it's funny how the usual suspects have taken this 'study' to be something other than what the OP said and intended it to be and applied standards and criteria to it based on comments from other posters who completely missed the point.post #70 of 1936/17/12 at 2:07am
I think Purrin was pretty clear with regard to what he was attempting to look at with this study...
"The point of this study is as follows:
post #71 of 1936/17/12 at 3:17am
- Is there correlation of certain measurement patterns to headphones which on average are subjectively rated good or bad?
- If there is, could it be possible that measurements are of use?"
i'm detecting some very defensive and ill tempered reactions in this thread to intelligent, questioning posts (and i don't count mine amongst them ). the thread topic and opening post has attracted a level of scrutiny that is entirely appropriate given the subject matter, and i'd be surprised if the op wasn't expecting as much.post #72 of 1936/17/12 at 4:00amHeadphone reveal (Click to show)Quote:Originally Posted by purrin
Average Rating: 4.07 (1 = good, 5 = bad)
Standard Deviation 1.14
- glaring, sibilant upper-end. I do like other facets of it
- heard it once at a meet..walked away thinking "people actually paid nearly XX for this?? Maybe they will buy my dog poop for 100 bucks per pound as a statement piece"
- the ED8 was your sacrifice to the Headphone Gods to give you a good YY
As a side note, to illustrate why measurements are important - they keep us honest. A reviewer once said of this headphone:
The XX’s are the flattest sounding, most extended headphones I have ever heard. No headphone I know of will equal them from top to bottom for evenness of performance... I have never heard or thought I would hear headphones this extended and flat in both directions in my lifetime, but here they are.
Average Rating: 1.78 (1 = good, 5 = bad)
Standard Deviation 0.83
- i wanted to love this HP when i owned it years ago, but ultimately found it a bit dull and lifeless.
- and despite the HD650’s flaws, I still like them quite a bit. Switching to them coming from sterile sounding cans makes music fun to listen to again.
- I think it's not the best headphone in every aspect but still a very capable headphone with a pleasant sound signature.
Average Rating: 2.00 (1 = good, 5 = bad)
Standard Deviation 0.74
- very airy and clear, wide soundstage,
- It's a damn good-sounding headphone. It's not perfect, but as AA said, EQ works well to correct its flaws.
- I can appreciate it from a technical standpoint but don't really want it near me. Like a professional weightlifter.
Average Rating: 2.95 (1 = good, 5 = bad)
Standard Deviation 0.94
- love the instrument separation and bass - especially the double bass as i find most recordings or HPs bloat this instrument. requires EQing and/or careful amp matching
- I like the clarity and separation from it, and I'm actually okay with the overall FR. Don't like the weird bump in the mids though, and the tonality is definitely artificial.
- properly amped, this is a good one. I gave it to a good friend when I decided it's not for me, and he's still happy with it, and I wouldn't give bad headphones as presents
Average Rating: 3.20 (1 = good, 5 = bad)
Standard Deviation 0.89
- bad sounding headphone. Pain = bad. I think it's about as simple as that.
- The RS1 is like a beautiful slut with a disease. I know there may very well be repercussions, but I just don't care to think that far ahead. Plus I know I've been bad, and I need to be punished.
- It was my main headphone for years, so I'm quite used to its sound signature.
- I feel they impart a certain body to music, but it's kind of weightless and not planted... kind of like an elephant in zero gravity. Aside from that I found them to be pretty thoroughly "meh."
- I bought these after the usual hype from the likes of YY, ZZ, and a few others kept harping on them. I do not consider them in the same class in the Audeze and HD800. They are thoroughly middling to my ears.
- I've heard several and they were all warm, bright, and metallic. All at the same time. Kinda like when my Dremel's cut off wheel shoots a stream of aluminum particles in my face.
Average Rating: 1.65 (1 = good, 5 = bad)
Standard Deviation 0.49
- I hate to nitpick this headphone because I found it so appealing. For the price I was expecting an ortho succubus that would be everything I wanted it in headphone.
- I'd buy it for $500-$600
It warms the cockles that the HD650 is so well regarded. Where is this Ark of yours purrin? Sounds like an interesting place with sensible people.post #73 of 1936/17/12 at 9:35amThread Starter
The intent from the outset was not to have random participants, but to have participants with already developed tastes. It's been my and many other hobbyists' experience that preferences develop and change over time, but that they will eventually settle. (There are exceptions to this of course.) I specifically wanted to poll this group. This is a constraint and limitation of this informal study.
As I've said, it is what is is, and anyone else is free to conduct their own studies. I can make my measurements available for anyone who wishes them for this purpose.
Perhaps a more apt title should have been:
Do Objective Headphone Measurements Correlate to the Subjective Experience of a Targeted Selection of Experienced Head-Fi Members
@Asr: I don't think it's necessary for most other people who have been around the block to have recently listened to a headphone (and on X or Y gear) to say in general whether they like it or not on 5 point scale. This study was meant to be informal and not get down to precise attributes. The distribution of votes did follow a bell curve in most cases, and in some cases, the distribution was quite wide.post #74 of 1936/17/12 at 11:03am
Obviously preferences play a big role, the problem is the people not being able to admit they have certain preferences and speak about their preferences as they would be the absolute truth and then may even go rambling about how great these are and ultimately the other person may have a bit different preferred sound signature and gets misslead. I've discovered my preferences by now so I know what kind of sound appeals to me, if they'd let me a draw a frequency response graph of how it should look like I'd be able to do that as well but then there's obviously things like soundstage that can't be measured just like that. For example Sennheiser HD800 is very anti-ideal signature to me, it's technically impressive with the sound but it doesn't give me the optimal engaging listening experience I search, I can probably find bunch of 100~$200 that would satisfy me a lot more even if amping was up-to-par. LCD-2/3 then again would be a lot closer but still not quite ideal either.
Edited by RPGWiZaRD - 6/17/12 at 11:10ampost #75 of 1936/17/12 at 1:04pm
I find measurements interesting, but ultimately quite inconclusive. For example, there is no real agreement on how a headphone SHOULD measure in order to mimic the experience of hearing an external soundsource, so what factor should combined with the raw measurements to produce a "perceived flat" experience?
Also, since the direct acoustic load on the phone is the ear and every ear is different, the differing load will also change the raw measurements, meaning they are completely relevant only to the dummy ear. In fact , it would be interesting to model various shaped ear/canal structures and measure how the measured response of a variety of phones change with the differing load.
- Do Objective Headphone Measurements Correlate to the Audiophile's Subjective Experience?
Gear mentioned in this thread:
- Head-Fi Buying Guide (Ultra-High-End Headphones (Summit-Fi))
- › New Release! The K7XX Massdrop First Edition - an exclusive from... 44 seconds ago
- › The Fiio X3 Thread. 1 minute ago
- › Final Audio Design Impressions and Discussion Thread 1 minute ago
- › Lake People G109 / G103 thread 2 minutes ago
- › Beyerdynamic T90 :D 3 minutes ago
- › DITA - The Answer (Truth Edition). 3 minutes ago
- › The "non-Grado" Driver Grado Thread 3 minutes ago
- › The NEW JVC FX850..Woodie perfection?. (A review) 3 minutes ago
- › The Official Beyerdynamic T1 Impressions and Discussion Thread 5 minutes ago
- › Audio GD NFB-28 FS 5 minutes ago
- › Dunu DN-1000 by screwedpeep
- › Sony BDPS500 Bravia Theater Sync Blu-ray Player by Joong
- › Audio Technica Sonic Fuel ATH-CKX9iS In-ear Headphones with In-line... by BenKatz
- › Asus Xonar DGX by TsukiNick
- › Edifier S730 by Croozer
- › Velodyne vPulse In-Ear Headphones (Classic Black) by spinnin
- › QLS QA360 Portable HiFi Lossless Music Player by YoYo JoKeR
- › Koss Pro4S Full Size Studio Headphones, Black with Silver Trim by psblend
- › Meier Audio Corda Classic by longbowbbs
- › Klipsch Image S4i II Black In-Ear Headphones with Microphone by laurentiusjan
- › The Great FiiO DAP FAQ Wiki
- › Audio Connections between components --...
- › Head-Fi Buying Guide (Introduction)
- › Head-Fi Buying Guide (Over-Ear Headphones)
- › Head-Fi Buying Guide (Over-Ear Headphones) 2
- › Head-Fi Buying Guide (Over-Ear Headphones) 3
- › Head-Fi Buying Guide (In-Ear Headphones)
- › Head-Fi Buying Guide (In-Ear Headphones) 2
- › Head-Fi Buying Guide (In-Ear Headphones) 3
- › Head-Fi Buying Guide (Gaming Headphones)