ruthieandjohn
Stumbling towards enlightenment
(Formerly known as kayandjohn.)
We Rank Our Own Headphones - Statistics of 2,000 Owner Rankings, Compiled
SUMMARY:
I have compiled an aggregated list of how we rank our own headphones, based on the thread "Rank the Headphones That you Own," ( http://www.head-fi.org/t/109756/rank-the-headphones-that-you-own#post_1284577 ). Data is based on the most recent 2,000 rankings of that thread, extending approximately one year back.
This list differs from other lists in which a single reviewer compares multiple headphones. This list aggregates the comparisons of many persons, each with some subset of the full list of headphone models. It has been often said that the best headphone comparisons are those made by persons owning all headphones being compared: this analysis attempts to capture these results.
This list is useful for someone considering a new headphone and wishes to know how folks who already have that headphone rank it against some others, with the hope that the inquirer will be familiar with some of the headphones to which it is compared.
To that end, I have:
GENERAL STATISTICS:
The 2,000 headphone rankings were extracted from posts on the thread "Rank the Headphones that you Own" with dates ranging from 5/31/2013 through 5/16/2014. For folks who (endlessly) kept updating their ranking, I tried to use only their most recent ranking (though I admit some double rankings slipped through!).
Those 2,000 rankings encompass a total of 340 users posting their rankings. The rankings included 816 headphone models. However, 544 of these models only had one person ranking them. Many of these were unique modifications to commercially-available headphones (e.g., "AKG Q701 w/ bass port sticker removed"). Others were truly one-of-a-kind users of apparently non-customized headphones (e.g. only one person ranked the HiFiMAN HE-300).
Because the goal of this work was to provide some statistically reliable characterizations of how a particular headphone is ranked against others, I chose to concentrate extra attention on those headphone models that had at least 10 people ranking them. The following table identifies the 37 headphone models that met this criterion.
Fig. 1: Listing of Headphone Models Ranked by 10 or More Persons, With Count Of Rankings And Positional Score (10=top rated).
The headphones each have a score ranging from 1 to 10, indicating (the reverse of) their average ranking across the persons who ranked them. For example, if all 53 persons indicated in the "Count" column ranked the Sennheiser HD 800 at the top of their list, that headphone would have a score of 10.0. The fact that it has a score of 8.9 shows that several of the 53 rankings placed other headphones above it. In some cases, those were headphones on this list of 37 (e.g., HiFiMAN HE-6, Audeze LCD-3); in others, they were headphones not ranked often enough to appear on this list of 37 popular models (e.g., Stax SR-009, which had 6 persons ranking it).
RANKINGS OF MOST POPULAR HEADPHONES:
The general score given in the above table is not as useful as knowing how a particular headphone was ranked against another particular headphone. This section provides model-specific pairwise rankings.
All 37 headphones appear as both rows and columns in the following table. Each cell shows the ranking of the two headphone models that label its row and its column position. For the diagonal, both row and column model are identical and it is filled with a black square. Below the diagonal, the information is simply reflected from above (i.e. ranking the Sennheiser HD 800 against the Beyerdynamic T1 is the same as ranking the T1 against the HD 800), so cells below the diagonal are left blank, since the same information exists in the table above the diagonal.
The color of the dot in the cell indicates who won the pairing... a blue or green dot indicates strong (blue) or medium (green) winning of the pair-off by the headphone model in the row title. This is easy to remember because the row titles are blue. Likewise, a red or orange dot indicates strong (red) or medium (orange) winning of the pair-off by the headphone model in the (conveniently red) column title at the top of the table. If the two comparisons were approximately equal (e.g., 4 persons ranked the HiFiMAN HE-6 above the Sennheiser HD 800 and 6 persons ranked the HD 800 above the HE-6), the dot is white.
Specifically, the colors are assigned by the fraction of multiple rankings won by the (blue) row title as follows: 80%-100% (blue); 60% - 79% (green); 40% - 59% (white); 20% - 39% (orange), and 0% - 19% (red). Hence, a red dot means the same for the red column model (more than 79% preferred) as a blue dot means for the blue column model (again, more than 79% preferred).
The confidence of each ranking is measured by the number of pair-wise comparisons that contributed to the dot. If there were 10 or more, the background color of the cell is a dark gray; 5 to 9 gives a medium gray background, and 1 to 4 gives a light gray background. If there were no instances of a ranking of one headphone model vs. another, the corresponding cell is blank (white, no dot).
Fig. 2: Pairwise Ranking of 37 Most Popular Headphone Models (click to enlarge).
Hence, it is interesting to note that at least 10 persons ranked the Sennheiser HD 800 against the HiFiMAN HE-6, and approximately equal numbers selected either as superior to the other. Hence, its cell is dark gray with a white dot.
CORRELATIONS OF HEADPHONE MODEL OWNERSHIP
Do "bassheads" tend to favor a certain set of headphone models in their purchases? Are there "trebleheads" favoring a different set? If an unusually large number of people that have a Sennheiser HD 600 also have a Sennheiser HD 650 (and the data shows that they do!), should you get one too to go with your HD 600?
This portion of the effort looked for pairs of headphone model ownership that were significantly higher than random chance. The table below identifies where an unusually high number of persons that have one particular headphone model also have another headphone model.
The red cells show modest correlation (specifically, that the number of times someone ranks that pair of model pairs at least two standard deviations higher than a random distribution); the black cells show strong correlation (more than four standard deviations above random).
I need to look at these correlations more to find other effects, but I submit the data shows a "moneybags" effect with its heavy correlation in the stratosphere of top-ranking headphones at the upper left... people who have HD 800s also are more likely to have other top-end headphones such as the HiFiMAN HE-6 and Audeze LCD-3 (they probably have Stax SR-009 too, but not enough are in the data set to appear here).
Fig. 3: Correlation of Headphone Models Owned: Pink Cells Are Significant Correlations; Black Cells Are Highly Significant Correlations.
This sort of table is a start in automated recommender systems such as Netflix or Amazon, where user product preferences are predicted from their preferences of product they already have.
SPECIALIZED HEADPHONE PAIR COMPARISONS
This data is really a small subset of the information that is available. If you are considering a headphone purchase or want to know how a headphone model fared in this data that did not appear here, just post to this thread and let me know, and I'll be glad to post both statistics and (if not too numerous) individual ranking lists (sometimes, such as for the HD 800, there are too many, 53 in that case).
For example, I have been pointed to the Beyerdynamic T 5 p as a great audiophile headphone for iPod portable use. Only 6 folks ranked it, but I can retrieve the six ranking lists as shown here and see what headphones are ranked better and worse than the T 5 p (and by whom, if I want to ask questions).
Fig. 4: Table of the Six Rankings for the Beyerdynamic T 5 p - each ranking list is a column, with the number at the top giving the rank order position, in percent, of the model of interest (0% = top of list; 100% = bottom of list). This is an example of the ranking lists that can be extracted for any headphone in the database.
Again, feel free to request individual model lists that I didn't already post here.
(edited: Jun 10, 2014 to reflect data omissions caught by jazzman7 (thanks!)... adds comparisons for HD 800 vs. HE-500, HE-50 vs. AKG K701, M-100 vs. PortaPro, PX 100-II vs. AD 700, AKG K701 vs Porta Pro, and Shure SRH840 vs. AD 700).
SUMMARY:
I have compiled an aggregated list of how we rank our own headphones, based on the thread "Rank the Headphones That you Own," ( http://www.head-fi.org/t/109756/rank-the-headphones-that-you-own#post_1284577 ). Data is based on the most recent 2,000 rankings of that thread, extending approximately one year back.
This list differs from other lists in which a single reviewer compares multiple headphones. This list aggregates the comparisons of many persons, each with some subset of the full list of headphone models. It has been often said that the best headphone comparisons are those made by persons owning all headphones being compared: this analysis attempts to capture these results.
This list is useful for someone considering a new headphone and wishes to know how folks who already have that headphone rank it against some others, with the hope that the inquirer will be familiar with some of the headphones to which it is compared.
To that end, I have:
- Computed statistics on the various models and how many times each is ranked;
- Developed an aggregated list of where each model falls, on the average, in a rank ordered list. (Note that this is really not too useful, as each person ranks a given headphone against the others he or she owns, and each person ranking that headphone is ranking it against a different list);
- Prepared a master ranking matrix, selecting only those headphones that appear on a ranking 10 or more times (to provide some statistical robustness), allowing each headphone of that population to be compared in ranking to all others (of that population), with indication of how many rankings are reflected for each comparison pair (and therefore with what degree of confidence each ranking has);
- Performed some correlation plots that show where having one headphone model increases the likelihood of having certain other headphone models;
- Set up the ability to provide, by request on this thread, ranking lists of any of the headphones ranked in the data set (even if they do not appear in the list posted of the most popular headphones... I can still search the database and post each comparison list that shows where that headphone is ranked by folks who own it against their other headphones).
GENERAL STATISTICS:
The 2,000 headphone rankings were extracted from posts on the thread "Rank the Headphones that you Own" with dates ranging from 5/31/2013 through 5/16/2014. For folks who (endlessly) kept updating their ranking, I tried to use only their most recent ranking (though I admit some double rankings slipped through!).
Those 2,000 rankings encompass a total of 340 users posting their rankings. The rankings included 816 headphone models. However, 544 of these models only had one person ranking them. Many of these were unique modifications to commercially-available headphones (e.g., "AKG Q701 w/ bass port sticker removed"). Others were truly one-of-a-kind users of apparently non-customized headphones (e.g. only one person ranked the HiFiMAN HE-300).
Because the goal of this work was to provide some statistically reliable characterizations of how a particular headphone is ranked against others, I chose to concentrate extra attention on those headphone models that had at least 10 people ranking them. The following table identifies the 37 headphone models that met this criterion.
Model | Count | Score |
| Sennheiser HD 800 | 53 | 8.9 |
| HiFiMAN HE-6 | 18 | 8.2 |
| Audeze LCD-3 | 13 | 8.2 |
| HiFiMAN HE-500 | 35 | 8.1 |
| Audeze LCD-2.2 | 23 | 7.9 |
| Beyerdynamic T 1 | 24 | 7.7 |
| Philips Fidelio X1 | 14 | 7.3 |
| Fostex TH600 | 11 | 6.9 |
| Sennheiser HD 600 | 46 | 7.1 |
| Fostex TH900 | 13 | 6.9 |
| Audeze LCD-2 | 16 | 6.5 |
| Sennheiser HD 650 | 57 | 6.2 |
| HiFiMAN HE-400 | 28 | 6.2 |
| AKG Q701 | 17 | 6.0 |
| Sennheiser HD 598 | 26 | 5.8 |
| Beyerdynamic DT 990 | 12 | 5.7 |
| Sennheiser Momentum | 20 | 5.7 |
| Beyerdynamic DT 880 | 16 | 5.5 |
| Denon AH-D7000 | 12 | 5.4 |
| Denon AH-D2000 | 19 | 5.4 |
| AKG K550 | 13 | 5.4 |
| AKG K701 | 17 | 5.4 |
| Sennheiser HD 580 | 10 | 5.3 |
| Beyerdynamic DT 1350 | 10 | 5.2 |
| MrSpeakers Mad Dog V3.2 | 13 | 4.9 |
| Sennheiser HD 25-1 II | 10 | 4.8 |
| Grado SR225i | 12 | 4.8 |
| Sennheiser Amperior | 10 | 4.7 |
| Shure SRH840 | 12 | 4.5 |
| V-MODA M-100 | 10 | 4.3 |
| Grado SR80 | 13 | 4.0 |
| Grado SR80i | 13 | 3.9 |
| Audio-Technica ATH-M50 | 29 | 3.6 |
| Koss KSC75 | 11 | 3.6 |
| Sennheiser PX 100-II | 10 | 3.4 |
| Audio-Technica ATH-AD700 | 16 | 2.9 |
| Koss Porta Pro | 18 | 2.5 |
Fig. 1: Listing of Headphone Models Ranked by 10 or More Persons, With Count Of Rankings And Positional Score (10=top rated).
The headphones each have a score ranging from 1 to 10, indicating (the reverse of) their average ranking across the persons who ranked them. For example, if all 53 persons indicated in the "Count" column ranked the Sennheiser HD 800 at the top of their list, that headphone would have a score of 10.0. The fact that it has a score of 8.9 shows that several of the 53 rankings placed other headphones above it. In some cases, those were headphones on this list of 37 (e.g., HiFiMAN HE-6, Audeze LCD-3); in others, they were headphones not ranked often enough to appear on this list of 37 popular models (e.g., Stax SR-009, which had 6 persons ranking it).
RANKINGS OF MOST POPULAR HEADPHONES:
The general score given in the above table is not as useful as knowing how a particular headphone was ranked against another particular headphone. This section provides model-specific pairwise rankings.
All 37 headphones appear as both rows and columns in the following table. Each cell shows the ranking of the two headphone models that label its row and its column position. For the diagonal, both row and column model are identical and it is filled with a black square. Below the diagonal, the information is simply reflected from above (i.e. ranking the Sennheiser HD 800 against the Beyerdynamic T1 is the same as ranking the T1 against the HD 800), so cells below the diagonal are left blank, since the same information exists in the table above the diagonal.
The color of the dot in the cell indicates who won the pairing... a blue or green dot indicates strong (blue) or medium (green) winning of the pair-off by the headphone model in the row title. This is easy to remember because the row titles are blue. Likewise, a red or orange dot indicates strong (red) or medium (orange) winning of the pair-off by the headphone model in the (conveniently red) column title at the top of the table. If the two comparisons were approximately equal (e.g., 4 persons ranked the HiFiMAN HE-6 above the Sennheiser HD 800 and 6 persons ranked the HD 800 above the HE-6), the dot is white.
Specifically, the colors are assigned by the fraction of multiple rankings won by the (blue) row title as follows: 80%-100% (blue); 60% - 79% (green); 40% - 59% (white); 20% - 39% (orange), and 0% - 19% (red). Hence, a red dot means the same for the red column model (more than 79% preferred) as a blue dot means for the blue column model (again, more than 79% preferred).
The confidence of each ranking is measured by the number of pair-wise comparisons that contributed to the dot. If there were 10 or more, the background color of the cell is a dark gray; 5 to 9 gives a medium gray background, and 1 to 4 gives a light gray background. If there were no instances of a ranking of one headphone model vs. another, the corresponding cell is blank (white, no dot).
Fig. 2: Pairwise Ranking of 37 Most Popular Headphone Models (click to enlarge).
Hence, it is interesting to note that at least 10 persons ranked the Sennheiser HD 800 against the HiFiMAN HE-6, and approximately equal numbers selected either as superior to the other. Hence, its cell is dark gray with a white dot.
CORRELATIONS OF HEADPHONE MODEL OWNERSHIP
Do "bassheads" tend to favor a certain set of headphone models in their purchases? Are there "trebleheads" favoring a different set? If an unusually large number of people that have a Sennheiser HD 600 also have a Sennheiser HD 650 (and the data shows that they do!), should you get one too to go with your HD 600?
This portion of the effort looked for pairs of headphone model ownership that were significantly higher than random chance. The table below identifies where an unusually high number of persons that have one particular headphone model also have another headphone model.
The red cells show modest correlation (specifically, that the number of times someone ranks that pair of model pairs at least two standard deviations higher than a random distribution); the black cells show strong correlation (more than four standard deviations above random).
I need to look at these correlations more to find other effects, but I submit the data shows a "moneybags" effect with its heavy correlation in the stratosphere of top-ranking headphones at the upper left... people who have HD 800s also are more likely to have other top-end headphones such as the HiFiMAN HE-6 and Audeze LCD-3 (they probably have Stax SR-009 too, but not enough are in the data set to appear here).
Fig. 3: Correlation of Headphone Models Owned: Pink Cells Are Significant Correlations; Black Cells Are Highly Significant Correlations.
This sort of table is a start in automated recommender systems such as Netflix or Amazon, where user product preferences are predicted from their preferences of product they already have.
SPECIALIZED HEADPHONE PAIR COMPARISONS
This data is really a small subset of the information that is available. If you are considering a headphone purchase or want to know how a headphone model fared in this data that did not appear here, just post to this thread and let me know, and I'll be glad to post both statistics and (if not too numerous) individual ranking lists (sometimes, such as for the HD 800, there are too many, 53 in that case).
For example, I have been pointed to the Beyerdynamic T 5 p as a great audiophile headphone for iPod portable use. Only 6 folks ranked it, but I can retrieve the six ranking lists as shown here and see what headphones are ranked better and worse than the T 5 p (and by whom, if I want to ask questions).
0% | 25% | 55% | 43% | 0% | 33% |
| Beyerdynamic T 5 p | Sennheiser HD 800 | Stax SR007 MK2 | HiFiMAN HE-6 | Beyerdynamic T 5 p | Audeze LCD-2.2 |
| Audeze LCD-3 | Beyerdynamic T 5 p | Sennheiser HD 800 | Sennheiser HD 800 | Audio-Technica ATH-W5000 | Beyerdynamic T 5 p |
| Sennheiser HD 800 | Sennheiser Amperior | Audio-Technica ATH-ESW11LTD | Beyerdynamic T 1 | Sennheiser HD 650 | Westone ES2 |
| AKG K3003i | AKG K701 | HiFiMAN HE-6 | Beyerdynamic T 5 p | PSB M4U2 | Beyerdynamic T 51 p |
| Ultrasone Edition 8 | AKG K450 | Audeze LCD-2.1 | Grado GS1000i | Sennheiser HD 380 PRO | |
| Shure E2c | Shure SE535 | Audio-Technica W3000ANV | |||
| Beyerdynamic T 5 p | Polk Ultrafocus 8000 (ANC) | ||||
| AKG Q701 | Logitech UE 900 | ||||
| Grado SR325i | |||||
| Shure SE425 | |||||
| B&W P5 | |||||
| Sony XB 500 |
Fig. 4: Table of the Six Rankings for the Beyerdynamic T 5 p - each ranking list is a column, with the number at the top giving the rank order position, in percent, of the model of interest (0% = top of list; 100% = bottom of list). This is an example of the ranking lists that can be extracted for any headphone in the database.
Again, feel free to request individual model lists that I didn't already post here.
(edited: Jun 10, 2014 to reflect data omissions caught by jazzman7 (thanks!)... adds comparisons for HD 800 vs. HE-500, HE-50 vs. AKG K701, M-100 vs. PortaPro, PX 100-II vs. AD 700, AKG K701 vs Porta Pro, and Shure SRH840 vs. AD 700).
