[Junglebook3]

Hi folks,

My name is Assaf, and I fell in the audiophile rabbit hole last March as COVID hit. Zaya is what I wish had existed when I started on my journey. It is a community driven review hub for headphones - Zaya helps you find the best headphones for you using tailored filter and search capabilities, community driven rankings, in-depth information including consumable frequency response graphing & analysis, and great reviews from the community.

Check it out:
zayareviews.com

I "launched" the site via a /r/headphones post 6 days ago and got a surprising level of engagement, insightful feedback, and tons of reviews. I'd love your feedback too! There's no place for reviews like head-fi.

• How can I make it more useful for you?
• What is missing?
• What is broken, or stupid?
• What headphones are missing?
• Is anything unclear or not intuitive?
• Share your thoughts!
• The site is only useful if it's populated with ratings and reviews, so... go ahead

PS: I just checked and I wrote the first bit of code on July 26th. I've been on paternity leave for the past few months and spent 2~ hours a night coding this thing up (because I've been getting way too much sleep as it is), of course it was WAY more work than I thought it would be (seriously like 20x). I did end up adding Amazon affiliate links, not because I think anyone would end up buying through those links, but hey, if I could eventually fund the $80~ / month running hosting costs that'd be nice. I'm not a fan of ads, so, here we are.

 

[Heyyoudvd]

I love the design of the site and how straight-forward and simple the UI is, while retaining a lot of powerful sorting features.

One initial thought I had is that it might be worth adding a bit of nuance to the review system. It’s 1-5 stars and that’s it, which seems a bit limiting.

If I could spitball, I’d allow for multiple categories - perhaps 3 categories being Technical, Tonality, and the a Performance/Price or Value rating. Each would have its own 5 star rating. And then you’d use some weighted average to calculate an overall score based on these three categories. And I’d include a toggle to switch between absolute score and relative score (ie. to turn on and off the inclusion of the value rating in the weighted average). That way, there’s a rating for how good a headphone is in absolute terms and how good it is for its price range. Also, I’d personally allow for half stars. 1-5 seems a little limiting, in my opinion.

Just my 2 cents. But I love the site. I think you really have something here that could become a very powerful tool as the sample size of ratings/reviews increases.


The other thing I’d consider is the possibility of people gaming the system. I’m not sure of the details, but it’s worth thinking about the possibility of people (or companies) artificially boosting ratings of headphones that they want to promote. So I’m not sure what countermeasures can be taken, but it’s something worth thinking about.

 

[Jose Hidalgo]

I would also vouch for more nuance to the review system. I'd definitely like half-stars. The first HP I wanted to review, I'd like to give it 4.5 stars. 4 is not enough and 5 is too much IMHO.

I would also see some categories, as follows :

• Overall build : 1 to 5 stars
• Comfort : 1 to 5 stars
• Sound Signature (before EQ) : Warm / Warm-neutral / Neutral / Neutral-Bright / Bright
• Relaxed vs. Analytical : Relaxed / Between the two / Analytical
• Soundstage and Imaging : 1 to 5 stars
• Bass ability (before EQ) : 1 to 5 stars
• Bass ability (after EQ) : 1 to 5 stars
• ...

I think the current BMT system looks nice but it's too subjective and sometimes it makes no sense. I mean, look at the LCD2 for instance. Currently in your list, it doesn't have bass, it doesn't have medium and it doesn't have treble ! I guess that's not the way to rate a headphone. Hence the fact that I suggested the "Sound Signature" which may be easier for a lot of people. For example most of us will agree that the HD600 is perfectly neutral. Some will prefer to say Neutral-Bright because of its midrange. But nobody will say it's a warm headphone.

 

[Junglebook3]

I love the design of the site and how straight-forward and simple the UI is, while retaining a lot of powerful sorting features.

One initial thought I had is that it might be worth adding a bit of nuance to the review system. It’s 1-5 stars and that’s it, which seems a bit limiting.

If I could spitball, I’d allow for multiple categories - perhaps 3 categories being Technical, Tonality, and the a Performance/Price or Value rating. Each would have its own 5 star rating. And then you’d use some weighted average to calculate an overall score based on these three categories. And I’d include a toggle to switch between absolute score and relative score (ie. to turn on and off the inclusion of the value rating in the weighted average). That way, there’s a rating for how good a headphone is in absolute terms and how good it is for its price range. Also, I’d personally allow for half stars. 1-5 seems a little limiting, in my opinion.

I think the success of the site is contingent (on many things, but on of them...) is the balance between the interface being slick, and the richness of the functionality. I think that for now it's an easy decision for me to make as far as 5 stars vs. 10 stars, or splitting the 'rating' into subcategories - I'd lean towards simplicity, certainly to start. There's a huge list of things I'd love to implement first, like search functionality, like adding at least 2-3x the amount of headphones, like adding more filters for IEMs, adding fitness and gaming headphones, it's a long list...

Just my 2 cents. But I love the site. I think you really have something here that could become a very powerful tool as the sample size of ratings/reviews increases.


The other thing I’d consider is the possibility of people gaming the system. I’m not sure of the details, but it’s worth thinking about the possibility of people (or companies) artificially boosting ratings of headphones that they want to promote. So I’m not sure what countermeasures can be taken, but it’s something worth thinking about.

That would be a good problem to have as they say. Let's get there first

 

[Junglebook3]

I would also vouch for more nuance to the review system. I'd definitely like half-stars. The first HP I wanted to review, I'd like to give it 4.5 stars. 4 is not enough and 5 is too much IMHO.

I would also see some categories, as follows :

• Overall build : 1 to 5 stars
• Comfort : 1 to 5 stars
• Sound Signature (before EQ) : Warm / Warm-neutral / Neutral / Neutral-Bright / Bright
• Relaxed vs. Analytical : Relaxed / Between the two / Analytical
• Soundstage and Imaging : 1 to 5 stars
• Bass ability (before EQ) : 1 to 5 stars
• Bass ability (after EQ) : 1 to 5 stars
• ...

I think the current BMT system looks nice but it's too subjective and sometimes it makes no sense. I mean, look at the LCD2 for instance. Currently in your list, it doesn't have bass, it doesn't have medium and it doesn't have treble ! I guess that's not the way to rate a headphone. Hence the fact that I suggested the "Sound Signature" which may be easier for a lot of people. For example most of us will agree that the HD600 is perfectly neutral. Some will prefer to say Neutral-Bright because of its midrange. But nobody will say it's a warm headphone.

The bass / mids / treble bars are calculated by looking at the error of each section from the Harman target curve (over-ear 2018 or in-ear 2019). It's not intuitive, but if you look at the frequency response graph of the LCD-2 [1], with the Harman target (normalized to the LCD-2 between 250 Hz to 1k Hz, which is a pretty standard way of doing it), you'll see that it matches in the mids nicely between 250 Hz and 1k Hz, but it has FAR less bass AND treble than the Harman curve. It can happen.

[1] https://www.zayareviews.com/headphones/?brand=Audeze&name=LCD-2

 

[Jose Hidalgo]

In that case I would suggest an improvement : to me it doesn't make any sense to have 3 negative (or positive) values for BMT, because that's not what we will feel when listening (we'll just adjust loudness accordingly). So instead, just calculate the average of the 3 values, and recalculate the 3 values around that average so that the new average equals zero.

For example if you have -56 / -31 / -45, the average would be -44, and the recalculated values would be something like -12 / +13 / -1. That would make more sense IMHO, because that's what our ears will feel for the LCD2 out of the box : lacking bass, midrange presence (in comparison) and neutral treble.

Another example : for the HD600, the -15 / +21 / 0 would become - 17 / +19 / -2.
For the Sundara, -51 / -51 / -8 becomes -14 / -14 / 29. Makes more sense.
Although I'm a bit skeptical about the Sundara, because these numbers do not seem to match the stock frequency response as shown by Oratory. The Sundara is much more neutral and has more midrange than bass or treble "out of the box". I should know, I own a pair. Where does that much higher treble value come from ? I don't know.

It would make sense if you did like Oratory : discard measurements below 50 Hz and above 10 KHz when calculating your BMT. That would yield more accurate results.
Maybe you could also add some kind of "frequency ponderation", because a dip at 10 KHz won't have the same importance as the same dip at 1 or 2 KHz.

 

[Heyyoudvd]

I think the success of the site is contingent (on many things, but on of them...) is the balance between the interface being slick, and the richness of the functionality. I think that for now it's an easy decision for me to make as far as 5 stars vs. 10 stars, or splitting the 'rating' into subcategories - I'd lean towards simplicity, certainly to start. There's a huge list of things I'd love to implement first, like search functionality, like adding at least 2-3x the amount of headphones, like adding more filters for IEMs, adding fitness and gaming headphones, it's a long list...



That would be a good problem to have as they say. Let's get there first

I absolutely agree about erring on the side of simplicity, but I think price considerations are an important factor in providing ratings. I mean, you even included a note at the bottom of every review to remind people about this ("For example, a 5 star rating for $200 headphones doesn't mean that they're better than 5 star $3,000 headphones"). That means it's something people think about and something that can confuse them.

People see scores and have a natural inclination to believe that's how good something is in absolute terms. I think it would be useful to have a clear and concise way to see how good something is for its price vs how good something is in absolutely terms relative the entire headphone market.


As far as headphone additions, can you add the rest of the Grado line? I see the 60e, 80e, 125e, and GW100, but I'd love to see the inclusion of the rest. You can find them here: https://gradolabs.com/headphones/

One more minor thing: When typing the brand into the search box, I think it should search all brands, regardless of whether you're set to True Wireless, Noise Cancelling, IEM, or Audiophile. Sometimes I begin typing a popular name and then I have to remember to switch categories or else it doesn't find anything. Also, I'm not so sure I like those 4 categories. If I were to subdivide the headphone world into 4 or 5 categorizations, I don't think that's how I'd divide it. But that's more of a subjective call.

Hopefully this post doesn't come across as negative, as I really like what you're doing with the site. I'm just offering a few suggestions.

 

[Junglebook3]

In that case I would suggest an improvement : to me it doesn't make any sense to have 3 negative (or positive) values for BMT, because that's not what we will feel when listening (we'll just adjust loudness accordingly). So instead, just calculate the average of the 3 values, and recalculate the 3 values around that average so that the new average equals zero.

For example if you have -56 / -31 / -45, the average would be -44, and the recalculated values would be something like -12 / +13 / -1. That would make more sense IMHO, because that's what our ears will feel for the LCD2 out of the box : lacking bass, midrange presence (in comparison) and neutral treble.

Another example : for the HD600, the -15 / +21 / 0 would become - 17 / +19 / -2.
For the Sundara, -51 / -51 / -8 becomes -14 / -14 / 29. Makes more sense.
Although I'm a bit skeptical about the Sundara, because these numbers do not seem to match the stock frequency response as shown by Oratory. The Sundara is much more neutral and has more midrange than bass or treble "out of the box". I should know, I own a pair. Where does that much higher treble value come from ? I don't know.

It would make sense if you did like Oratory : discard measurements below 50 Hz and above 10 KHz when calculating your BMT. That would yield more accurate results.
Maybe you could also add some kind of "frequency ponderation", because a dip at 10 KHz won't have the same importance as the same dip at 1 or 2 KHz.

I'll have to dedicate time to do some more thinking about this topic, to see if there's a way to minimize the number of headphones with all 3 B/M/T values above or below 0. Your re-averaging method may work out, I just need to sit down and do some thinking. Unfortunately I only have 1-2 hours a day to work on Zaya, what with a full time job, kids, etc, so prioritization is key. I just added a search bar the other day - pretty basic stuff

As a side note I do get rid of everything below 50 Hz and above 10k Hz when calculating B/M/T values. Those extreme edges of the graph are also colored in a more subdued color to signal to the user to not pay as much attention to those ranges. This is based on Dr. Sean Olive's research papers over at Harman.

 

[Junglebook3]

I absolutely agree about erring on the side of simplicity, but I think price considerations are an important factor in providing ratings. I mean, you even included a note at the bottom of every review to remind people about this ("For example, a 5 star rating for $200 headphones doesn't mean that they're better than 5 star $3,000 headphones"). That means it's something people think about and something that can confuse them.

People see scores and have a natural inclination to believe that's how good something is in absolute terms. I think it would be useful to have a clear and concise way to see how good something is for its price vs how good something is in absolutely terms relative the entire headphone market.

I have plans to add "buying guides" style articles. For example for audiophile headphones it could show you the top open back headphones per price bracket - essentially a tier list, but it would pull the data from the ratings automatically.

As far as headphone additions, can you add the rest of the Grado line? I see the 60e, 80e, 125e, and GW100, but I'd love to see the inclusion of the rest. You can find them here: https://gradolabs.com/headphones/

It would be cool if you could add your request to reddit.com/r/Zaya, it's one central place for me to keep track of all of these requests. Another thing to consider is that I don't want one brand to overwhelm the site - Grado has... a lot of headphones. Which one do you think I should focus on?
https://gradolabs.com/headphones/

One more minor thing: When typing the brand into the search box, I think it should search all brands, regardless of whether you're set to True Wireless, Noise Cancelling, IEM, or Audiophile. Sometimes I begin typing a popular name and then I have to remember to switch categories or else it doesn't find anything. Also, I'm not so sure I like those 4 categories. If I were to subdivide the headphone world into 4 or 5 categorizations, I don't think that's how I'd divide it. But that's more of a subjective call.

I added a search bar the other day, so I removed the 'Brands' list in the filters menu. Check it out

Hopefully this post doesn't come across as negative, as I really like what you're doing with the site. I'm just offering a few suggestions.

I sincerely appreciate the feedback!

 

[Jose Hidalgo]

I'll have to dedicate time to do some more thinking about this topic, to see if there's a way to minimize the number of headphones with all 3 B/M/T values above or below 0. Your re-averaging method may work out, I just need to sit down and do some thinking. Unfortunately I only have 1-2 hours a day to work on Zaya, what with a full time job, kids, etc, so prioritization is key. I just added a search bar the other day - pretty basic stuff

As a side note I do get rid of everything below 50 Hz and above 10k Hz when calculating B/M/T values. Those extreme edges of the graph are also colored in a more subdued color to signal to the user to not pay as much attention to those ranges. This is based on Dr. Sean Olive's research papers over at Harman.

No worries, take your time. I'm just trying to help.

I think the re-averaging method is the way to go. You just need to make sure that you have appropriate ranges for B, M and T (e.g. 50-200, 200-2000 and 2000-10000). Once you're sure about that, just apply some math so that B+M+T=0.

On second thought, why get rid of the 20-50 range? Beyond 10000 it makes sense because there are internal reflections and lots of weird stuff (plus the influence of 10KHz+ audio is much smaller than the influence of, say, 1KHz audio). But the 20-50 range should be pretty accurate for most headphones, as shown by measurements. Not to mention that some headphones owe their success specifically to their 20-50 range. I think you should consider it, it would help to keep similar ranges for B and M (e.g. 20-200 and 200-2000).

 

[silent-circuit]

I agree with what several others have already said -- a 1-5 star system alone is effectively useless. Multiple categories are needed; at bare minimum something like "tone" / "accuracy" / "value", or something like that. There's a bunch of really mediocre things on there that already have 5/5 with multiple reviews.

 

[Junglebook3]

I wrote up an update on Reddit: https://www.reddit.com/r/headphones/comments/mbfk2k/zaya_headphone_reviews_update_search_box_weight/

TL;DR

• New headphones - the majority of the AKG, Fostex, and Grado lines
• A new search box, a highly requested feature! Search (with autocomplete) for brands or specific headphones across all use cases (true wireless, IEMS, etc)
• New filters that I personally find very useful and couldn't find on other sites:
  • IEMs - whether the wire loops over your ear or goes straight down
  • TWS - Wireless charging, and active noise cancellation
  • Audiophile - A weight slider! Also known as the "Don't show me Audeze stuff because my neck is that of mortals and not NFL linebackers" slider, also special shoutout to the HEDDphone, good lord

 

[Jose Hidalgo]

It's great seeing that you are finding time to work on this.
Lots of nice stuff, but the most important thing in my opinion, the BMT part, hasn't changed yet. I hope it will soon.

There may be two ways to make the BMT part meaningful (currently it's meaningless) :

• Either B+M+T = 0, which may be the simplest way in my opinion,
• Or you pick an important frequency (e.g. 1 KHz, although I'd prefer the average frequency for vocals, something in the "several hundred Hz" range), and you align your frequency response so that the picked frequency is at 0 dB. Then you recalculate B, M and T.

 

[Junglebook3]

It's great seeing that you are finding time to work on this.
Lots of nice stuff, but the most important thing in my opinion, the BMT part, hasn't changed yet. I hope it will soon.

There may be two ways to make the BMT part meaningful (currently it's meaningless) :

• Either B+M+T = 0, which may be the simplest way in my opinion,
• Or you pick an important frequency (e.g. 1 KHz, although I'd prefer the average frequency for vocals, something in the "several hundred Hz" range), and you align your frequency response so that the picked frequency is at 0 dB. Then you recalculate B, M and T.

I get 1-2 hours a night to work on Zaya so I try to prioritize aggressively, and frankly the tonality stuff is the most complex part of it and often by the time night comes I’m too tired to work on something complex. That being said it’s definitely on the list.

Edit: One issue with the sum of the 3 bars being 0 is that the tonality information has to make sense both for a single headphone, but also to compare headphones, so headphone A that has more bass than B, but proportionally has less bass than mids or treble, still has to have more bass than B.

 

[Jose Hidalgo]

Here are maybe some hints :

1. There's no such thing as "headphone A has more bass than B" if we don't state "at that precise loudness level". Moving on.
   
   
2. The amount of perceived bass depends on the amount of midrange and vice-versa : every register's perception by our ears depends on the other registers. Bass depends on midrange (and to a lesser extent on treble), midrange depends on bass and treble, and treble depends on midrange (and to a lesser extent on bass).
   
   
3. Consequently, in order to compare headphones 1 and 2, only comparing B values would be a bad idea. Here's an example :
   • Headphone 1 = 8, 0, -8 (B1, M1, T1) and headphone 2 = 5, -5, 0 (B2, M2, T2).
   • Headphone 1 has a bigger B, but the truth is that headphone 2 has more perceived bass when compared to midrange (10-2= 8 and 5-(-5)=10). That's why we can't just compare B1 and B2. We need to compare (B1 - T1) and (B2 - T2).
   • "So headphone 2 has more bass, right ?". Well, yes and no. Because even if headphone 1 has a bit less perceived bass when compared to midrange, it would still sound more bassy than headphone 2 to our ears, because its frequency is always descending, and headphone 2 would sound more balanced and slightly V-shaped. Headphone 2 would sound like headphone 1 "bass and medium-wise", but with more treble to balance it all.

So ultimately, maybe you could implement an automated comparison system that would take into account these 4 factors for every headphone :

• Bass factor : Bfactor = xb * (B - M) + yb * (B - T)
• Medium factor : Mfactor = xm * (M - B) + ym * (M - T)
• Treble factor : Tfactor = xt * (T - M) + yt * (T - B)

All the x and y factors you would have to decide on.
xb would be bigger than yb (to minimize treble influence on the bass), xm would be equal to ym, or not very different (because both bass and treble have influence on the midrange), and xt would be bigger than yt (to minimize bass influence on the treble).

You could play with this and see if you come up with something useful.