I believe such a test would be simple to set up now that we have computer audio. A range of different headphones and systems could be measured by Head-Fi on whatever test rig as a starting point. It doesn't have to match anything that was done in the past actually. As long as it is a measurement rig that is now commonly available. EQ the rig to whatever starting curve they wanted to test. 1000's of different listeners would then spend 1/2 hour adjusting a simple 10 band graphic equalizer while listening to a few various tracks.
I'm sure some notable statistical modes would appear with a number of samples in the 1000's. Head-fi already has expert opinions on a starting point for a modified curve which we could start with which would streamline the process with the listeners. Maybe a new Head-Fi curve would find people adjusting almost nothing.
Who would want to do it? Head-Fi has staff that are already spending time doing individual measurement and extensive reviews and are already traveling to all of the shows with Headphones.com already dragging a truck load of gear. Audeze is dragging a truck load of gear. Sennheiser, Burson, etc. Manufacturers would bid to buy in with support to have their equipment featured. It would be a draw for the event for the patrons who would look forward to participate.
I'm afraid I don't share your optimism. On the bright side, I'm nobody, not an admin and not an organizer, so my opinion has no impact whatsoever. On the other hand, I can do nothing for you.
It's just that projects in general have more issues than we think, and take more time than planned. It's the one reliable rule of project management.
Anyway, here is my very personal opinion:
-Good luck organizing something for even 200 people to spend half an hour on anything other than what they came to do.
-Hope for a miracle when asking all those people to meaningfully handle an EQ. I would expect your plan to get us the noisiest and least exploitable data. Sorry, but that's likely what would happen.
For reference, Harman decided to have listeners chose between a handful of curves/headphones. In another trial, they offered a basic bass and treble sliders with predefined curves and frequency. And it went on like this in very incremental steps instead of leaving full EQ control to the listeners. I'm guessing that's because they shared my views on what the average untrained guy will do of an EQ and "set what you prefer" as a guideline.
And just as a basis, I don't see evidence that there is a much better curve waiting for us when using good old stereo albums + headphones. Sure, I see various reviewers and websites implementing their own variations over Harman, based on how they and, like 3 of their friends, agreed with the idea. I mean no disrespect, they have their reasons and some(minority) are grounded in logic. It's just that statistical significance is a powerful thing to have for this topic. They usually don't have that, so all in all, I have a hard time seeing how any of that can stand up to a dozen of pretty rigorous experiments over a decade, leading to the Harman curve for headphones.
And if not that, what would be the incentive to look for a new curve?
For the IEMs curve, I couldn't say if Harman had enough transferable data from previous headphone and speaker studies, to allow such a fast result(comparatively!).
And yes, with all that, I'm still in favor of just about any extra testing, to serve as replication study if it's good, and for participant to hopefully learn about testing or/and themselves.
