Audio used to be taken seriously. Some decades back, the three biggest purchases for men, in order, were a house, a car, and a stereo.
Now audio is just a small niche and isn't taken that seriously.
Back when audio was a major hobby, it was mostly driven by science and measurements. Audio magazines took measurements seriously and there was actual progress. Then, by the 1980s, most of the tough engineering challenges were solved. Speakers have improved somewhat and digital servers have become very popular. Aside from that, the industry has stagnated.
Instead of improving quality or lowering prices, the audio industry went the fashion route. They turned audio gear into status symbols and competed for higher and higher prices. Consumers lost interest.
Around the same time, objectivity was thrown out. All sorts of snakeoil got pushed on audiophiles. Sure, there's a cable controversy around here and other audio forums, but 99% of the public think cables, magic rocks, green markers, et al. are pure BS. There's zero scientific backing for any of that, so the vast majority think that audiophiles are a pack of deranged lunatics. The only contact most people have with audiophilia these days is when someone posts the Amazon reviews to a $3,000 cable to a news aggregator. Then 100% of the comments mock audiophiles.
If audio is to be taken seriously, the pseudoscience and mythology has to go. You can find believers in every audio forum, but any time you suggest good headphones or speakers to outsiders, they associate it with snakeoil.
The other problem is pricing. Manufacturers keep recycling designs and upping the price. Prices have to come down.
Keep in mind that audio remains very popular. The iPod was a genuine advance and Apple sells millions and millions of them. People are interested, they just don't want to pay too much for something they associate with BS.
Another huge problem is today's music and the state of music education. Music ed is more or less dead. People used to grow up playing instruments and hearing live, acoustic performances. That rarely happens today. Instead, people are exposed to badly recorded music on bad speakers. They have no frame of reference for good sound. If all you've heard are crappy car subwoofers, you start to expect bloated bass. When headphones produce bass that's similar to real life, it becomes "weak bass." Today's popular music doesn't help much, either.
TL/DR: Snakeoil has poisoned the public against audiophilia, everything costs too damned much, and the public can't tell good sound from bad any longer.