Mr OP, I agree with what you say. For people who care about such things, packaging and album-art would be major selling points. I personally love to look at a nice CD "package" as I'll call it, I just like a need tidy case and artwork, doesn't need to be special but it still adds appeal.
But do you think many, even most, people care about this stuff? I don't think so personally, and it's unfortunate for CD.
There are plenty of awesomely packaged CDs out there. That won't make people buy them. (it'll contribute to me and you buying them though since we like this feature.
I buy CDs for many other reasons also.
I'm probably a simple person, but I just like going to a shelf, selecting a CD with my hands, taking the CD out of it's case, putting it into a CD player, maybe selecting tracks if I have to.
I've thought that this is what some of the appeal for vinyl must be too? I'm not sure, I'm too cheap to invest in that scene.
Also, I like having a central, reasonably permanent and lossless (i.e. in it's distributed form) version of music (CD quality is ok for me, I've never heard SACD... living in Australia sucks I assure you, we don't have sub $100 universal players!). Computer storage just conflicts with years of using my hands to pick things up and read things in an unconstrained way. Yes this is personal preference.
I'm also the sort of person who prefers to write on paper instead of computer, to do hand/head arithmetic instead of using a calculator and the kind of person who likes to prepare fresh food vs. eating anything from a packet.
I hate to be so cliche and general. And I don't mean any disrespect to anyone, everyone has their own reasons for doing anything.
I just think that people want their music served like their food and like their whatever else (be that good or bad, I won't judge). They find it convenient to do things as they do. Convenience will beat anything, including fancy packaging, in my opinion. I'm not grouping convenience with laziness - that is not fair. It is possible, but I don't know if it is so.
I do things in a slow, lumbering manner. I can't do convenience because I just have a preference for procedure and seeing the results of intent form with effort.
I like the "romance" that the inconvenience of CDs provide. Flicking through tracks on a tiny screen on a music player doesn't have the same appeal to me. I'd rather pop in a disc and look at the back of the package.
And maybe I'm really strange but I kind of form bonds with CDs. I see them a lot, use them heaps.
One other thing is that I'm an "album" person... I like to listen start to finish to CDs. I like the cues that get created when you know which song comes next (sure playlists can have this effect also... if you use them that way). A fair percentage of people by my reckoning prefer select tracks from various artists, and just hit shuffle.
Again, it's different uses of music, different appeal to different people. The non-audiophile probably isn't going to be as obsessed with music and form connections with albums/works on a whole. As such CDs again lose out to the ability to have random tracks as you desire (via retail or more usually pirating).
Disclaimer: clueless guy about to ramble and make unsubstantiated claims.
I don't think CD is going anywhere. Would it be fair to say that bluray and hd-dvd have gained market share at a greater rate than SACD/whatever has in audio?
I don't know, but I'd guess with some confidence that the answer is yes.
I'd also guess that portable audio exceeds home theatre in market size (though perhaps not total monetary value)? Why so slow to change formats? My guess is that blu-ray/etc is a new way to sell high-def screens in the same way that MP3 was a way to sell new portable music players... i't's not just the format their trying to sell, it's all the other stuff.
Formats are not about the consumer in my opinion. They're about the best way to make money for companies. There's nothing wrong with that I don't think, we choose to live in this sort of society, we all want to make money
It can just be unfortunate for some people, but that's the way it goes.
I'm way out of my league discussing this stuff obviously. Hopefully I can be slowly enlightened.
So long story short: the average consumer probably has different aims and needs to you and or I.