For music, when I still bought my own music (as opposed to now, when I realize I should rely on other peoples' tastes more than on my own limited one), I used to go to a store called Newbury Comics -- I think it's a Massachusetts-only chain. Has practically no classical music titles and high prices for DVDs, but there are some real steals to be had there for rock stuff.
For classical music, I usually shop online -- better selection, better prices. I'd stay away from buy.com though -- I've been screwed over several times there (although I've made a bit of money off it... pm me if you want to find out how
). Amazon.com is very reliable and has a good selection, and cdnow looks good, although I've never used it.
For audio equipment, I almost exclusively buy used. I can just afford better stuff that way. But for the other items, I buy what I can using shopping.yahoo.com searches -- easier and cheaper than audiophile sites.
For used equipment, the best seems to be here and on Headwize, of course. But barring that, there's always eBay, audiogon, and audioweb.
Why am I rambling on like this? I don't know... but I hope it helped at least somewhat.