My experiences with Columbia House has been fairly positive. They don't carry most of what I like nowadays, but they do have a pretty big selection, and even I could probably find 12 CDs that I wouldn't mind having. My advice to you, however, is to fulfill your membership requirements as quickly as possible with the cheapest CDs that will fulfill requirements and then cancel your membership. The way CH gets money is by getting you on the monthly featured selections (if you forget to send in the response, they'll rape your arse with a $20 britney spears CD). If you check out Columbia House DVD threads, there are people (myself included) who have gotten many DVDs for about $10 a piece (usually expensive one too). I've heard of people joining and quitting as many as 35 times, and CH doesn't really care. They still make money so they don't inquire. I'm not sure about the CD club, but I know the DVD club allows you to have 2 memberships open at any one time. There is one caveat, however... and that is it usually takes them a bit of time to get you your first CDs out to you... like 6-8 weeks. If you have the patience, it can work out to be a very good deal, especially if you do like I say and join and quit.
My BMG experience was less satisfying, but that is because of my own fault. I did not join and quit quickly enough.
