Hi redshifter,
There are lots of car with "fun&cool factor". I think you have to be more specific. What type of car are you looking for? How many passenger do you normally carry and what are your major concerns?
Since you have used a large passenger car before. Change into smaller sedan might make you feel a bid uncomfortable. Here are my primary opinions
Passenger Sedan
Japanese : Honda Accord,Toyota Camry ,Mazda 6 and Nissan new Maxima or Altima. These car are fresh,more safety,more responsive and morefun too drivethan the previous models. Honda has higher resell value and has good reliability. Camry is one of the best car in terms of safety and relaibillity but less fun to drive and has weird styling in my opinion. Mazda 6 is probably one of the most fun and responsive car but not as fast as Accord and Maxima/Altima and has lower resell value. Nissan has improved all their models to be more attractive for younger customer. Altima and Maxima should also be very inteeresting for those who don't want to drive Accord or Camry likes others. Styling depends on taste. I also like Subaru Legacy but do not recommend since it's old model and not sure about maintenance cost on Boxer engine and 4WD system. But it will be a goood choice on snow.
Not recommend new Subaru Legacy since it's smaller passenger car and maybe slighly noisy inside (it is made to drrive fun).
And also styling,though i like it, could be out of trends pretty soon.
German : BMW 3 series - base model ,Audi A4 and VW Passat VR6. With BMW is most fun. Audi has 4wd and is best organized and best assemble quality. Passat is a good choice if BMW/Audi are out of your budget.
American : Lincoln LS V6 and Cadillac Citera are good choices.
Their skidpad are impressive (oveer 0.80 g for sedan!). Too bad that LS looks too dumb and Citera could be too wild for someone's taste. Both are sharp handling though.
Also for the station wagon- Mazda MP5 is only smaller car under
20k that i would recommend due to the cargo use. (Which is why I havn't mentioned about Sentra SE-R ,MP3 or WRX).
For around 300 grands - goes for Maybach 57/62 or RR Silvercloud
NEO
PS. For me, i would go for A4 due to superb finishing qulity,4wd system,lifelong styling and unique feeling. (I also like Citera too)