Logwed: Personally, I believe that there are people who would be better off towards surviving such an occurence. Like you, it is something I've thought about often and often times, and I agree that we have many people in today's society who can be labeled smart, but then if one digs a little deeper, one could see that many of these people would be ill equipped to survive in a post-apocalyptic environment or a return-to-the-past type society. Look at myself for example:
I'm terrible at gardening = likely be unable to develop enough land to sustain enough growth of food crops to sustain my little old self.
I have no real skills at metal working, woodworking, etc - exactly the kind of skills that would be beneficial to building shelter/tools, etc
I can handle guns, knives quite well, though I only own knives. That said, I have no real ability to make bullets so when I run out of bullets, any guns I have are useless, and in any case, I've never been a good hunter/fisher so that would likely be another failed attempt at cultivating food.
I could go on and on with a list of my shortcomings, but as you said, if any and every person who lives in modern society made such a list, I'm sure most will realize how truly ill-equipped they are to deal with a post-apocalyptic society. No doubt, there are people who would say that given time, you could learn and improve yourself, but in the majority of cases, unless you have the sheer luck to find yourself as part of a group where everybody's strength are compatible with everybody else's shortcomings, your'e not going to have enough time to learn what you need to. And you can't expect to survive long enough to get yourself initiated into such a group if you weren't already part of one. That's why I don't dream about returning to a simpler agrarian based society, but rather, use the fact that I can lead a comfortable, modern lifestyle to better learn the kinds of skills that would be useful in that kind of society. Every opportunity I get to learn from someone else how better to grow food, work with wood, hunt, etc, I take because I know that it would benefit me if I ever found myself in need of said skills.
To put it shortly, I'm in total agreement logwed!