I did something (that I realize was stupid in retrospect) odd in that I started with ME2. I had wanted to play ME3 when it came out, but I somehow convinced myself that I should go back and start from the beginning. As I never found the time to do that, I ended up not playing ME3 at all. From your remarks though, I don't think I'm missing anything mind-blowing.
It's definitely best when played from the very beginning.
ME3 is still rewarding in its own way if you import your character from ME2. Not *as* rewarding as if you started from the very beginning, but nowhere near as bad if you start with ME3. In that case you start off with certain characters already assumed dead (whom you could have saved otherwise), and things are pretty depressingly bleak. Not to mention the story is just fun to play through. It's especially well told in the first game, and the social interaction is a lot of fun.