I think, in general, you need to get your hands wet with a decent/good pair of headphones first. Who wants to listen to a Marantz with earbuds or some Sony's that they got from Circuit City or Best Buy? Plus, as others have said, it's easy and cost-effective to sample different sound signatures/house sounds with headphones. I would also think that the cans make the most difference with sound taste. Can you make a Sennheiser sound like a Grado with a certain source? Of course I suppose it also depends on what kind of source you're looking for and your budget. For instance, I'm all digital-/computer-based, so my source was hardly over $100 (hmm now that I think about it my portable source cost more than my home one ..

); for a Redbook or vinyl enthusiast, a really good source runs in the thousands (right?).
I suppose in some ways source-first makes sense, because if you know you have a decent source, you can rest assured as you dabble around with difference can house sounds that you're hearing a good fraction of the headphones' potential, with the amp as the final touch on the head-cake.
So whether or not source or cans come first depends on your situation IMO. Either way, I'd say amp third.