Since I have a crapload of homework I'm trying to avoid, here you go:
The first and most important component is the headphone. It has the biggest impact on overall SQ and should almost always be your first buy when building a new rig. The exception to that is when you start to get into the upper-tier, power-hungry headphones, for which an amp is a must - for those you shouldn't bother getting the headphone without getting an amp at the same time. When I plug my Darths straight into my laptop for example, they sound awful - there is a lot of distortion and a complete lack of control over the bass, making any sort of enjoyable listening all but impossible. When I plug them into my Arietta amp, however, all that disappears and they sound wonderfully articulate and musical - whereas I was being constantly distracted before, I can now effortlessly lose myself in the music.
The second most important component is the amp (unless you are running very low-impedance headphones, I'd say <50 ohms). Again, an amp is a must to drive the more power-hungry headphones. It should be noted that the impedance alone is not a good measure of how power-hungry headphones are. The Darths are only 80 ohms, but needed a home amp to shine (my solid mid-tier portable couldn't quite cut it, even though it was ranked to drive cans up to 600 ohms). It's essential to know that amps are not only about power, but quality. Each amp also has it's own sound signature too, which you'll want to research before settling on a purchase. I'd recommend getting a desktop amp, as pound for pound they are much better value than portables.
The third most important component is the source (again unless you have very low-impedance headphones, which don't really require amping). If you are playing music off a computer, you have two basic options for improvement - getting a higher quality sound card, or getting an external D.A.C. (digital audio converter). You'll also want to encode your music files at higher bit rates or use higher-quality formats like flac, apple lossless or aiff for the best possible sound quality. As with everything else, the more you spend, the better results you get, but there are plenty of budget products with killer value out there.
Another consideration - cables and interconnects. Though the general consensus is that cables have the least impact on sound of all these components, I'm reluctant to place it last (so I haven't, it's kind of kept separate) as I've heard first hand what a big difference it could make. I had two iPod line out docks (cable used to connect iPod to amp), a budget SPC wire one and a top-class cotton-insulated one, and the sound quality difference between them was immediately apparent. music just sounded so much fuller on the cotton dock.
Even though I've ranked these, it's important to know that any rig is, to a degree, limited by it's weakest component. This means you should eventually seek to address all of these areas if you want to get the best out of each.