The "basic" rig consists of 4 main components.
1) The source. This is either a cd player, or a turntable for spinning records. To split this up further, a CD player can be split into 2 parts... a "transport" (spins the disc and gets the digital information) and a "DAC" (digital-analogue-converter) which converts the digital bits into an audio analogue signal. Some people use a DAC with their computer, streaming digital information out to the DAC, for better computer sound. A vinyl rig can also be split into component parts, as many people will have a "phono stage" or a "pre-amp", that amplifies the signal from a turntable before it reaches the next part of the chain.
2) Interconnects and cabling. These range from the absolute basic $2 cables that came with your CD player, to some drain-pipe sized monsters that probably cost more than your house. Their function is to transmit the electrical signals from your source to your amplifier. Different cables have different sonic characteristics, however over a certain level of quality (which I put at about £50 personally, though this is only my personal opinion) the differences in quality become extremely negligable in anything but the highest of high-end systems.
3) The amplifier. In simple terms this makes the volume from your source louder. Headphone amplifiers come in many different shapes and sizes, from the firestone "cute beyond", to the orpheus HEV90. Their main function is to make the sound louder, but also to impart their own sound characteristics on the sound, giving it richer bass, or higher top end. The amplifier in turn sends the signal out to the final piece of the puzzle, the headphones themselves.
4) Cans, 'phones, earspeakers... etc etc. There are many words for them, but these are the most important bit in the system. These convert the electrical signal into sound that you can hear. The main distinction is between "dynamic" and "electrostatic" headphones. Dynamic are by far the most common, and in basic terms they use coiled wire and magnets to move a small speaker cone near your ear and produce sound. These come in many shapes and sizes, and they are the object of lust and upgraditis in many people here. Electrostatic headphones use a clever arrangement of highly charged plates that move when current passed over them changes. These require a specialised electrostatic amplifier, and are often the most expensive. This category includes the much-talked-about HE90, often thought of as the best headphone ever created.
I hope this answers a few of your questions, and hope it doesn't give you too many wallet-hurting ideas...
Have fun in head-fi!!!