Most subwoofers have RCA input jacks and speaker level inputs. Additionally, subwoofers may have a low-pass filter which filters out any high frequencies so that they the subwoofer doesn't try to play them. Depending on the subwoofer, this low-pass filter may be defeatable or may only work on the speaker level inputs. It just depends on the model, so read the subwoofer's manual.
Amplifiers designed for home theater purposes (what you find at Best Buy) usually have a single RCA out jack labeled "LFE" (low frequency effect) or "SUB OUT." In this case, the amplifier either sends the LFE channel in a 5.1 mix (i.e. from Dolby Digital or DTS) directly to this jack or mixes in low frequencies from other channels and sends it to this jack. This LFE RCA jack should be connected to either the left or right RCA input jack on the subwoofer. It doesn't matter which one since they get mixed together by the subwoofer. Additionally, you should turn off the subwoofer's low-pass filter since the amplifier takes care of that job.
Stereo amplifiers, on the other hand, usually do not have an output jack specifically for LFE signals because they do not mix low frequencies into a signal channel like a home theater amplifier does. In this case, you can either connect the pre-out RCA jacks (both left and right) to the subwoofers RCA input jacks (both left and right), or run another set of speaker wires for both the left and right channels from the amplifier's speaker outputs to the subwoofer's speaker inputs. In either case, you will want to enable the subwoofer's low-pass filter since you will be sending the full frequency spectrum to it.