If you mean a headphone amp here and hook it up to an electric guitar, I wouldn't try that. I'll let an EE get into the more technical bits of this, but basically there's a reason why you shouldn't just hook up an electronic instrument into just about any amp; something about the way the waves and peaks are vs mastered/recorded sound.
There are however some guitar amps that have headphone outputs, even the 5w practice amps. Even better, I think I've seen some on eBay that are purely headphone amps for guitars - the 1/4" plug sticks out of its side, you insert it into the guitar jack, then hook up the headphone. Make sure they fit your guitar's jack though, the ones I've seen probably won't work with guitars that have the jacks flush and angled into the body (so you can use straight plugs without them sticking out).