The two most obvious advantages would be cost particularly if you need a large value say 100uf
Space is the other big advantage true metal foil capacitors tend to be substantially larger than a metallised films fitting them in can prove a challenge.
Cons sometimes you can get a smearing effect but I've used bypasses extensively and have never experienced it myself though I have tried bypasses that I've not liked the results of using but that's more personal taste.
A well worn path in the DIY community is using inexpensive low uf value surplus military russian teflon capacitors off of eBay as by passes. Most often it's a case of suck it and see but when the capacitor is say the cost of a coffee there's not much downside to experimenting.
Have a Google about on russian teflon bypasses
Edit here's a link to a piece by John L who posts here with some thoughts on a few of the more popular ones
http://www.enjoythemusic.com/diy/0708/capacitor3.htm