I have now changed the 0.1uF film caps to 100uF electrolytic ones, and it all seems to work now! Thanks for that!
BTW, is there any downside to using electrolytic caps instead of film caps here? I'm not going to get my hands on any right now, but I might order some when I start ordering stuff for my next project.
Also, my dad says that using electrolytic caps in the output stage isn't good, but I've never noticed anything. Any merit to that?
Remember that it's a Starving Student amplifier, so some shortcuts have been taken.
Basically, there are several types of capacitor construction. One is electrolytic, another is film. Electrolytics, quite frankly, are the only technology that can economically produce uf's in the hundreds. So, electrolytics range in size from about 10uf to 2000uf or more. Film caps range from about 0.01uf to 10uf. A polypropylene or polyester film cap of 10uf is about the size of half a pack of cigarettes. A film cap of 150uf might be as big as a typical headphone amplifier. Some people have built amps with film caps that big - by paralleling more than a dozen 10uf caps together. That's an extreme case, but you can see why film caps are limited to lower values of uf.
So, as you essentially asked, "What's the difference?" Well, electrolytic capacitors do wonders for power supplies, but when you put one in the signal path - they sound like a dog's rear-end. Electrolytic capacitors are reasonable size for many hundreds of uf's, but they are slow and add significant distortion to audio high-frequencies. They're fairly OK for bass frequencies, but that's mostly because the human ear typically doesn't discern a lot of detail in lower bass frequencies. It's more of an impact feel, rather than nuanced detail.
You will see electrolytic capacitors made in special "audio-quality" selections. These versions of electrolytic capacitors are almost always more expensive, but less effective in power circuits. Their ESR is higher than other electrolytics and their ripple current ratings is much lower. Both of those qualities are important in power circuits, but less so in audio signal circuits. Typically, mfrs don't even publish the ESR and Ripple Current specs for those caps. So, audio-quality electrolytic capacitors are designed to sound better, but not necessarily sound that great. Some of the more notable audio quality caps are Nichicon Muse KZ, ES, FG, etc. Elna Silmic RFS II and Elna Cerafines are also great audio-quality electrolytic capacitors. Black Gates used to be the Holy Grail of electrolytic audio capacitors until they quit being made. I've found that Elna Silmic RFS II capacitors are every bit as good, though.
Film caps, on the other hand, sound better than any other type of capacitor in the signal path. It's difficult to put a number on things like this, but let's just say that film caps probably - on average -
sound 10 times better than any electrolytic capacitor - audio-quality electrolytics, included. Even so, there are special audio-quality film caps available, too. In fact, the most expensive capacitors for audio applications are boutique, audio-quality film caps. They can run into the $hundreds of dollars for something like a 10uf capacitor. That said, there are many film capacitors that are used for their speed in power supplies, but can sound quite good in an audio application - certainly, better-sounding than ANY electrolytic capacitor. These are typically box-style film caps like WIMA MKP10, FK4, or similar box film caps like Vishay-Roederstein and Epcos. You can find these capacitors for a few dollars each in sizes like 1.0uf. They won't sound as good as a Dueland or Jupiter, but will blow away any electrolytic capacitor in terms of sound quality.