No, the problem with feedback is that it introduces high order distortion while it corrects low order. For instance, say you have a circuit that produces 2nd order distortion. If you feed back the signal from the output to the input, the fed back 2nd order cancels the generated 2nd order. However, because the amp produces 2nd order, the fed back signal will be distorted, generating a 2nd harmonic of the 2nd harmonic -- that's 4th harmonic. While this new distortion will be lower in amplitude, and indeed some will be canceled by feedback, the higher the order of the distortion the more impactful it is. In other words, 1% 2nd harmonic is no big deal, but .001% 13th harmonic is.
There is a second problem, too, which is that the back EMF generated by the speaker can interact with feedback to inject junk in to the circuit.
This does not mean that feedback is necessarily awful, but it is to suggest that it is not a panacea, and that an amp that measured at 0.001% THD into a resistor load may behave very differently into a real one.
There is a Nelson Pass article on this at http://firstwatt.com/articles.html called Audio Distortion and Feedback, but the link is not working right now.