Are you using iTunes?
Here's the way I think it works, based on what I've read on various forums. When you launch iTunes, it reads the settings in Audio Midi Setup. If a given song has the same sample rate as this initial Audio Midi setting, then all is fine.
However if the iTunes track that you're playing is not at the same sample rate as Audio Midi Setup (as determined at the time of iTunes launch), then iTunes will resample the track in order to pass it at the same sample rate. So resampling happens, in this case.
If you change Audio Midi Setup while iTunes is open, then iTunes will continue to try to match the sample rate of the initial Audio Midi Setup values. However, the Audio Midi settings are now different than the original. So the consequence is that Audio Midi Setup is receiving something other than what it expects -- and so it will also resample to get that incoming stream from iTunes reset to the sample rate that it has now been set at. So it's possible to end up forcing the over all system to resample twice.
If you want to change the Audio Midi Setup sample rate -- and you want to do this cleanly -- then you need to exit iTunes, change Audio Midi Setup sample rate, and then reopen iTunes. And what a bother this is!!!
Many feel, by the way, that when iTunes resample that it does a better job than when Audio Midi Setup resamples. However I have also heard others say that they like it when iTunes upsamples a 44.1 kHz track to match a 88.2 kHZ Audio Midi Setup setting. YMMV.