I'm assuming from your post you are using iTunes. I'm hoping you're using a Mac, since the software I'm about to recommend is Mac only. There may be a PC software package that is similar though. EDIT
: looks like appophylite posted a PC version.
Getting song tracks to sound equal volume can be tricky. Part of the reason is that the way software determines equalization is pretty much up to personal preference. If the software only set all the peak volume values of a set of tracks to the same level, you can imagine that a song with very minimal dynamics might sound wholly louder that one that had very wild dynamic range.
There are very good normalization options available that will ensure that songs sound very very close to the same volumes (radio stations use such hardware) but the fact is that you are normalizing the music, and thus altering it. Besides if you were to normalize your entire library of music, it would take probably months of computer time to do so with software normalization.
Ok, ok, so I haven't answered your question yet. I'm getting there! So you can see that if a piece of software is going to scan your music, and then raise or lower the virtual "volume knob" to compensate, how much that knob should be turned is up to personal preference. You can't simply go by the loudest or quietest sections of music. You need to look at the entire song, and come up with a compromise.
This is why the iTunes "Volume Check" option is mediocre. It's main goal is to be unobtrusive to the user, meaning FAST. It does a really quick, and I mean QUICK scan of the music, and then turns that virtual volume knob. It's not accurate by any means, but it's usually better than nothing, and as I said, it's quick.
So enter a piece of software called iVolume
. Unfortunately, it is not free but I believe it has a demo period. This software does a much slower, and thorough look at the music (read: it takes much longer to process your music than iTunes would) and applies a different model of sound adjustment.
Note that iTunes would typically take 2-3 hours to scan and adjust my 40GB (3 days running time) of music with it's volume adjustment. iVolume took about 20 hours to do that same library. So it's about 10 times slower (on my dual processor machine), but it's much better IMO. The good news is that adjusting new music that comes into your library isn't hard, and since you're likely to only be doing a few CDs worth at a time, it only takes a few minutes if that.
I also like that iVolume changes the volume adjustment for each song, and archives iTunes' volume adjustment so that you can quickly go back to iTunes' adjustments if you like. Oh, a quick note here, leave the commenting option in iVolume on. I turned it off, thinking it was messy that it was tagging my ID3 comment fields with a date, but then I realized later why it was necessary. I'm not going to go into detail here, but just know it helps with song synchronization across multiple devices, and it's a good thing to leave it on. I found out the hard way, so I pass that tip on to you.
Anyhow, I hope this helps. I found this software to be the best non-normalization volume equalizer for iTunes.