The AKM4358VQ DAC chip inside the Denon should be fine and not the limit of your system. To me the improvements you are seeking are most likely going to be found in a more fun sounding headphone like the Denon AH-D2000. What prompts you to upgrade? Does your music sound too bland? Give us a list of your top 5 CDs please.
Sure you can spend more, but unless you are going to do a direct comparison you may be wasting a ton of cash.
I was unable to find the complete spec sheet for the Kenwood CD-323M, but I imagine it has a low signal to noise ratio and perhaps a narrow frequency response. If it were my system I would first go with a higher quality CD player and then replace my headphones or use speakers.
Things to test your system with:
1.) Go here and see if your current system allows you to hear the difference between mp3 or not
http://mp3ornot.com/
2.) Listen to this track and report back with that you hear. Can you hear the nice kick drum? Does it sound like it is moving air in your head? Can you hear the nice metallic sound of the guitar strings? Do the vocals sound level with everything else or are the distant or muddled?
http://www.youtube.com/watch?v=VjATZYHs4Tc
3.) With nothing playing, but all of your components plugged in, and the volume set to 70% what do you hear? Any hiss or noise? With the volume knob all of the way down to zero what do you hear? Any hiss or noise?
4.) Do you hear skipping that is not caused by a CD scratch?
I know Headroom ( headphone.com ) gives this statement which holds true to me.
"As mentioned above, the source is generally (keep reading!) the most important aspect of an audio system. We build a variety of digital to analog converters (DACs) to further improve your audio source. HeadRoom amps with internal DACs, and our stand-alone DACs, both contain high quality DAC chips in our designs; and, these contemporary high-end digital recievers and DACs are extremely good at reducing or virtually eliminating jitter from the digital bit-stream. Now, here's the exception to our rule about sources being so critical: when constructing a front-end for your stereo system, it is much less important where the digital signal is coming from when a very high-quality DAC is used. We've even found, in fact, that sometimes very expensive CD transports sound only slightly better than the digital output of inexpensive CD players or USB audio outputs from computers when using a good DAC. So, keep that in mind when putting together your rig!"