I'm thinking that maybe once the amp detects the presence of a headphone it is bypassing the preamp section, for the most part, and acting mainly as a poweramp. I've seen this situation occur before, although not with NAD gear.
The only other reason I can think of is that you have the RCA's running directly to the jacks labled main in, which would have the amp acting solely as a power amp. Same as terrymx said. Make sure that your jumpers are running between the pre-out and main-in and that your are connecting the RCA's to a line level input.
One other thing, make sure that if you have any kind of 'defeat' switch that it isn't enabled as it might disable the functions described, although I've never really seen a defeat switch take out volume control, just the tone controls.