Please note that the 3.5mm analogue input does not feed directly into the amp section.
This qobuz review shows that it first goes through an active low pass filter based around the NJM-2115, and is then digitised at 24bit 96kHz by a Burr Brown PCM9211. Hence how it will deal with a 300mV input depends on how the Marantz engineers have designed it. Ideally the active filter should be driven into soft-clipping before the ADC starts clipping.
I would imagine that you might need to attenuate your 300mV output a little bit though, -3dB might be enough (= half amplitude). It also depends a bit on what the 300mV output level of your amp really means. Is it 300mV peak, RMS etc.? To be on the save side you might want to attenuate -6dB or so (= 1/4 amplitude).
Bear in mind that the PCM9211 ADC gives you 24bits of dynamic range to play with, so you can probably afford to be a bit conservative with the attenuation. Do keep a eye (ear) on the noise levels though.
If I find some time I will try to o-scope a function-generator signal through the HD-DAC1 to see how the analogue input clips, and at what levels.
In any case, I don't expect the use of an RCA to 3.5mm cable to affect the sound quality much at all, provided you buy a good and short one, or make one yourself.