So I got the transparent audio hdmi cable and tested it out and dsd still wasn't working.
So I decided to give a hail mary and uninstall all the amanero drivers and reinstall (should have done this first

. Then when I opened up jriver to select the audio output, a new option popped up that was not available to me before. The combo384 asio 1.03 driver.
I was previously trying to use the wasapi driver and it wasn't working.
After selecting the asio output, it works fine now at DSD512. Sounds amazing.
So afterall it is a configuration issue and the AVPLAY cable is fine from di20he > r8mk2.

Edit : it may help if you reinstall the drivers with the i2s cable connected for about 30 seconds first, the driver may not install correctly if the output device is not recognized as present by windows.