Ya, make sure your not running optical from Xbox --> TV --> to mixamp. A lot of the time TVs screw up the DD 5.1 signal when you pass through them.
When I had my old xbox I used HDMI out to the TV, and used the microsoft audio dongle adapter to go Toslink straight into the mixamp.
Now I have the xbox slim, and that has a dedicated optical out port.
If its doing pro logic it would sound like the sound is just pushed out in all 4 directions, but there is no clear directionality.
Dolby Headphone should sound cleaner and have discrete directionality in the sounds that you can hear.