Hmm! That's interesting, but I don't fully understand... You mean to say, if you sit to the left of the sweet spot, it will sound like you are sitting off to one side? Uuuuh, could be realistic... But can you make it so you're always sitting in the sweet spot?
If you don't use the head-tracker, it will always seem that sound is coming from the speakers exactly as they were when you made your PRIR calibration, which presumably was right in the sweet spot of the listening environment. No matter where you really are now in your home theater, the speakers will always "seemingly be" directly in front of you and arrayed around you, completely unchanging no matter what your actual head angle or seating position is. There is no head-tracker effect whatsoever.
The point of the head-tracker is to keep the virtual speakers stationary in your virtual listening environment presented through the headphones, adding an additional sensation of virtual reality. Just as turning your head or moving off-center from the speakers in the original measured listening environment represented by the PRIR would give signals to your brain to confirm that you had moved or rotated your head, the head-tracker purpose is to produce that same effect via headphones. So if you eliminate the head-tracker, you also eliminate all such cues and thus all such effects. And you also eliminate all such "virtual realism" based on head angle and seating position in your playback theater.
To each his own, but for me I have been operating with my A8 since 2009 without using the head-tracker. I know, I am a heretic, but I am never turning my head when I sit in my chair watching a movie or HDTV on my display. I never move off-center, and I never turn my head. I just sit still, looking straight ahead anyway. So even if I used the head-tracker the virtual audio results would be the same as they already are, with me sitting in a perfect viewing/listening position (assuming I actually had a real multiple speaker system in my playback room, which I don't). I'm perfectly satisfied having a "frozen sound field" through my headphones that exactly matches my PRIR measurement in the original listening environment.
To be honest, the most important thing is to be lucky enough to have your own personal PRIR measurement performed in a truly superb listening environment (e.g. AIX studios in LA). Borrowing someone else's PRIR, or using a "stock delivered PRIR" is NEVER going to produce the effect possible from having your own personalized measurement no matter how good or bad the listening environment is that you are actually trying to capture. Having your own personal PRIR measurement in that superb (or any, for that matter) environment is the only way you'll really have an accurate "duplication" of that environment when you use the PRIR for playback. Otherwise, it's always going to be like using somebody else's prescription eyeglasses instead of having your own, but as applied to hearing.
All things considered, and if some compromise must be made, THE most important factor to getting truly the maximum effect and realism with Realiser technology is to get your own PRIR measurement made from a reasonable (if not superb) listening environment, as it will reflect your own ears and auditory structure of your head and thus playback virtualization will be optimal for your brain's hearing system. Head-tracker adds some virtual realism, but having your own personal PRIR for ANY listening environment is really what it's all about.
Is set up complicated?