The idea should be to still have translation in accordance with rotation. If you want to move your POV toward what you’re seeing, you’ll just have to move your head toward what you’re seeing.
That is, if you add an angle of 90°, Y tracking become X for display and X tracking become Y for display, but only for relative moves. The location of POV before angle offset should remain the same.