can you check if shaking of red rectangle is not related to your hand?
because when you use kinect + xbox controller then xbox controller is not really used for hand tracking. i realize that this might be confusing.
if you select virtual trackers for hand tracking then it will respond only to controller.