hi.
this is by design and allows you to aim what requires to have two controllers in line with eyes what physically is not tracked by leap motion. my assumption was that you get used to in few minutes which was what I found during testing.
also distance from you head is a bit changed to allow you move controller right to your head while have your hand still bit ahead and track via leap.
if that is needed I can provide option to map physics 1 to 1 to controllers however imho it makes movemenet more limited.