Okay, got it to work. What you have to do:
1) Do NOT use angle correction feature in Driver4VR in FreeTrack settings
2) Use PiTool room setup to center your headset and PSMoveService tracking direction (so now you should see your ps moves moving in the same directions as you move them)
3) Invert axi x and z in PsMoveFreepieBridge FreePIE script (NOTE: I needed that, not sure if you do)
So now it works, and damn it I am rather pissed it took me so long to figure it out.