- This topic has 60 replies, 18 voices, and was last updated 4 years, 3 months ago by
.
-
Posts
-
Hi, any chance of adding support for Windows Mixed Reality? It would be really cool if Driver4VR could use the Kinect to supplement the hand tracking.
I have a Samsung Odyssey and this would be a great feature to have. Is it even possible for the kinect hand tracking to supplement the Windows Mixed reality hand trackers?
The issue today is that the Windows MR cannot see your hands at your side or if they are too far above you/behind you. If Kinect can step in somehow to augment that that would be brilliant.
would really like to see this if possible. im am working on getting my dell visor and kinect v2 tracking together but am having a little trouble.
i thin that is possible but it would depends on few things how it is actually implemented.
sinece wmr is 360 degree and kinect only 180 you would get only small improvement.i was assuming wmr works great based on reviews i could see on youtube?
@gmodoza,
let me know what problems you have.From what i am seeing when i try to use it the trackers tend to be to the somewhere around me but not under me. And it is tracking but it seems and though it us inverted, when I kick forward, the little white ball goes the opposite way. So I guess it’s some kind of orientation problems. There both tracking just not connected or somethin g.
I’m actually buying into a Windows Mixed Reality headset this month (Dell Version) which is making VR a tad bit more affordable; (since its $120 cheaper than Oculus (not including the additional $60 sensor for fully working 360*) wallet approves.
I’m making this comment just to toss my hat into the ring of people interested in using WMR with Driver4VR for Body/leg tracking, and whatever else can benefit from using it. 🙂 (v2 kinect)
Hi Greg,
Yes, Windows Mixed Reality works great! But yes, you do lose hand tracking sometimes. For me it is very tolerable, but it would be awesome if there was a way to minimize that. If you’re interested in picking one up for testing, they have gone down in price significantly. I’m seeing a bunch on ebay for around $200 US.
@imfm
do you have kinect one ?ok. i think two topics might be confused in this thread:
1. does driver4vr supports properly body tracking?
2. can driver4vr supplement drawbacks of WMR system.I would like first to address first topic and I am going to get my hands on that this week.
For 2nd topic, I will also test and see if there can be benefit from using Kinect. WMR is 360 degree system. Kinect is front facing. so it will potentially solve 50% of tracking problem for you. Depending on which direction you face.
Hello, I wanted to add myself to the poll of people wanting WMR support, and if possible, change my VR setup to WMR (I’m not sure how to change it in my profile). I just got my Dell WMR headset and was able to give it a test run with Kinect today, unfortunately, I was trying it with Kinect, but I keep having a few issues.
1) For some reason, when the Driver4VR is running, seems like the controller is prone to lose pairing and jitter a LOT. The headset is fine, though.
2) When doing the calibration, I am doing head off, hands off, bodytracking on. I am able to calibrate the body and it looks like it gives little error even on the stats, even the skeleton in the pink screen looks pretty good, however, when trying it in a VR experience, the Driver4VR bodytracking controllers are always on the ground or spazzing about and screwing up the WMR controllers. (Also, trying it in VRChat, instead of the balls, I see additional controllers with red buttons, which are on the ground)=
3) The VRChat avatar is pulled downwards, probably trying to reach the controllers on the ground
4) Would it be possible to add support for the WMR controllers to act like Oculus Touch? I tried emulating the Oculus controllers but VRChat still detects them as VIVE. This is useful, because that way we could comfortably use the joystick in the controllers to move the character
5) At times, when finishing the VR experience with Driver4VR when playing a SteamVR game, the headset goes “bluescreen”, and the mixed reality portal won’t respond anymore, hanging up the process explorer for some reason. I haven’t had this problem when not running D4VRI will post more issues as I find them. I really think Driver4VR has great potential, but it’s a little “raw” at the moment. If you could add support for WMR, I will probably purchase another license for my wife, as she also got a WMR for her computer. 🙂
EDIT: I forgot to mention, I am using Kinect 360.
hi. i was to get wmr to try this week but for some reason it did not work. i am trying to get it anyway.
can you send me info by contact form ? we can meet on vrchat and I will try to understand where problem is even now.
thanks.Hi, I’m interested in buying an xbox one kinect for my WMR Dell visor.
1) Can driver4vr be used just for lower body tracking?
2) Does the xbox one kinect interfere with the controller tracking?
Thanks!i hvae not verified it personally but I hear that One can interfere. i am trying to get my hands on wmr so will be able to confirm.
Greg, depending where you are living right now, you can get a WMR off Amazon.com for a much cheaper price than usual because of an offer Microsoft did.
People seem to argue that either Dell and Lenovo are the best ones. We got ourselves Dells, although I have a problem that from time to time the headset disconnects for some reason. (Seems like the specific USB port you connect it to might have an influence on that, there are some specific USB 3 ports in my motherboard which are pretty stable), but she doesn’t have that issue herself. I can’t talk about other headsets, however, I found this Youtuber named Blunty who tried them all at the Paxaus (or something like that) and recommended the Dell over the others. That influenced me on my decision.
I was able to get mine using a mail proxy service (One that receives the packages and then ships them to me), although depending where you live this can get more or less expensive.
My wife got it for $200, I got it for around $260+customs because the price started going up by the time I got paid. Good luck!
i am from poland and I can see u are from spain?
so we basically in same position. i have a friend that might send me such set.
but $200… it is so damn cheap 🙂You could buy it from microsoft and they allow you to refund it within 30 days (excluding shipping) if you just wanted it for testing.
the price of shipping is comparable honestly to price of whole set…
is any of those models recommended?They’re all basically the same, excluding the samsung osydsey which has a higher res oled screen and modified controllers. I have a dell visor, which imo is the most comfortable, the other models just have different head mounts.
Yeah, I am from Spain! And yes, customs can get bad. If you could get a friend who’s coming to bring it to you to save shipping, it’d be really convenient.
Regarding the headsets, yeah, people say the Dell is the most comfortable, but I’ve heard the Acer (and possibly the HP) have issues connecting. The samsung behaves a little different to the others: it includes its own headphones incorporated, and the IPD is set through hardware, not software. It is also more expensive, and I don’t think it has gone down in price like the others during the offer, and at that price you could get the Oculus, which although doesn’t have as much resolution and such a simple setup, doesn’t require win10 (I believe), and doesn’t have the tracking limitations of the WMR.
I would rather Greg to get one without refunding so he can dedicate more time to develop for it. I believe it is a great, cost-effective option and it could use some love.
I thought of a few ways you could improve the default WMR controller detection using the Kinect skeleton, but I’m not sure if they’d be possible for you to do. I’ll give them anyways in case you want to investigate them. Perhaps you could add them all and let the user choose the way they want it to act?
A few things to note regarding the controller tracking of the WMR:
– It has a limited range because of the headset tracking (Field of View)
– The controller rotations seem to be always accurate regardless of their position, they should always be used if possible.
– When a controller is out of range, it will completely freeze in place position-wise until it comes in range again. Rotation still works.
– I couldn’t find anywhere if you can check if the controller tracking was lost or not. I do notice the controllers start blinking when they are out of range, though. At least, in the Mixed Reality Portal, I can see the controller status goes from Ready to “Lost Tracking”.Now, for the ideas/suggestions on how to improve the tracking, (Note, they are separate ideas, unrelated to each other). Since the controller rotation is always accurate regardless of its position, it should only override positions, not rotations. Anyway, here they go:
– Use the controller positions until goes out of range relative to the headset rotation (perhaps a calibration is needed to calculate the area where the controller goes out of range?) then Kinect hand position takes over.
– When controler stays in the same position for too long, d4vr assumes it lost tracking and lets the Kinect skeleton takes over.
– Another one could be to always use kinect skeleton but still reads the controller rotation. Try to use controller if the Kinect skeleton is too obfuscated
– (Probably one of the most practical) If the Kinect skeleton hand is too far from the controller hand, and the Kinect skeleton isn’t obfuscated, assume it has lost positional tracking and let the Kinect skeleton take over said hand. This method could be good in case a controller is obfuscated by the other (It has happened to me, at least).
– (Assuming you can read if the controller is correctly tracked or not) Let Kinect simply take over a hand while a controller has lost tracking.Hope you find any of these ideas useful 🙂
yes. some of thesre can be done. will yo be happy to improve tracking only for front facing and potentially limited fov of kinect 360 ?
I’ve been wondering about that. Is Kinect restricted to front only? I remember reading you had plans on doing 360º (back facing) gameplay.
I don’t own a Kinect One (v2) though, and I don’t know how many here do. Does it differ too much from the 360?
yes. kinect is front facing only.
also I am not sure how well it works wiht kinect v2.
i am going to get my own WMR so I can look it closer.
even though with 2 x kinect you would still get limited area compared to how much you can go with WMRI have noticed the limited space with Kinect (or even two kinects), when using iPiSoft for motion capture, although iPiSoft supported backwards facing, just buggy. However, some sort of body tracking to compensate for the WMR limitations would still be nice, even if it’s front facing only.
I’m not sure if this is asking for too much, but how about using the WMR orientation to determine if you are facing backwards? Just tossing ideas.
Either way, I’m glad you are going to support WMR. I can’t wait to see what you can come up with. People like you are making the full VR market accessible to more people, and that’s a great thing.
Just wanted to post here to point out that each headset slightly differs from one another.
DISPLAY:
Acer WMR headset: 1440 x 1440 per eye (2880 x 1440 combined), LCD, 90Hz.
ASUS HC102: 2880 x 1440 single display, LCD, 90Hz.
Dell Visor: 1440 x 1440 per eye (2880 x 1440 combined), LCD, 90Hz.
HP WMR headset: 1440 x 1440 per eye (2880 x 1440 combined), LCD, 90Hz.
Lenovo Explorer: 1440 x 1440 per eye (2880 x 1440 combined), LCD, 90Hz.
Samsung Odyssey: 1440 x 1600 per eye (2880 x 1600 combined), AMOLED, 90Hz.Field of View
Acer WMR headset: 95 degrees.
ASUS HC102: 95 degrees.
Dell Visor: 110 degrees.
HP WMR headset: 95 degrees.
Lenovo Explorer: 105 degrees.
Samsung Odyssey: 110 degrees.The Samsung Odyssey (while also costing much more) is said to have better lenses (the sweet spot on the other inexpensive headsets have a smaller clear ‘sweet spot’ Odyessy offers better resolution, built in headphones, 110FOV etc)
The Dell (while it has good build quality and 110 FOV) is heavier than say the Lenovo, and because of the build design; its as if you have 105 FOV because your face can be further away from the lenses.
The Lenovo is light; but the build quality is… not too great (I can see dell lasting up to two years; and remain desirable (higher selling price) whereas the lenovo could fall apart in the first year. If I had a choice to choose again; Id probably go with the Lenovo for the lighter build (DELL is a bit heavy; and sometimes I lose the sweet-spot while playing games)
The Acer is a pass in my books cause FOV
The HP while it has a cool design is a pass cause FOV (unless the PRO version improved upon anything)
ASUS… they have a WMR headset? Not seeing one.
——–
@Greg – I hope you succeed in getting a good headset and work some magic for some external tracking! The benefits would be huge since tracking slowly falls apart when you put your arms out of the camera’s view (up above your head; or down by your sides; or stretched outwards for too long)TIP for any WMR users: I used to get a lost tracking error often in the first 10 hours of using my DELL because of poor lighting; once I solved the lighting issue- all games run much better. (A well lit room; and batteries under 3 hours of WMR usage equals good WMR experience.)
Here’s reddits poll results for wmr headsets owners https://www.poll-maker.com/results1878182x95AE47Cf-53
i think i will get dell. tried lenovo and did not like it for quality.
can anyone here confirm any success with wmr and kinect? and which version of kinect. i know many poeple use it already.
I’ve tried with Kinect 360, Dell as well. I managed to get it somewhat working once with VRChat, but not for long, and it was very finicky. I couldn’t get it calibrated correctly and it only worked once, every other time I was sinking in the ground no matter what I tried. Also, it seems that driver4vr makes me lose tracking of the controllers very often as well.
I tried both full body and upper body, I couldn’t get consistent and/or satisfying results so far.
can you do me favour ?
try to uninstall driver or simply don’t start it.
and run kinect via other program from kinect sdk. for example body/skeleton trackking sample. i want to confirm if there is any interation between those.Do you recommend any app in specific? I’m not familiar with apps that use Kinect other than iPi recorder, so I don’t know which ones would gather the information you need. Thing is, your driver, when hitting “configure”, tracks the skeleton perfectly (The pink background with the green skeleton), even when facing to the sides. I only seem to get problems when I try to get SteamVR to track them. (Maybe I could just be doing something wrong, though).
While waiting on your reply, I will try installing the beta branch of SteamVR and trying to clear up more space in the room, just in case.
(I can’t seem to edit the post anymore so I have to make a new post)
Okay, I did a few things and it seems to have improved things, some:
· I decided to empty the room entirely, just to make sure the kinect wouldn’t be confused with any objects in the surroundings
· I have relocated the kinect to a higher altitude and used the elevation range, so it would have a better coverage of the floor area
· I installed SteamVR, beta branch
· I updated Driver4VR to the latest versionThings I’ve observed that are different to before:
· The calibration error of the HMD vs the Kinect head seems much lower now
· I get proper hip and leg trackers now, it seems, although the hip and legs seem a little off at times. I’ve used the elevation modifier, it helps, but it’s still not as good as your full body tracking videos
· At random times the Mixed Reality Portal goes nuts and won’t detect the headset anymore for some reason. Although it was a lot of trouble in the past, for some reason this time I was able to get the headset detected again after closing everything and reopening.
· I’ve tried VRChat, it seemed more or less good, however my legs don’t seem to stretch out enough, and avatars have legs bent. Doing air kicks don’t seem to stretch the leg entirely, as if the skeleton doesn’t scale accordingly.thanks for detailed info. it is difficult to determine actual impact of kinect on wmr. there is no need to be on beta.
best is kinect sdk. and there you can run Skeleton Basics app. it does not matter if that is tracking anything as long as it is pointing are you work on. this is to determine if there is any impact of IR light from Kinect.
i got my Dell just purchased and will be shipped in next 1-2 days 🙂I’ve been doing many tests and doesn’t seem like the Kinect and WMR are interfering. However I’ve noticed the Kinect isn’t calibrating quite well. I’ve tried raising my foot to hand level and placed it right next to the controller, and there was quite a considerable distance between them (almost a thigh of distance).
Also, (this is a suggestion, not sure if it is possible to do for you) I’ve noticed the hips and feet rotate with the headset. Do you think you could eventually add an option to use the angle of the thigh+leg bones to calculate the rotation of the feet and the position of the beginning of the thighs to calculate the pelvis? I wish I could code to write for you the solution I have in mind… but in a nutshell, if it helps, my idea is grabbing the beginning of the thigh joints, draw a vector between them, and set a perpendicular vector that serves as rotation for the hip bones.
I have been using the Dell + Kinect v1 for awhile and from what I have seen I mostly just glitch out or do random “T-Rexing” despite being in the kinect and my controllers being in view of my headset. Also they added movement with the sticks that does locomotion and rotation which messes up the calibrations even more.
I too have been running a Dell Visor with the Kinect V2, and it seems to work sometimes. While the tracking is there, in VRchat my feet sometimes glitch out and are either both attached to one foot, or the tracking becomes minimal, as in large motions with my feet are barely registered. I will say though that the Kinect tracking doesn’t interfere with the WMR tracking at all, at least in my case. Really looking forward to V4 since I have limited space to work with, about 1.6 x 2 meters.
I’ve had the feet being attached to one foot problem too. I’ve found that sometimes setting the character to run partially fixes it, however I still have the issue of the feet of the avatar not reaching far enough to make a full stretch when doing a kick, or not reaching the hand when I put the foot and the controller right next to each other.
I still need to make more tests, but seems like my WMR is having connection issues with my ASMedia usb based ports (the ones I have for my motherboard, basically). I’m waiting on a USB expansion PCIe card to arrive to see if that fixes the issues I have with it.
Being an owner of a Nolo VR, Pimax 4k, and now the HP VR1000 WMR, just gonna put this out there that I genuinely think the money maker is now going to be improving the WMR system with the Kinect… They are just so cheap for what you get experience wise! Like wow, i got this for under $200?! Wtf? (In a good way)
yes. it is like you say. it happened after MS stop producing kinect.
I am gonna get for myself non genuine version for testing as they have reasonable price about 60usd on aliexpress.com .Greg
i have not seen problems with body tracking on my dell.
so my plans are now:
1. lower body tracking
2. using 2 kinects for body tracking
2.1 this will potentially allow add 360 tracking for nolo
2.2 also add supplemental tracking for wmr.so it might be in 2 months approx. however don’t take this as any commitment 🙂
what games cause most problems for you?
ps. i cannot get used to very small area of sharp image. it seems like sharp image is only when I look forward directly. and if I move up/down by 1 milimeter it is blurry again…
++ 2.2 Supplement WMR
+ 1 Lower body tracking for WMROnward game within SteamVR has a hard time tracking when grabbing items on the body below the head or outside WMR’s general controllers tracker cone.
I have a Dell WindowsMR Headset, along with both a Kinect v1 & v2. I tried using the Driver4VR a couple times in VRChat but had issues with trackers being on the floor. I’ve been waiting for the lower half update but after finding this forum I’m excited for any improvements on the hand tracking.
I was curious if a person could use a LED ping pong ball trackers with the Kinects without using a PS Move camera or controller (though that airsoft tracker you did is boss, I have a few airsoft guns myself)Anyway if you want someone to help test, or even BS with in VRChat, feel free to hit me up!
thanks. again, i am a bit skeptic for this particular piece of work cause you will add only add to missing part but decent piece will be still left. however I will plan my future work to easy fulfil this. but no promise 🙂
Hi all,
Apologies if another thread already exists for this.
Has anyone tried or heard of using external sensors to track the motion controllers, in the same way oculus does?
- You must be logged in to reply to this topic.