cancel
Showing results for 
Search instead for 
Did you mean: 

Simple positional tracking with Kinect v2

daloots
Honored Guest
Hi there,

I put together simple positional tracking using a Kinect v2*.
The new Kinect uses a time of flight method for the depth sensing. For this it emits a pretty strong IR light that produces a nice IR image. I stuck a small piece of reflective fabric to the front cover of the Rift and used the depth map to retrieve the location of the reflective marker.

The results are probably nowhere near those of the Crystal Cove prototype, but it's good enough to experiment with and couldn't be simpler. The new Kinect has a very wide field of view and a (preliminary) range of over 4 meter. Of course with the single marker setup, it loses tracking if you rotate your head more than 90° away from the Kinect.

Using the Kinect body tracking seems obvious, but I think this method has less latency, higher accuracy and the Kinect only needs to have line-of-sight to the marker.

http://www.youtube.com/watch?v=N46IWihUE1E



Cheers,
Davy

*Kinect for Windows v2 Dev Preview: This is based on preliminary software
and/or hardware, subject to change.
True 3D video for VR: http://q3d.quaternionsoftware.com
14 REPLIES 14

dopingVR
Expert Protege
This looks pretty sweet! How accurate does it feel when you're in the Rift?
The demos I tried that used the Hydra for positional head tracking were not too accurate and didn't feel that good to me.

daloots
Honored Guest
Hi VRdoping!
It feels pretty good. Because I use a single pixel from the IR image for tracking, there is some slight jitter, so I use a lerp (smoothing) function of 100ms, which introduces some latency, but still pretty good. I have no way of accurately measuring the accuracy, but I think it's in the millimeter range.
Since the Kinect tracks a point a few centimeter in front of the eyes, I should do some math to take the orientation of the headset into account for deducing the position of the actual eyes. But I'm currently too lazy to do that 😄
Cheers,
Davy
PS. thanks again for your help earlier 😉
True 3D video for VR: http://q3d.quaternionsoftware.com

kojack
MVP
MVP
I wonder how much the Kinect 2 for PC will be when it's finally released to consumers?
The dev version is $399 (and gives you 2 kinects, the dev one now and a consumer release one when it comes out).

There's another 500 slots open for devs to buy one apparently.
Author: Oculus Monitor,  Auto Oculus Touch,  Forum Dark Mode, Phantom Touch Remover,  X-Plane Fixer
Hardware: Threadripper 1950x, MSI Gaming Trio 2080TI, Asrock X399 Taich
Headsets: Wrap 1200VR, DK1, DK2, CV1, Rift-S, GearVR, Go, Quest, Quest 2, Reverb G2

daloots
Honored Guest
As far as I know, they haven't released any information on pricing yet.
I think the release is planned around the summer.

You're right, there's another 500 spots available for developers. Register here:
http://www.microsoft.com/en-us/kinectforwindowsdev/newdevkit.aspx
True 3D video for VR: http://q3d.quaternionsoftware.com

mrjazz
Honored Guest
"Daloots" wrote:
Because I use a single pixel from the IR image for tracking, there is some slight jitter, so I use a lerp (smoothing) function of 100ms, which introduces some latency, but still pretty good.

After you have found the brightest pixel, I would compute a weighted average of a small area (for example 8x8 pixels) around this pixel. Perhaps this simple solution increases accuracy without having to do lerps.
A more sophisticated method would be mean-shift tracking.
Watch panoramic 3D photos, videos and do FPV-flying with LiveViewRift! viewtopic.php?f=28&t=11001

daloots
Honored Guest
"mrjazz" wrote:
After you have found the brightest pixel, I would compute a weighted average of a small area (for example 8x8 pixels) around this pixel. Perhaps this simple solution increases accuracy without having to do lerps.
A more sophisticated method would be mean-shift tracking.

Unfortunately it's a bit more complicated: the reflective piece itself doesn't yield a depth value (it's a blind spot for the depth sensor), so I first have to find pixels around it that do. But your point remains, I should use more pixels than 1 and I would probably no longer need the lerp.
True 3D video for VR: http://q3d.quaternionsoftware.com

chaosgrid
Explorer
hey,

seems like the Kinect v2 has very stable and precise tracking..
Did you come around testing the body tracking while using the Rift?

kevinw729
Honored Visionary
Good job @Daloots - I was just wondering, with the sale of the developers of the Kinect to Apple, will this still be an open platform?
https://vrawards.aixr.org/ "The Out-of-Home Immersive Entertainment Frontier: Expanding Interactive Boundaries in Leisure Facilities" https://www.amazon.co.uk/Out-Home-Immersive-Entertainment-Frontier/dp/1472426959

chaosgrid
Explorer
Kinect v2 is developed by Microsoft in-house. It is a very open platform (MS Kinect SDK).

PrimeSense, the developer of Kinect v1, is/was an Isreal-based company that has been bought by Apple.