Custom Depth Map from Kinect & RealSense Point Clouds in TouchDesigner - Tutorial 

For accurate alignment, you could try using either the Color Point Cloud option of the Kinect TOP's Image parameter, or you could instead set the Image parameter to Color and the Camera Remap switch to On. Elburz covers this functionality in the Kinect Azure Point Cloud video (ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-P_PjAr2Yzao.html). Although the video is focused on the Azure, you can get similar results with the Kinect v2. Hope that helps!

The Player Index texture is a little different, but somewhat simpler to deal with since it's more like a mask rather than a 3D point cloud -- try using a Threshold TOP and adjusting the Threshold parameter until you are able to isolate the person's outline. Hope that helps!

Yes this can be useful because you can re-orient how your tracking space is defined regardless of how your camera is setup. So you could have a camera facing forward and then turn it into something that feels like top-down tracking. Does that answer your question?

Elburz is using a Kinect Azure in the video, but the Kinect 2 also has point cloud support. You can access a _Color Point Cloud_ and _Depth Point Cloud_ via the Image parameter of the Kinect TOP. Hope that helps!

Hello, did yo do it?

hi man, well I got a kinect azure for a bit that one actually works on laptop with the usb. But even using a strong laptop there is just not enough processing power to really do that, best way would probably be using lidar on a drone. I had much more success with photogrametry to scan stuff then animated a portion of the points where the kinect detect your hand then project on the wall now made of pointcloud, actually I think kinect is good for the bin, it cost a fortune and might be good for small live pointcloud, now you can track skeleton via video alone. I actually think touch designer itself has serious limits unless you handle point cloud with glsl. I sort of gave up, too much hassle and even if you do it right it kind of slow to work with. Perhaps when computing power is multiplied by 3x or 4 it will be worth it.(I waisted enough time in front of computers for a lifetime) Good luck to you still @@TheTretistaKelverian

You might be able to achieve this with the Remap TOP, which would allow you to apply/map a specific texture to the points in the scene. Might also be worth trying to normalize the point position data with the Limit TOP, but that might cause some color shifting artifacts of its own.

That's great, we're glad to hear it! Yes, the Azure Kinect SDK allows you to connect as many Kinects to your computer as you have bandwidth for. The Kinect V2 was the version that only allowed one Kinect to be connected to the computer at a time.

Yes i've used this technique for live human body projection mapping, works well, you just need to calibrate correctly between your point cloud and your rendering camera (VP intrinsics/extrinsics) and it's also a bit computer demanding (I was using nvidia flow to set the performer's body on fire at the same time). Actually for correct projection mapping, you need the actual xyz values, so don't use the custom depth map, but keep the glsl filtered point cloud.

No problem! It's tricky to wrap your head around the first time but then becomes such an incredible workflow to keep in the back pocket. Haha I've been corrected so many times it might be one of those things I say wrong forever *face palm*