6DoF Video Point Clouds - Combining Multiple Cameras 

I could see this being a thing in the future, but unfortunately without much more accurate depth maps, it's not really going to practical for most projects. I can think of a few specific applications though for it, as-is though

These videos were from Google Jump. I'm not sure if there's a big difference between Kandao and Google Jump. I've seen some very nice images from Kandao, and also some not great stuff. Google just improved their depth maps, though, and that seems better. I guess I'd give the edge to Jump right now, but neither are perfect. I definitely wouldn't put Stereo2Depth in the same class as either of them. Stereo2Depth uses a much less sophisticated algorithm.

Hi, sorry. Just saw this. No, while the player does display a point cloud, it does not export anything. There are other solutions out there for converting depth maps to point clouds. Just off the top of my head, I believe OpenCV does it.

Jump is already capable of 8k video, and the videos I'm using here are well below that. Higher resolutions just sort of make things crisper and clearer, but they don't really help with accuracy. More accurate depth maps along with multiple cameras could achieve cleaner results, so it's possible lidar could help. Although, I'm not familiar with any lidar solutions that can capture the full 360º at video speeds. And beyond that, even if you had absolutely perfect depth maps, you're still going to have issues with transparency and reflections and things that keep it from being totally photorealistic. The real question for me is how far does it need to advance before it's a viable option for storytelling with enough immersion and realism that the viewer is able to get into the story? So I'm trying to put a small short film together to see if we're there yet.

Oculus has some solutions for those reflections. Check it out. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-RJxoDggqLoo.html

Lightfields also solves the issue to an extent, but that's a whole different thing.

Specular, and other secondary qualities of light are naturally captured by light fields but a light field camera that can capture a scene from all angles is obviously far from practical. Perhaps, since we have the depth data, these other secondary light qualities will be simulated and then added on top of the real 360 video as a 3D generated blended layer. Seems to me that these secondary light qualities are pretty subtle in most cases though. The focus should be on high quality depth maps, no?

Yes only to essentially total diameter of all of the 360 camera's lenses. If LIDAR isn't fast enough and, on the other hand, a single 360 cameras isn't wide enough to give very detailed camera depth data then it's perhaps necessary to have the depth data from at least TWO cameras spaced much wider apart for EACH new/unseen direction the viewer will be looking in rather than only looking for visual RBG coverage from multiple cameras.

These videos were shot with a GoPro Odyssey, which is a Google Jump camera. Google Jump provides 360 video stitching, as well as 360 depth maps. Other 360 camera solutions such as Kandao Obsidian, and Nokia Ozo also provide depth maps out-of-the-box. I have also released an open source python script that can be used to generate depth maps from stereo 360 footage. For more info, check out this video: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-BHGvcEBRA98.html, and head over to reddit.com/r/6DoF.

Not the combining multiple cameras part necessarily, but you can absolutely export a depth map from Blender, and then combine it with the color picture. Should end up looking something like this: www.dropbox.com/s/ff5026xg0bylbwq/Morgan_lives_in_a_rocket_house_6DoF.mp4?dl=0

@@JoshGladstone very cool.. what software do you use to make the 6dof model?

The whole project or the app? I did release a 6DoF viewer app on all VR platforms as well as desktop mac and pc, links can be found in the sidebar at reddit.com/r/6DoF.

@@JoshGladstone the unity project, would love to create something based on it.

@@sumo242424 Sorry, no I don't have any plans to release the Unity project.

Everything is in Unity, and the point clouds are from a custom shader and script that I wrote. There's a bit more about it here: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-xB-OnSYiPfo.html