Temporal is a Chicago-based AI company focused on the multidisciplinary intersection of LLM research, Machine Learning in 3D Graphics, and Conversational Interfaces.
Have you guys tried this on high quality data? It would be interesting to see how the details hold up zoomed in. The 4D gaussians sample data is quite low quality.
While impressive this really feels disconnected from the environment and jarring. It reminds me of the early days when games like Myst and Riven included video clips of actors inside their games. A cg character in a cg environment will always feel more grounded for now until we are much further along with this tech.
It's very likely the 'real-time' here implies the gaussian splatting element in the scene (the two people and the ball) can be rendered at a framerate high enough to be considered real-time. This is how other related papers refer to real-time in 3DGS demos. The people and ball were likely pre-recorded using multiple cameras to produce the final 'play-able' temporal gaussian splat. This is really cool btw and hope to see more 3dgs stuff!
There’s a demo you can try out.. the data sets are already done. I’m not sure how many cameras we would need .. you can’t use one camera.. and then for example shoot that same thing from different angles.. it would not match up to camera position from take 1 for example. So I’m hoping 4 cameras would be enough .. shot all at once .. but probably not enough. The depth camera on an iPhone might be a way too. That would be interesting.
How are you getting lighting to work with gaussian splats? From my understanding they cant cast shadows? Can you please share some insight, I would like to research this more.
This is amazing! How do you Imported this in UE? With plugin from MarketP? . How long takes character capture for "10 sec" points sequence. And how big is this in MB or GB. Thanks for your answer.
Yes, it can run on mobile GPUs. The focus of our research is on compression, which is the biggest challenge with Animated GS. We've currently achieved a 60x compression rate over raw Gaussian Splat frame representation, and are working on making Animated GS content streamable.
@@temporalgames great job! really looking forward to it, we are actively trying to produce 3d gaussian content, if you have any beta version that we can try, please let us know. thanks!
Our midterm goals include bringing Animated Gaussian Splatting to Unreal Engine and the web, which should make it possible to view content in Mixed Reality mode.
@@Aero3Dour goal is to make livestreaming possible, but that requires substantially shortening the processing of raw footage and GS frame reconstruction. Existing third-party solutions might take up to 40 minutes per single frame to process. Our method only processes the changes between the frames, thus making rapid GS frame reconstruction possible. However, this would require further work to cover edge cases and various scenarios (e.g., when the scene composition changes drastically).
@temporalgames can I stay in touch or follow progress somehow? I am creating a VR live events platform. As far as talent, we have multiple ways to capture, 2D/greenscreen for budget, but we also want to provide 3D solutions, such as a Kinect Azure. I tried 8 Kinects with volcap, and with Unreal, it's just not supported and all the livestreaming projects gave up or failed. I am hoping Gaussian Splats and a team like yourself may still make it happen. Thanks!
We are actively working on improving the quality. The 'non-realistic' aspect is due to the low resolution of the source material (360p video). Soon, we will demonstrate reconstruction based on higher-resolution video, which should alleviate the synthetic appearance.
