I stumbled across this video and I'm blown away. In a former life I use to do a lot of graphics and old-school 3D work. Could I request a beginner's guide on "how to get started" with the tech you covered?
This can be used to google maps street view. Imagine instead of clicking arrows to move forward, you just need to press “w” maybe as if you are walking in a video game
@@hallo_ween07 Starting with a town maybe? Then slowly moving towards a country? Maybe a different type of google car can be made to use photogrammetry instead
Brother your channel is going places. Your content is presented in a very straight forward manner and will get you far. Keep at it. See you at the 1 million Sub mark.
Bravo Bilawal! Well done. Straight to the point why GS is such a promising approach. I especially enjoyed the comparison to the "wonky looking broccoli trees" ;-)
this is genuinely the best explanation i've heard on gaussian splatting. especially the part i don't think i've heard anyone talk about that is HUGE, which is *spherical harmonics*. actually showing reflections and the sun through the leaves that you're showing here. massive.
Very comprehensive explanation, I actually do research on these reconstruction and view synthesis topics and your intro explanation made it very clear where/how to start thinking about gaussian splats.
Thank you, that was clearly explained and very exciting. A very minor quibble is that Gaussian splatting is named for one of the greatest mathematicians of all times. His name is pronounce gawse and the curve he discovered is called the gawsian and is central to the study of probability and statistics. And now, computer graphics!
i am confused, watching the video right now, and the pronouciation sounds very typical american? i think its good, whats to complain? the real pronouciation of course is difficult for non german speakers, listen here en.wikipedia.org/wiki/File:De-carlfriedrichgauss.ogg
@@sirleto I don't want to make this a big deal. Mathematicians and physicists all pronounce it this way and thought you might care :) @daverayment explains it better. Gauss rhymes with house.
Excellent work! 3dgs can be used for any presentations, including those for ordinary consumers, real estate agents for project visualization, doctors, VFX/CGI, artist....
now i just wish to see some kind of style filters. since it uses point clouds, we definitely have depth as a parameter, giving us stuff like edge highlighting or idk, zoom effects. Could probably train a stylized cartoon filter. Since the splats are mostly gradients, maybe flatten the colors to specific styles? Would be fun to mess with if my PC would be able to process all the stuff :/
I am a VFX artist in Los Angeles who does digital sets, Matte Paintings, and Environments. I am trying to understand Gaussian Splatting from a non white paper stand point. It seems like it takes the capture camera Meta data ...turns it into a point cloud......and when you view it through your camera...it takes the photos or batch of photos that are near by and basically shows you photoprojections from the photo capture? Its basically viewing a flip book of pictures as your moving your camera based on the capture data positions? Forgive my laymen understanding. As a matte painter we do the same thing manually........doing a series of projections based on camera. How far off am I?
This is awesome. But what if I wanna take the splats to Blender, to insert a 3D statue in the middle of scene? it's possible, or we need some kind of file conversion ?
Imagine having a 3D rendering engine that can automatically harness the power of Gaussian splatting for 3D animation, rendering only the needed Ray traced frames to create the splatting and turbo charge the rest of the frames making it possible to render long animations on a single consumer machine in record time.
*@Creative Tech Digest* 0:11 No, I have no idea what any of those words means, and several other words, THAT was why I went to this video in the first place, to learn more, but I just get to hear a lot of meaningless words. :(
This is good feedback. I have added a “3D Capture 101” video to my queue! This video def assumes you know the basics of photogrammetry and 3d scanning tech
I think 3d video's will also be a great thing when they are ready? I just have seen a demo of a video that probably had a depth map, so it was a 2d video but made 3d. But when you went behind an object to the side where there was no camera, then there was no object. I think AI could automatically predict what is behind an object and in that way create 3d maps from just normal videos. Then you could watch videos and walk around in those video's. I think this will be amazing.
I wonder if you can transform a 3D scene with textures into a gaussian splatting format to make games more optimal without sacrificing too much visual quality
class Gaussian: def __init__(self, center, scale, covariance, color): self.center = center self.scale = scale self.covariance = covariance self.color = color #Is this enough? Or is there more to it to attain spherical harmonics?
What is the best gausian splatting software for using drone footage. I have the original DJI MINI so I can't plot routes. I'm assuming video to luma ai would be easiest
For the initial sparse point cloud / posing of images - colmap is fine but reality capture, Agisoft Metashape, etc would be better. Google also released CamP code this week which is also bette than colmap.
I have a question that almost certainly reveals my total ignorance of this tech! When rendering 3d GS environments in a game engine, is it concievable that game mechanics could be added to this, for instance, pathfinding, collision, decals, all that traditional 3D stuff.
it's still early days. you could certainly query metadata you attach to each splat. you could also mix and match - make a mesh version using photogrammetry but keep the visibility off but keep it on for physics collisions, path finding etc.
So...how much will it cost me to get into this game? All I've got right now is a reasonably capable PC, a few VR headsets I would like to view this stuff in, and a bit of free time.
There’s a ton of interesting research that uses radiance fields as the 3D representation but diffusion models for the generation. Quality isn’t quite where MJ is for still imagery tho!
Could you render a 3D scene into Gaussian Splats for more rapid playback in game engines? I imagine if you could you could purposefully distribute the points intelligently, creating a better looking final result.
Maybe. I don’t believe the density of those RGB scans is good enough unless you really put the tripod in a bunch of locations. Though I guess matterport could try to use their depth data in the training process to make up for that sparsity?
Gaussian splatting is good for view synthesis. For real world measurements - you’re better off using classical photogrammetry to densify the point cloud or turn it into mesh and measure that. Of course you need to make sure your scan is metric accurate and to scale. iPhone apps that use the AR metadata eg Polycam do a good job of automating that for you.
Its math. It was always math. Except chemistry. Idk how that works. Because if all compounds wanna turn into noble gas, then why is LiN not on pubchem? And why doesnt nitrogen bond with 5 molecules???
it's a radiance field so that's a feature not a bug - instead of rasterizing triangles as you would with a triangulated 3D mesh for example - you instead rasterize these gaussian splats. you can still use a mesh under the hood if u want for collisions etc.
Thank you! I think the photogrammetry tools had largely written off Nerf as a fun research toy. Gaussian splatting is making them pay attention and I could see it quickly become a common artifact alongside other reality capture data products in AEC. Amazing for visualization and progress snapshots!
Would be Interesting to see applications dealing with heritage preservation demands considering we’re typically using Point Clouds. If we could see GS as a resource for model driven Digital Twins and other digital processes for maintenance operations, I think we can cut away hundreds of hours of modelling work.
Thanks. Can you make a video of how this can be used for product marketing. Like creating a 3d structure of a product for websites. Can you also confirm if this can be done using some mobile apps
