Gaussian Splatting! The next big thing in 3D! 

3D Gaussian Splatting is actually not a NeRF technology as there is no neural network, but the splats are directly optimized through rasterization rather than the ray tracing like method of NeRFs.

Imagine Google Street view built with this. It could then be used in a GTA type game with the entire world.

If scanning software is actually capturing and rendering details as fine as leaves of plants, without all of the artifacts, then that is absolutely mind-blowing.

I spent lots and lots of hours in my life unwraping UVs and correcting meshes to use in my archviz projects. The amount of development in this field is insane. And we are in the first days of this. What a time to be alive.

Ok I did a quick reading of the paper. This is a clever thing however what should be kept in mind is that it doesn't preserve details so much as makes them up. (The explaning bit is in the chapters 5.1, 5.2, and fig. 2, 3, 4 of the paper). Basically you reconstruct the environment not by analysis but by statistically informed guesses. After which you then do analysis on whether the guess was too big or small. Then you refrence the solution to the original data to see how close you were with your guesses. If the guess was too small you duplicate it near the point; if the guess was too great you divide it in to two. Meaning that if you need to estimate a curve, instead of trying actually solve the curve you keep guessing the shape of the curve, but because of the process of duplication and division of the guesses you basically approach faster to the solution. However it is important to keep in mind that you don't actually get THE SOLUTION, you get approximation of the solution based of guesses. Basically this is the way you can do square roots and cube roots in your head to 2-3 decimals, by estimating upper and lower and iterating that (for those that don't know: if you want to estimate square root of 6, you can in your had calculate that 2x2 is 4 3x3 is 9, so the solution is between those; then you can do 2,5x2,5 you get 6,25 which is more, so you know that solution needs to be less than that so 2,25x2,25 you get 5,0625... so on and so forth. You will never practically get the solution of 2,2449489743 because we only go to 3 decimals but lets be honest 0,04% error is more than enough. To simplify a bit: Imagine you are sculpting with clay and want to replicate a shape. You can only add or remove instead of shaping it with your hands. If you have too much material you cut away hald of the amount of you know to be too much. If you added too little clay, you add another same sized lump. And you keep repeating this untily you get close enough approximation of the thing you are replicating. What is important to keep in mind is the limitations of this. You can't replicate things accurately for the simple reason that if you lack information on details you can't just guess them! Your data resolution doesn't increase. You only actually know the datapoints that you gathered. So for historical, scientific or engineering purposes you have will not be able to get any extra information (And I hope that people realise this, before they try to use details from this in a court of law or something), you really can't know anything more from this than you can get from just looking at the frames as pictures.

I fell in love with splat engines when I spent time in Dreams on the PSVR. It's fantastic for creatives, it makes it very easy to make new and unseen surfaces.

I feel like those galaxy-like ellipses with feathered edges are THE new polygons and soon this rendering method will replace them, especially in the gaming industry. What a time to be alive

All the time, I thought I saw videos and was wondering about 3D. 🤣 Then it hit me: These ARE 3D renders. Absolutely stunning.

Reminds me of that moment on Star Trek into Darkness when they are looking at security footage and are able to move around the footage just like this

Great video! The RTX 3070 has 8GB of Vram though, not 4. I'm super excited to see where NeRF will take us in another 5 years! It's a boon for indie developers who don't have the time or budget to create high quality assets.

Those gaussian splats looks like galaxies in space at 2:07 .. ;O

2.7k subs is a ridiculously low amount for such quality videos! Great work, brother

Great, best videos about this niche of nerf's etc. i found so far. Keep it up!

guassian splatting has been around for many years. I used it in the late 90's. It's a method of generating implicit functions, which can then be contoured.

bringing old family photos will be a huge market boom

Thanks for the shout out! You can now view the scene in the NerfStudio viewer which unlocked smooth animation renders.

Man, thank you for this video! That stuff is really next gen! wow! And top notch moderation! Very relaxing and informative!

thanks for linking to the nerf guru. could come in handy some day if i decide to try this!

Astonishing achievements were made, that is great! I hope this may lead to set builders to make more models which can be traced and recreated in 3d space, keeping these sculpting jobs relevant. Thanks!

fantastic and very professional youtube channel. appreciate your work

I remember when i first tried Nerf. Since then, they have evolved into insane quality!

Great summary, insight and practical example - thanks!

This is gonna be huge in real estate, once it’s a streamlined app on both ends.

It's getting better and better, that's an impressive method, thanks for sharing ;)

Hot damn, it should be possible to extract depth , normals, and glossines from such capture, this is insane!

It's beautfiul that the first image I ever saw rendered in this way now is a Captain Haddock figurine ❤

so exciting! thanks for the explanation:)

Terve Olli! Another Finn with an awesome RU-vid channel (I'm not including myself 😁)! Thanks for keeping us up-to-date on what is going on with this cutting edge stuff.

ohhh thanks for explaining, I saw a couple things on twitter and was wondering what this gaussian splatting was about!

Great overview and certainly exciting times! 😊

Glad I found your channel ;) .. nice to meet you Olli

This is fascinating! I hope there's going to be some kind of support for blender/ unreal/ unity soon I would love to play with this

Yesss i cant wait for this to be utilized in vr, I assume we could render absolutely insane detail in realtime for simulating reality or having big budget cgi movie visuals in games

I'm convinced that this is how our vision works irl

Very cool. Thank you

these are so good that you can probably use screenshots of these models to make 3d models with old photogrametry software.

This is literally what will follow photographs and images in general where you can see captured moments of your family and friends in full 3D. Now we need a device that would capture these automatically with a click of a button.

All we need now is a scanner in every phone for taking those selfie pointclouds and we'll be in the world of tomorrow.

Wow, now we’ll see really interesting development in video game production and of course in the results.

Incredible. Love and blessings!

Great video man, thanks for sharing

Oh I love your content! Should have been subscribed last time I met your channel. I didn't, but I do it now! )

Absolutley phenomenal!

If you watch two minute papers, there’s a new radiance, field, technique that is over 10 times as fast and better quality so look forward to seeing this in VR/AR

Very interesting, thanks.

This is so freaking cool, I hope one day I can make a game using this tech.

Stunning!

Just imagine a machine that can generate point clouds around itself at a rate of 60 per second, and a technique like this that can render that point cloud at the same 60 per second rate. Truly 3D video. Would be amazing.

I'm deeply uncomfortable with how fast everything is moving right now. Feels like anything you touch could become obsolete in months.

looks really impressive

amazing thanks man for this video

Good stuff mate. Very interesting indeed and great to see such in depth look into things with self made examples. As a sidenote, music is a bit big for this, I mean it´s not cure for cancer (just yet) so perhaps go bit easier on "Life will win again, there will be beautiful tomorrow" soundtrack :p . Anyhow, cheers, will be back for more.

My mom walked in the room and asked what the hell I was doing. I told her to just relax. I'm gaussian splatting.

This reminds me of the the old "Unlimited Detail" video promising infinite rendering

Impresiona como avanzan estas tecnicas. En un futuro poder aplicar esta tecnología para la resolución de crímenes u accidentes, con solo tener un video del mismo va a ser genial.

Fascinating! could you make a video on what would be the most relevant use cases for this type of technology?

good video got a new sub!

Nice work! Well done :)

One more time for gauss to show the world who is the king of Mathematics.

Interesting and seriously promising for VR applications!

You could walk in your memories with VR!

Technology from Minority Report movie, showed 20 years ago, that's how long it takes to make.

Next step is getting it working with animations and physics and you have a new game rendering method. I have always felt mesh rendering is limited, been waiting for a new method such as this. Hope it's the one this time since there have been quite a few duds in the past

liked and subscribed ... hi from New Zealand

This is basically how the actual visual field works. Overlays of sensations, sounds, and smells complete the illusion of subject/object. It is the zero dimension quantum wave field. The scene ‘moves’ in relation to the ‘eyes’ of an apparent subject.

absolutely amazing technology! super excited to see where the future takes us. thanks for sharing! ..side question - what is the music track on this video?

Awesome!!! Though, for me, all that matters is getting a correct mesh and I couldn't care less about textures personally. I hope the mesh generation will soon also make leaps like this :)

Good watch. Subscribe earned. Haddock is one of my faves.

It's a cute way to make use of point clouds. I'm certain it'll be handy for MRI's and CT scans, but it's nowhere near as useful as an actual 3D model. You couldn't use it for video game models or 3D printing. It could be extremely useful for real-time, point-cloud, video conferencing, since it's so fast.

great to see

This really makes me feel like living in the future.

Idea! 1. Make a low-poly 3D scene in Blender. It's a 3D skeleton. Use colors as object IDs. 2. Using real-time fast OpenGL engine, quick-render some hundred images, placing the camera to different locations like photographing a real scene for the 3DGS creation. The distribution of the camera should be easy using Geometry Nodes. 3. Using these images, use Runway-ML or ControlNet etc. to re-skin them according to a prompt. If possible, use one image to ensure consistency. 4. Give the re-skinned images to the 3DGS creation process to create a 3DGS image for the scene. Et voilà, a 3D AI-generated virtual reality is converted to 3DGS.

Very cool.

This is awesome, is there a way to export this to an estimated actual polygonal model to use in 3d softwares?

Hi Olli, great video and thanks for the intro to such fascinating tech. What’s your opinion on whether Insta360 or full frame camera a fisheye lens will provide a better result or workflow?

very nice video

I really hope this gets picked up and adopted quickly by companies that are training 3-D generation on nerfs. The biggest issue I’m seeing is resolution. I imagine this is what they were talking about coming in the next update with imagine 3D. Fingers crossed that would be insane.

This point cloud technology is much older than 2020. It has been tried for a decade at this point, on and off.

I remember a Australian company making infinite graphics with something similar. They made games and other stuff. Cant remember the name but it was many years ago. :(

Woow, que bien, un punto más para seguir andando en la IA

Once video games and 3D software rendering engines start to use this… Oh boy, that’s gonna be something else

How big are the files used to render the scene? Could this be used as a light field video format?

I tell you this: is a game changer for the industry...

Well one thing it'll be useful for is helping me examine images for clues to hunt down replicants.

Perforated disck layering at one point with bezier curve translations and HDR data mining ...

solution to the floating artifacts would be just make procedural volumetric fog and use it to your advantage 😎

Gauss strikes again!

exciting news!

wow this is awesome ! is it possible to get 3d objects out of this? and if so, how good are the results?

Perhaps it can't be used this way, but I'm really hoping for similar technology to dramatically improve the speed accuracy of camera tracks. Would be amazing to just need a video recorded on the phone and get back the 3D camera track data and a point cloud of the environment.

The entire VFX Industry is under massive disruptive growth that now prioritizes INDIVIDUAS..... ....a huge paradigm shift.

Next gen of graphic cards apparently will all have a neural network chip on there.

How long before micro drones are just buzzing up and down our bike paths, sidewalks, streets and so on, grabbing HQ images, beaming them to the cloud, and by the end of the day you can do a virtual walkthrough of the local fair, or the car dealership, or a garage sale on the other side of town, or the crowd at a football game. Only thing stopping us is CPU power and storage, and that is getting solved fast. Exciting times! P.S.- How long before people stay home, and just send out their micro drones, and view everything in VR at home. A lot safer than getting mugged.

I wonder if the high demand of VRAM could be circumvented by using some other memory to compensate, like normal RAM or a part of your SSD?

Thanks for showing this! I am also studying various technologies with success. We should collab sometime!

Nerfies! What a time to be alive!

Unlimited Detail did this before it was cool. It also did not require high-end GPUs nor that much power; it worked fast with software rendering!

fantastic , i need your suggestion?

amazing. How is this rendered? In what program?

Closer and closer to Esper from Blade Runner.