4K, 6 Months of Voxel Physics Engine Optimizations 

Thanks! It was more my fault. Because of the long timespan my clips covered, I had to redownload old clips from YT. So that caused them to be triple/quadruple compressed. It took some time but I got freshly recorded 4K clips for this.

@@GrantKot definitely worth it

I think it's a good idea. This could save CPU by not activating node tiles with too few particles. I have considered something similar for rain, if the tiles have too few particles with low chance of collision, don't waste resources doing MPM on those tiles.

@@GrantKoti mean rain really doesnt need collision until it lands so it could be the opposite and only become physical after landing

@@GrantKoti’m glad that an idea like this might be feasible. something i’m really hoping overall for this technology is that it will leave room for stylisation and art direction beyond just the voxel ‘look’. one beautiful feature of water is the way the otherwise cohesive surface of an ocean can break up into a spray of droplets so small that, from afar, we just see mist. i’m also curious if something like dithering could be used to give powders like sand a finer, grainier look without too much of a performance hit

Oh yeah, something like this would be great for mist! I didn't actually realize that was the use case OP was probably referring to and not purely a computation saver.

@@GrantKot also, i’m not sure if you’ve already tried/implemented this, but a nice touch could be slightly darkening the colour of the sand when it gets wet, to represent the ‘total internal reflection’ effect (where rough surfaces refract less light when water permeates them). you’re doing amazing stuff here. thank you so much for sharing your journey in such an informative and accessible way! : )

side note, i downloaded the demo and have been playing around with it, its so fun

Thanks! I'm really glad to hear you like it!

Thanks! Yes, Liquid Crystal. There is also a demo of it on kotsoft.itch.io with local split screen.

I think first I want to make some products with it. That way I can fund further development. At this point it's not just a basic MPM simulation anymore (which I do have opensource of but kind of outdated and might even have bad multithreading/SIMD habits), but getting close to products. Sandbox, then game as well as possible VFX simulator.

@@GrantKot I completely understand, I'm just happy to hear that it isn't off the table in the long term. Not like I'm gonna use it anyway, my brain smooth as a marble. The Doom engine wasn't released the day the game came out, after all. I don't even understand how people begin to learn all this stuff.

Stop asking people to give away for FREE their very hard work. That has to stop especially in the gaming industry which is already tight.

@@sumofat4994 I don't understand, I never asked anyone to give away anything. I only asked if it would be open source eventually, out of curiousity. I could never do anything with the engine, and I know of people who charge a price to anybody for access to their source code.

Yeah, definitely. With the dirt, there is some of that mud behavior when it mixes with water. The material properties and effects are blended a bit through MPM.

I have the simulation running on CPU and the rendering on GPU. The game renders at 4K, might even try for multiple screens. Also targeting handhelds with iGPU.

On the clips with FPS counters, the simulation rate is the same as the render rate. I’m not using any substeps. The coarse pressure (only added one 1/8 level so far) helps with that.

Thanks! I think at 12 cores and 2 CCDs and maybe the PC builder skimping a little on memory latency (CL18) all contributed to amplified my poor multithreading practices. Initially there was some trouble with non-temporal stores on one thread being slowed down by atomics going on another thread, because I think atomics will cause the write combine buffer to flush. I also had an interleaved like pattern for updating particle tiles so I could write to the global grid because I thought not having a copy would be better. In the end, using thread local scratch memory was the way to go.

@@GrantKot Oh wow! Thanks so much for the detailed explanation. To me, the fact this can be done is a little mind-blowing. Have you run the revised multithreading routines on 5900x and see a drastic improvement over 8600k? When you ran it on your new laptop, did you have to rearrange the tasks and distribute them over P and E cores manually in your program for the new Intel CPU, like heavy loads to P cores and others to E cores? Or does the CPU do that for you somehow even if you run the same code(as the code you wrote for the CPU with homogenous cores)?

At the time when I did the Liquid Heating demo, when I merged the G2P and P2G (and other optimizations), for the same particle count performance was 2x faster on 8700K and the 5900X went close to the theoretical of being 2x faster than the 8700K. But then I got a bit demotivated because that whole thing about not knowing how to do the structures. I don’t differentiate what I send to the cores, just use taskflow which does work stealing

Yeah, that is a possibility. I think though the data of this hash table would have to fit within L1 or L2 cache for it to work well. If any larger (these days CPUs are very memory-bound), and you will have issues with cache coherence. The thing is, it can get pretty large, especially if you have different materials with different properties. It is something perhaps Machine Learning physics is trying to solve, actually now that I think about it, are there any successful multiphase ML fluid simulations out there? I'm going to do some digging.

First I will work on my own products sandbox/game/vfxsim with it and try to use them to fund further development and even accelerate the pace of development by forming a studio.

@@GrantKot sounds great! Good luck man!

I first want to give a go at releasing my own sandbox/game/vfxsim. If it is successful, then I could start a studio and further accelerate development into this.

Oh and there is the OS gvox system by Gabe Rundlett

Yeah it might be something I could try. I wonder if the water could be point rendered so the NN could see what is behind the water and refract it accordingly.

The main one here is MPM, or the MLS-MPM/APIC variant of it. I also take in some stuff from SPH (I think I was responsible for making the usage of density summation rather than integration over time more popular in MPM, it was common in SPH) and PBD (shape matching). For MPM, there is a Chenfanfu Jiang who is involved in a lot of MPM papers. So his publications page is a good list. Utah CSAFE also seemed to be very into MPM. MPM: Material Point Method MLS-MPM: Moving Least Squares MPM APIC: Affine Particle in Cell SPH: Smoothed Particle Hydrodynamics PBD: Position Based Dynamics

@@GrantKot thank you for answering! That's good info. Density summations instead of integration over time? Is that why your system is that stable?

Yup, that's the plan!

Yeah, I will try to do cars soon. There is a slight issue with wheels but I will try to figure it out and improve the game's structure system.

Yes, it is a 4K60 reupload. The previous one had major compression issues so there were many requests to redo it in 4K. There are some new 4K clips used as I had to recapture any footage that I no longer have originals for. In the future I plan to always upload 4K the first time.

Nowadays I usually build on DX12 to take advantage of async compute, where I run things like Intel XeGTAO and FSR.

@@GrantKot which build system do you use? cmake? msbuild? other?

Yeah, now I use cmake with MSVC clang. I find the clang vector extensions to work well for writing cross platform SIMD code.

@@GrantKot cool, and which code editor are you using? and which os?

i'm usually using vscode on windows