My Flow Field Pathfinding with Unity DOTS/ECS 

No offence, but really bad example. Using physics for costs generation and rasterization obstacles to flatten grid - terrible idea. Get rotated rect projection and rasterize object to flatten map will be much more efficient. Many unnecessary copying, managed types, FlowField and cells as managed types unnecessary etc. Better is implement pretty simple, and much more efficient own native container and couple of unsafe types and operate on pointers. It was about the implementation of general things. Now about algorithm implementation - naive approach. This approach not scales at all, and wouldn't work efficiently on even 500x500 map with 2k of agents (map scale extremely matter in flow field pathfinding approaches) with multiple destination points (current implementation don't even support multiple fields). Caching and field reusability does not exist at all here. This should be implemented with regions separation and Hierarchical A* on the top level of regions with regions merging and per region flow fields generation, in that case, most part of agents will reuse already generated fields, and longer you'll play - lighter pathfinding become as cache populates (up to some reasonable size) and units don't even generate new fields and just reuse them even in the future path requests. Moreover, regions approach handle dynamic obstacles change much better as it requires only rebuild a small piece of the map (a couple of regions). I recommend reading these two papers: www.gameaipro.com/GameAIPro/GameAIPro_Chapter23_Crowd_Pathfinding_and_Steering_Using_Flow_Field_Tiles.pdf grail.cs.washington.edu/projects/crowd-flows/ Again - no offence, you asked, I'm answered :D P.S. As some foundation, to make it clear that I know what I'm talking about, you can watch this my pinned thread on the Unity forum (in the area of 2 and 3 pages): forum.unity.com/threads/unity-ecs-and-job-system-in-production.561229/page-2 Or just look at the game we are making to understand that we have enough experience in this to make recommendations: store.steampowered.com/app/1272320/Diplomacy_is_Not_an_Option/

@@eizenhorn91 No offense taken - this is some extremely valuable feedback, I really appreciate you for taking the time to write such a detailed response. The cost field generation was meant to only be a quick (to implement) and dirty way of going about things, but I do appreciate the suggestions to improve performance. This project has been my first implementation of flow field pathfinding so I was definitely focused on simplicity. One of the things that really appealed to me during my research on the flow field algorithms, was how you could break it up into sections and combine it with other algorithms like A* like you mentioned. Super smart to cache those sections to reduce generation time over the life of the game. I've seen a bunch of your posts on the forums and they've been very helpful as I try and piece together Unity's ECS and the best way to go about things - so thanks for the tips here and on the forums 😀 Again, really appreciate the time you took to write such a detailed response with reference material included. Also your game looks really freaking cool and I wish your team the best as you head into early access and towards release. Looking forward to seeing more of it ☺️

​@@eizenhorn91 Thanks, I got value out of reading your feedback too. It's great to see you and TMG handle this so well, info-sharing in the industry is essential to increase standards and help people learn the necessities after all. Do you know any good resources on using pointers and unsafe types? My only formal experience with programming is C#/Python/JS, and we never covered pointers at university, sadly, so anything that offers a holistic view of pointer-use I'd appreciate a lot, as all I can find when I start to look into it, is specifics, much too often separated from the whole..

Fair points. I do think the job system is really powerful and it's really nice that you can implement it with existing code fairly easily. The more I dig into ECS, the more I see just how complex it is. Yes you can get excellent performance but you are also locked in to a very specific way of doing things with lots of boilerplate code. Will be interesting to see how it evolves to become more user friendly and widely used.

@@TurboMakesGames I just finished converting my bois Monobehavior + Jobs + Graphics.DrawMeshInstanced code to ECS yesterday, and I'm actually getting very similar performance, with ECS being a tiny bit less performance. I might find a way to optimize the ECS version a little more. Either way, it is still a LOT of boids. I'm thinking ECS might not really be that much better for a single system, but the more systems and complex the game or simulation, I think the more ECS will shine. That is a little harder to test though. I do admit that once you have the ECS bugs worked out and have sifted through the documentation, it is rather satisfying to get it all working. Even though I didn't get the massive boost I wanted, now that I've given it a proper comparison I think I'm even more excited to see what ECS becomes.

I like the idea, I'll definitely note that one 😀

Simply to use it in the DoTs system I suppose

ECS is just all data. The entity is just a reference, you can have an empty entity with no component data, but I'm not sure if that would be useful for anything. You store component data on that entity. You can have all kinds of unique data on individual components on that entity. This is a good reference www.dataorienteddesign.com/dodmain/

Check out this video I made that introduces the concepts of ECS in Unity - ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-HVzSTEIAXi8.html Like Calabi said, the entities don't contain any data, they just point to data components in memory - each data component can have its own unique values. Entities with the same components are grouped together in memory for quick access in things called archetypes, but you can have pretty much as many archetypes as you want (until you run out of memory) You can convert GameObjects to entities but you can't convert entities to GameObjects. If you still want some behavior that entities can't do yet (there are still a bunch of things that don't have full support yet) you can inject a GameObject into that conversion, then sync the transform of it with the entity each frame. Again yes, entities can have references to data components with unique values. Entities with the same component references are grouped in memory for quick access, but you can still easily add/remove components for dynamic behavior at runtime. There is also a concept of SharedComponentData where many entities reference the same data component with the same values - can improve efficiency and memory usage if many entities need to reference the same values. The ECS info is quite sparse but I look at the official Unity documentation a lot, the Unity forums, and some other blogs and RU-vid videos. Hope that helps make more sense of it all!

I know I'm pointing outside of Turbo Makes Games channel here, but I think if you want intro to these concepts / a good mental model of it, Brian Will has some great videos on this. Probably has a lot of overlap with things discussed on this channel, but it's more collected there and was a great primer for me. :)

Hybrid Renderer - 0.8.0 Entities - 0.14.0 Burst - 1.3.2 Jobs - 0.5.0 Physics - 0.8.0 Those are the main ones I'm using in this project. You may just need to clear the errors or restart Unity

@@TurboMakesGames Thanks, it seems like I need Unity 2020.1 instead of 2020.2 also Unity.Physics version 0.4 instead of 0.41 to get it to compile without any errors. But then when I run it, I'm getting errors like "The managed class type `TMG.ECSFlowField.EntityMovementSystem` is not supported. Loading from a non-readonly static field `TMG.ECSFlowField.EntityMovementSystem.instance` is not supported" also lots of burst warnings.

I changed to Burst 1.3.2 and I'm still getting errors like " The `try` construction is not supported" I'm missing some version.

I am using Unity ECS Physics - definitely seems that the slow down is caused when all the entities are close and lots of collisions are happening

Unfortunately, Navmesh is not compatible with ECS right now - I will definitely make a video once this support is added 😊

@@TurboMakesGames It is. There's a thread on the unity forums.

Didn't end up doing one as I didn't really like how the ECS implementation of this turned out... Though now that I have much more experience with DOTS and the API has changed so much, this is certainly something I'd like to revisit

I'm using the latest - version 0.14 preview 18. Looks like the GetBuffer function was added in entities version 0.12

Great point, I do think some sort of flocking algorithm would help so all the entities aren't essentially fighting over the same position on the map. Lately I've been trying to work on writing cleaner "self-documenting" code that doesn't need to rely on comments to explain what it is doing - though I could see how it would be helpful for the showcase/tutorial videos 😋

@@TurboMakesGames My first thought is collision physics becoming an issue. Since you have already sub divided the space into cells, I would look at using some 'bucket' system that limits the number of units per cell, and only tests for collision among units and objects within the same bucket, greatly reducing the collision tests. I think thats what the Unity 2017 ECS demo did.(I would profile the code first though)

Yeah definitely feels like the physics interactions are slowing things down, haven't had a chance to profile it to be sure but that is likely the first place I will be looking. Thanks for the suggestion though 😊

@@TurboMakesGames I just finished writing my own implementation of a Monobehaviour/non-ecs version of the flowfield (with the help of your other tutorial, thank you so much btw) and i thought of a possible solution that wouldn't require writing a flocking algorithm to help alleviate that performance hit. What if you just made it so when an entity reaches the destination, it sets a boolean value "arrived" to true, and then any entity that collides with an entity that already has that boolean set to true, then it also stops, and sets it's boolean flag to true? Could even add a short 0.2 second timer before it sets the flag, just to ensure they really bunch up, but not excessively. Might cause performance hit while traversing the world, but you could just ignore that check if the entity is too far away from it's destination.

I like that idea a lot. I've already got the destination reached value in one of my data components so it'd be as simple as setting that to true on collision with an entity where it is also set to true. I'm a fan of the simple solutions 😊 I'll definitely experiment with that - although this wouldn't help with any performance issues experienced while traveling to the destination. I guess I'll just have to play with it and see how it runs. Thanks!! (glad you made your own flow field BTW)

Thanks for the suggestion, I think that would make for a good comprehensive DOTS/ECS tutorial!

You can get waaay more complex than this if you need the extra features 😋