This video is incredibly high quality! I cannot imagine how much effort went into it, both in making the actual AI, and the editing of the video itself. The little v1 animations that play throughout are also really cool. This is the kind of quality content that keeps me on youtube. Great work!
A machine created to be the machine to kill the machines. Give it a year and we are doomed. I initially thought that this was a Uni project but the jokes made it clear it wasn't lmao. Super well scripted and edited, wonderful video. The code was very easy to follow from start to finish in the way you presented it, dynamic all the way through. For the added mechanics I think coin tech is very complicated to do considering the sensors you mentioned in the video, but since the coins inherit the velocity and crosshair placement of V1 a way to shoot the coins would be to look up (or down too, which would make the coin tosses more precise but it will have to jump), toss a coin and shoot on the sliptshot window, though it may be easier for the AI to do this without the splitshots and rather just coinshots. The sensing for tossing coins would be an image reference of the Marksman's display where the green discs mean it's ready, and once it changes weapons the AI can read the display and tell if there are coins that can be tossed. I'm unfortunately not very knowledgeable on how to code so for the coinshots I'd just make it go in a straight line as it tosses the coin in the air and shoot it after a second or two, then return to moving and killing the way it normally does lmao. Great job! Hope you blow up soon!
thanks :) i did some testing with the coins and basically as u said the easiest way to shoot them is to stay still, look fully up, and then toss. this is really useful for shooting enemies that are placed on a high ground bcs the coin actually flies up pretty high
What a gem of a video! Instant subscription! I'm already knee deep into your repository. What about training your sensor script to detect red balls, and prioritize them, using their size on screen as a trigger for parry?
Would love to see an updated video where you use a secondary GPU to have both the depth and object detection networks on separate GPU's to fix the reaction time issues or find a light depth network or maybe even make your own as a video then use it in an updated video instead either way really enjoyed this video.
How would you give this thing object permanence? If it wouldn't run too bad, I'm assuming making it take older scans into account would be a good improvement Also, you've almost convinced me to learn python solely to make a cracked Ultrakill AI
Random side note: I wonder how this process would work on an isometric view ARPG (like Diablo, etc.). The depth mapping wouldn't be necessary, and calculations can largely remain 2D. I think it'd probably take more object detection learning due to both enemies and obstacles needing to be identified, but once complete the bulk of the operations can live in the weights, reserving more power in the loop for gameplay tweaks and what not...
Try to extract the depth buffer instead of using extra neural network for this matter. Most likely, the game uses OpenGL to draw shaders on the screen, so this thing already has depth buffer calculated in it.
is there no modding API for ultrakill or steam? I mean, surely there's a way to get input from V1's perspective without being JUUUST screenshots, right?
lmfao rip code bullet (it does confuse me why he tends to prefer completely remaking games himself but I still love his channel and understand such a desire to c r e a t e)
I was wondering if you could pull the depth buffer from ultrakill's shader, it is generated while the game renders, and determines what polys to cull from the render queue. They are formatted in a similar way that the ai estimates depth. You might even get clean silhouettes from the entity's on the map. I am unsure if this process would be faster than scraping the screen though.
a few ppl suggested that and when i think about it there is probably both a depth map and some sort of enemy mask on there. maybe its even possible to see enemies thru walls with that idk. thanks for the suggestion :)
Half motivates me to make an AI to play ultrakill that actually learns how to play the game, but the game mod I'd have to create would be at best mildly annoying. Maybe if I end up finishing my machine learning library :p In any case the editing in the video looked pretty good, the way you went about the actual ai seems to be pretty good in terms of playing without any internal access. Maybe if I get bored tomorrow I'll add some features
@@8AAFFF It's nothing very impressive as of yet, I'm mainly just throwing together my own version of the gym standard with shared memory buffers to make everything as fast as can be, then I'm gonna code some example algorithms and environments and see how fast it is
@@ctag07 i once tried making sort of a pytorch clone for training small models, but got stuck on backprop and abandoned it XD. Good luck with the library though
@@8AAFFF Backprop is basically where the complications come from, so that's understandable lmao. I did the same thing over a year ago and had to end up stealing backprop code from someone else, and I didn't use numpy so it was as slow as could be. I'll definitely be taking a look at the code tomorrow and possibly reworking or adding a buncha stuff, depends on how bored I am :D
theres a minor assist option that causes all enemies to be a silhouette of a single (customisable) colour. Couldn't this be used to make the enemy detection script much more lightweight? idk if it'd mess with the depth sensor though. dead bodies dont have silhouettes so itd also solve that issue too
bruuuh where were you when i was making the targeting sensor thats such a good idea! besides maybe explosions warping the color but its prob fixable thanks for the suggestion :)
@@8AAFFF do u have a discord or smthn, I was considering contributing (I’m working on another ultrakill-related project rn but once that’s done I’ll see what I could do)
well initially i was trying to make a genetic algorithm to get scored by how much style it got in N cybergrind waves and that would have worked off of screenshots but especially with the depth estimation youre right it could be way faster pulling directly from the gpu
I feel like a sound sensor would be good to give the ai an extra level of security in its assumptions as 3d audio can clue the ai in on enemies and dangers and that it cannot see.
@@8AAFFF it has baked in surround sound. you just have to have physical outputs connected, or a soundcard that can do physical outputs without needing to have any drivers connected. (Sound Blaster Z SE user here, I've used it over 2.1 surround sound.)
Idea on how to improve navigation: I'm not that great at ideas, but ULTRAKILL has a navmesh (I think), so that could potentially be used. If not, then maybe a way to generate your own navmesh? Also, amazing video, top notch editing and explanations. (also it's about coding, and ULTRAKILL, currently two of my favorite things, so I'm probably biased, but this is absolutely amazing keep it up!)
There are multiple problems with using the navmesh. a) Based on the rules stated, navmesh isn't usable b) Navmesh wouldn't be that useful (a little bit subjective) Navmesh is, simply saying, a bunch of invisible nodes/points in the map that enemies use in the game to move around, specifically, enemies move to existing points a) The AI can't see the navmesh. To get access to it, you would need to hook it up to the map files of the game, which isn't possible b) Navmesh isn't set everywhere. You could generate it, but doing that would require some level of connection to the game files (s. point a). Besides that, it's only useful for understanding where the ground is. Which is... useful, considering how AI sometimes falls down the Cybergrind arena I only know about how navmeshes are used in other games, maybe ULTRAKILL has it's own way of using it
Cool video, just one little thing... please check your audio mixing. I could barely understand you with all the heavy bass music in the background, which made it muffled.
cant you hook into the game's memory and get real game data? then you can train a model based on runtime game data and player inputs. would be interesting to use the scoring system as an actual reward/punishment system, or maybe your own artificial system so it uses weapons in a specific way, to make it not just good at playing but also good at playing cool.
Never played ultrakill but love the editing man! Since you're labelling the youtube chapters, get rid of that top timeline and let me see more of that editing full screen 😝
This is almost just V2. Make it learn how the other weapons work, further develop the phases, and this could probably compare to an actual boss in the game.
my friend made an ai that literally could play any game its name was bob bob has been retired and is just gonna be used as a reference to make a new ai sadly bob had a fucking personality
This reminds me of when microsoft made A generative AI that plays minecraft! I was thinking you could try to use A photo scanned replica to search for the cameras position based on screenshots, but that sounds unreasonable in real time. Maybe if you run the game slow enough?
idk because the game time is just a slider in the accessibility settings that goes down to 0.5 but yes actually mapping out spaces to not visit them twice is a good idea :)
feedback: 1: I added a neon glow to all of the texts, even though yt compression kinda ruined it, does it look good? I like the sharp contrast berween bright text and a black background, but I also really love this subtle bloom. It helps the content stand out, without making the text hard to read. I'm also a big fan of the ASCII progress bar lol. Very cool. 2: I tried to write the script in more of a [to solve this problem, we can do...] format instead of the usual [I solved this problem by doing...]. was this a good decision? Yes, that's it! It's a pretty efficient way of making sure you don't go on for too long on tangents that should become their own video, and helps explain the process of what you're doing to the viewer.
I think one of the important steps is to make it see more than 1 frame at a time. Most AI I've seen playing games are basically like having lobotomy and forgetting everything 60 times per second (or whatever the framerate is) There is no object-permanence, no mental objective list, it's basically like someone new walking up to the computer, seeing 1 picture, and deciding what to do before the next person walks in without any communication. An easy way I've seen before is to give it a few inputs and outputs that loop on themselves, maybe the AI can be smart enough to figure out how to store useful data in them?
For better maze navigation you could use the SLAM algorithm (Simultaneous Localization And Navigation). There are python libraries for that, although I never used them. SLAM is made to create a map of your environment while positionning yourself in it. To use SLAM you will probably have to use a depth field (which you do) and an estimation of your speed/traveled distance between time steps.
just some ideas for you or anyone that wants to work on this, ultrakill allows you to turn on enemy outlines that can trigger based on distance or you can have them always on, you can even make outlines go through walls but thats probably just gonna confuse the AI. another thing you can do with outlines is making an outline cover an entire enemy so you make the color something really easy for the AI to detect like a bright pink or purple. the problem with that is that the AI cant tell which enemy is which, it can only tell that its an enemy But if you want the AI to be able to recognize each enemy, you can make a custom color palette and enable it in the settings somewhere, custom color palettes let you make every color on every enemy different so you can make every enemy a different single color for the AI to recognize far more easily, the problem with this is that the AI will still get mad at corpses (outlines only show up on alive enemies) I made this comment at 3AM so it probably doesnt read that well but I hope this comment can still help someone that wants to improve the AI
I'm not sure if it's feasible, but what about accessing the game's memory during runtime? There might be addresses containing useful information, which could reduce the reliance on a model for screenshot recognition.
VEEEEEEEEEEE UNOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO DURNMUMRUD MFUDM PAPPAPAPA FDURDRMDUMDURMRURMUDMJRMDURU AOAPAPAPPA XURMDURMDUMRUDMRJRJRURU PAPAPA KILL IT DURMDURMDURJDMRMDJRMDJRMUDMRDRM PAPAPAPAPA KILL IRT KKI KI KI LL IT
