One of my favorite things about your videos is that you're not scared of giving us technical details a bunch of other creators leave out. And I always find the discussion of how earlier versions failed really fascinating. A thought I had about the item usage reward, have you considered doing something along the same lines as the guy who made the trackmania AI? His AI never learned how to drift, because driving solidly without drifting is quicker than starting a drift without knowing how to capitalize on it. He then trained his AI to prioritize drifting over everything else basically and then removed the drift reward after it learned how to drift constantly, and it then managed to unlearn the constant drifting, while still retaining the knowledge of how to start a drift. So once the AI is comfortable with using items you don't need to reward it for using them anymore, other than the indirect rewards that come from getting ahead in the race through the item usage.
I look to include even more details in the future! Starting with rewards and phasing them out over time is something I should really do more often, it’s a very elegant solution
Very cool! I'm developing my own Mario kart neural network/reinforcement learning code right now too. This series is really entertaining and has helped with planning/problem solving 🙂
That's really cool, hope it goes well for you. I know Reinforcement Learning can be tough at times, but hope you can get things working! Also, check the description to join a discord server! It's pretty new, but hoping to get many interested devs on there
@@aitango i remember in a previous video some time you said you used the rainbow deep Q learning algorithm. do you still use that? how often does architectures like the algorithms you use change like should you ever restart the learning if a slightly better algorithm comes out?
@@Litschi21 Still in development, learning different neural algorithms and seeing which type best fits the best for my system and provides optimization with both disk space and processing time to execution
You are playing as Dry Bowser as it's shown in the AI coaching section. Also it seems the AI model hasn't learn to take the shortcut, probably a situation too unusual for the model to have learned it casually.
I mean, even for humans, casual players who never looked at other people play usually dont know about most shortcuts, even I was blown away when discovering on the internet certain shortcuts that I never even atempted because they need things like mushrooms and I never found them while having a mushroom as a kid
12:53 You are playing as Dry Bowser. The timestamp shows DB backing up and getting an item, which is very unlikely to be an action with a positive reward for such a niche play.
Also missed a POW Dodge while the others didn't and, unless I missed something in the footage, was the only one that knew what a stand-still mini turbo was. It's probably Dry Bowser.
Gone back through some of your old videos and this is really impressive… is getting to the point where AI as a whole can beat the design of the original developers coded AI.. it’s scary and I hope companies find a way to implement stuff like this in the future for other games to become super adaptive Earned a subscription and a like from me and a tip for just how cool this is
This makes me wanna see AI handle a co-op game, but like this case where there's two or more independent AI that train the Master AI, rather than one AI holding two virtual controllers
I was waiting for a new video on AI Mario Kart! I am so glad it’s here, seriously underrated channel for how interesting this content is. Also everything makes sense which is the best
Yaay a new video!! I think you was playing dry bowser. And that is because dry bowser plays different and hold his drifts longer compared to the others
As someone who loves Mario Kart and has no idea about IT, I like the way you put the most important information in a nutshell. I appreciate the videos and I'm looking forward for some more!!
For the items, maybe try giving the AI a continuous small reward for holding items behind it? That would protect it from most assaults, and if it presses and holds the item button shortly after hitting any item box, it'll still nearly immediately use items like bullet bills and stars, while holding things like green shells.
I think the title is a little too complicated! And the thumbnail has a bit too much information in it. I didn't click the video the first two times I saw it because of that. Hopefully a little feedback helps!
Fantastic video as always, love seeing your content :) I guessed who you were! Shortcuts with item turbos + item usage + using reverse view gave me a strong suspicion, and the backing up to get a bullet bill was the confirmation ^^
Very cool! Do you think it would be possible to teach it more advanced tactics by connecting a racer to the main AI and have it repeat tactics like dodging blue shells with mushrooms? Or practicing shortcuts?
So I think using what's called "curriculum learning" could definitely be used to do that. Effectively, I'd just get a savestate right before a blue shell when the AI has a mushroom, then the AI would be likely to learn that.
@@aitango The movement. Drifting on general at the curves and mostly item usage. The Ai's would instantly use their items, however DK would strategisely use them to cut the track and to his advantatge. (However I saw some other people say Dry Bowser, hope that I'm not wrong)
Great Vid! Did you have to account for the different visual placements of the lap counter, position, and item, as with 4 player they're on opposite sides, or did you pull this info directly from the emulator?
Also grew up with Mario kart in various forms. On the mobile app version of the game I don't even play multiplayer since I'll often get pitted against really dedicated players who can consistently lob bananas in front of me. I can only imagine the ai here doing that on an even greater scale and it terrifies me
I know this would take longer but I’m wondering if you could train it without the position rewards, rather just giving a huge reward at for high positions at the end i’m curious if it will pick up a bagging-type strat also (this would take even longer to train but whatever) maybe lessen the slowness punishment and and increase the checkpoint reward or something so that the ai can find shortcuts?
Drybones is the only one who changes their strats based on items. Any time you get a mushroom you offroad the corner and actually time item usage. Also the only character that doesn't have pixel perfect drive lines lol.
@@aitango Would it be possible to the AI to play online with random people? Since it seems people are using the dolphin emulator to play now. Obvious problem being the AI can't drive on all the maps.
Quite a bit of work would need to go into setting that up, but it is possible. I'd likely need to get in touch with someone who runs something like Wiimmfi and get special permissions
I’m genuinely curious to see how long the AI would take to beat something like Elden Ring and what the gameplay would look like once it fully mastered everything. Also, great video and commentary. :)
If there was an AI in the future to learn any game (or Dolphin games only), I'd definitely try it on Shrek Smash n Crash Racing, since I'd love to see what it would be like seeing them melee and such lol!.
All your videos are really interesting! Do you plan on training a model for time trial runs that could beat world records? I race a very mean ghost valley 2 and id be curious if and how fast it could beat my PB! Its 53.918 no shortcuts/glitch
I think instead of rewarding for just using an item, they should be rewarded for successfully using an item. Like if they hit another racer with a shell, or boost through offroad with a mushroom
it seems that for optimal play the AI should focus more on finishing first, not just being first during the race. maybe after it reached a sufficient skill (not sure how to measure that) there should be an "finish position reward" that is slowly scaling up. first i thought it should maybe replace the position reward completely at some point, but i don't think it makes sense to to deprive it of such a useful source of reward altogether.
I really want to do something like this, but I have no patience to search through memory values for useful data, nor an idea on how to access them (memory values) in real time in python.
I could tell that you were playing as dry bowser because you were adjusting your angle or rapidly moving your joystick while using standstill miniturbos
Cool AI. It plays almost like a pro but I wonder does it can run on different map with the same knoledge. Cause I mean right now you have the basic of what is a mario kart race on a simple map, so theoreticaly it can generalize to other map. Maybe with sub optimal line but you know... I would love to see your AI able to play on different map.
if you now give each of the four ai drivers full access to all of the screen then coul it probably get to be even more devious and maybe even ambush more
You were Dry Bowser. How can i tell? The AI was releasing their mini turbo frame perfect, whereas we could actually see the blue flame on Dry Bowser as you drove.
Would be cool if you gave it reward for going through offroad without a speed detriment, like with a mushroom or whilst hopping (as long as it actually saves time)
There are so many tells that you're playing as dry bowser, I'd be shocked if anyone didn't get it. Forget the backing up for an item at 12:55, the way the AI drives can only be described as 'indecision'. It still struggles to complete drifts, it loves to hop out of them instead of holding it for the entire corner. It does stupid wheelie-hop-wheelies that no human player would do unless they were looking behind them trying to dodge a shell or something. Also, current MKWii players are much much better players than yourself, as these are the people who have stuck with the game and downloaded things like CTGP and played online with Wiimfi. You'd be below average in online servers today (
idea i notice that if you connect mario kart trough wiimfi the in game cpu's get a little more difficult then normal if you could apply this to the game and train the AI's again may be the then get better by learning from te cpu's
The only problem is the AI doesn’t understand concepts like bagging and so it can only front run. If it was set to play the team mode that is popular for competitive, I wonder if it would figure it out by itself.
Would it be possible to make a reward for hitting an opponent with an item to boos the chance of that heppening? I believe the game keeps track of it in one way or another because if you hit someone with an item, your character will react, so I feel like it should be.
I wonder if it would preform better or worse if it was given only specific memory values rather than images. I also wonder if it would compute any quicker.
It would certainly compute faster, however the challenge would be collecting enough memory values that actually give the AI enough information. In theory though, it could work very well
I think they're just what the game defines. However it must also interpolate them to show your position, so, using that data would give effectively unlimited checkpoints.
Yeah so I can technically use as many as I want, but the reason I don't use too many is because they aren't evenly spread out when you start looking really closely. If I use fewer, they tend to be a bit more consistent
