Can AI play a track it's never seen before? | Mario Kart Wii 

I think putting the AI on a track it already learned, but with a texture hack could be interesting, or putting the AI on a different track but with the same textures.

Now I want to see how well the AI does on every track, and then have it go against a human.

I think I get why the AI struggles so hard the moment it makes a single mistake outside of training mode... If it doesn't fall off a cliff, it has no idea how to recover! I think if you could figure out how to train this AI to recover from mistakes? It would be a top player.

I really like that you keep pointing out the problems with AI approaches (not conceptually, I mean practical problems, like how you need a convolution, and how improvement quickly stagnates). Too many people out there who only point to the cool things possible with technologies from the AI field and extrapolate from there.

video idea: ai plays against itself (as other racers) and can use items. reward is based off its distance ahead of the others (i.e winning the race)

I think you could get better consistency if you added some random filter to the different images. Making it less relying on the specific looks of levels

There's so much stock footage in the intro I thought it was sarcasm

It would be cool to compare how the AI's average reward compares to that of a human.

Now for the ultimate challenge: AI vs. Rainbow Road

I’d love to see this ai race on a track where every texture is replaced with a color for its corresponding material (like off-road, road, wall)

when approaching a limit like a lap time, progress in terms of time will go down, but the skill is improving. I would not say that Twice as fast is Twice as good, it's way more than that :) if someone is 1% faster than the other close to the theoretical limit, I would say that he is way more than twice as good because it gets harder and harder approaching the theoretical limit :)

It would be cool to train it on every map, and then give it custom maps and see what it can do with that knowledge. If it’s possible maybe even other Mario kart games

I’d love to see the AI learn some custom tracks from CTGP as well as any AI’s you’ve had to kill because of failure. Great video as always though!

Thanks for putting the music u use in ur summary and notes! Love the tracks and was grateful you gave the artist credit!

Awesome video bro!! Great AI you created there. Would love to see more

ML engineer/researcher here. Good stuff. Try training but with occasional input perturbations based upon the average score achieved by the network (will self-normalize as it continues to get better). This should make it more robust towards running into walls, etc, trying to approach a learned invariance. You'll likely need a power distribution of multi-button inputs to simulate things that would cause you to thonk into the wall like hop cancels and etc. This will also provide the requisite noise for your network to explore new strategies, ala genetic evolution. Hope this helps, and this should greatly ameliorate and/or fix your issues here in terms of/w.r.t. the retraining, esp as the 'learning space' of the network becomes so much larger. Gating it based on performance as well ensures that the regularization/extra data-learning/learning to be robust happens naturally as a curriculum, and affords a consistent level of information ingest to the network over the course of training.

been waiting for this stuff thanks, instead of just training one for each map

I hope winter comes soon for you to keep your room cool. These videos are awesome!!!

I love these vids mate. Your explanations are good n all! Keep it up!

"Less savvy side-kick" *shows picture of a robin*. I see you, editor! 😂

Your videos are genuinely such good quality and keep getting better! I really enjoy watching them and I'm sure you'll soon get the recognition you deserve!

Loved the commentary on this one! Defintiely feels like an improvement to the last video. Overall the video felt made a bit better, but I can not put my finger on what it was.

My biggest wish is to see how it fares vs a human player after like the 120 hour mark. Cause just the limitations you put on it with the controls would probably make it pretty easy to beat

I wasnt a big fan of the random images and gifs but the racing visuals were entertaining :)

Hi! I love your videos and they're super interesting to me as I go through my software engineering degree. I'd just like to say something about a thing you said in your video. 5:27 "Here's where our AI brings an objective perspective to the table. It doesn't possess these personal biases or preference; it simply just learns and adapts." I think it's important to make sure viewers know that reinforcement learning models are not necessarily always free of bias. It's definitely true for this example and your other Mario Kart AI videos as there's a definitive metric for measuring success in a goal's completion, but reinforcement learning is not a perfect solution to erasing biases from a problem. Whatever biases the creator imparts into the AI (i.e. what is considered rewardable/punishable behaviour), unknowingly or otherwise, will influence how the AI makes its decisions. It's up to us to determine these biases and be aware that we have the ability to make them intrinsic to the AI. Obviously you never claimed that reinforcement learning is free of bias, so I'm not trying to make a callout. I just think this is info that more people should be aware of :>

i heard jort storm in the background lmao, also amazing video

Finally! An AI video where the AI actually learns to drive, and not spit out an evolved series of inputs!

This is your best work yet! The improvement since your previous videos is incredible. Keep it up man!

Really cool to not only see that the ai is doing decent on the unknown tracks but also manages those after very little training. Why dont you do this more often and always start from ground zero ? I would love to see you give an ai a really good control over the game, then start with one track until it can beat it consistently and then go to the next track. It would be interesting to see how long it takes for each new track and how its driving on an unseen track might change depending on the tracks it has seen. You could then see how it performs on the original stage after it has seen so many others. You could then make changes again like enabling npcs or items to see who that changes it or how long it needs to adapt. The important parts are to show enough footage of the new stages and performance that we can see the improvement. You could also turn it into a series, a challenge of sorts to master the entire game.

Great video! Love it! Thanks :)

"its going to just rely on the screen" finaly someone did what i always wanted.

Again another amazing video on how AI can adapt and learn then carry its previous learned attributes to the table in the next stages. It is an awesome way of showing how it can retain its reputation and keep up. All it has to do is tweak a little for the next track and its all good, and all of that is part of the previous tracks processes. This is the AI I want to see impact the future of gaming. Let us Pray 🙏

these videos are super interesting, kinda nuts that after you trained the initial 4 tracks an hour of training was enough for more tracks

I've been watiting for something like this!

What happens when you put the AI back on the original tracks? Does the new track's learning hinder it? Does generalizing it to additional tracks lost its specialization to the original? Maybe this is a silly question lol.

I’d be interested to see a pro MKWii player go against this bot! Keep working hard!

its almost like it has an infancy stage for the first day

This was great, exactly what the previous videos made me curious about! Though a note on the "objective difficulty" of tracks: wouldnt a reinforcement learning AI be equally susceptible to gaining a bias due to the track misaligning with how its learned to drive up to that point as a human? Not to mention issues with the reward function, say it can drive off road losing all its speed but hitting the next checkpoint faster. It needs to drive well through that part of the track for it to stop running off the road for that sweet reward bump.

Is it only me but the first 55 seconds kinda went crazy

I know this would probably take way longer, but I’d love to see the ai with all normal controller inputs (minus 360* turning) and only being rewarded based on what place in the race it came. Wondering what weird strategies is would come up with to win, maybe find shortcuts.

Nice Vid man!

If we were to read memory well enough, we could be able to learn when the player character is offroad and punish the AI simply for going offroad. This may speed up and teach the AI to better map out the safe areas of the circuit.

Ah, yes, the type of literal video editing where if the voice says “square root”, the video shows a square shape, and, next, the root of a tree. Premium stuff

I'd love to see every track added in little by little ! See how much that reduces the training time every time, maybe even to the point that it would actually be able to complete a race on its first try ?

this is what we've been waiting for

Love the vids and have learned alot about how ai works

Nice video ! I have a idea to make it learn faster : using its current position in the reward. I think at the beginning it makes zigzags to gain speed, but it is wasted in zigzags. Giving the position will ensure the speed gain is efficiently used

This was great, thank you, sir

Now, take this AI trained on 4 courses and put it on 4 other ones and see how it adapt. Do it 3 or 4 times and you should have an AI who can plan every tracks or almost

Now try changing from Funky Flame to a Kart after it’s mastered an entire GP

It’d be cool if you let the ai train on one track only for a whole year straight, and then have it go against a human on that track.

Sup @aitango love your content. A lot of people doing neural networks on youtube seem to overlook something. In your input layer. For whatever inputs your using. Its often better to use the variance of the value as the input (or sometimes both). So instead of say, reading RGB absolute value of a pixel, it would see the change of the value. Absolute values are often too arbitrary in nature which can make learning them take longer.

These videos are always so interesting. Keep AI gaming coming! ❤

You should try making a similar ai for a game called track mania, the scenery is so much less diverse and the driving mechanics are simple but elegant. I also feel like only allowing the ai to see the screen isn’t doing us humans justice, we memorize tracks, we don’t just look at the screen and decide what to do based on that. Either way, great video 👍

This is amazing. Would love to see this AI attempt one of my tracks!

Amazing! I would like to see what happens if you train it on even more tracks and see if it becomes better at driving on new tracks. Also, I have an idea that might be a bit far-fetched for this type of AI. Could you increase the randomness of the AI and try to make it learn either a built-in shortcut or even a glitch (if those are common in MK Wii; I don't know much about this specific game)?

This video gave me a interesting idea what if you trained a ai on a track but then once it learnt the track you put it in mirror mode

5:39 I laughed because he was doing so good and the speech sounded so motivating

Loving these AI mkwii videos, you should outsource your compute to your subscribers to make something extremely strong!

You should make a tutorial on how to train your own AI, it would be fun! I love your content!

I love how the footage used to represent the AI being taken is someone robbing a car

Really impressive that it was able to adapt to the new tracks so quickly!

It really makes sense the AI progress slows down after reaching a certain point, cause like in any speedrun there is less ways of saving time so you get just some micro optimizations of the run

i like how the thumbnail is basically "ai never seen before" sounding like this is some crazy god ai :D

Jortstorm, Jortstorm’s coming tonight.

Okay, video was fantastic. Great job. But I have to know... Are you using AI to choose the b roll footage? It's like... Related to what you're saying but only by a thread.

I would really like to see a video of how an AI (similar to the one in your training with items video) could handle HARD CPUs, unless it is already doing that (which would be amazing!).

After training it for a short time on the new courses, is there a possibility that the AI would actually perform _worse_ on the initial courses it had spent most of its time training on? Basically like, could the AI have become more intelligent overall for a diverse set of courses, yet less specialized for the initial four it had been tailored made for?

Now put one on ctgp (Maniacal laugh)

Have you thought about putting together a tutorial (video or text) about how to setup the felk/dolphin repository you are using? I was trying to get it running but its setup documentation was not that great.

I get the tiktok text intros are good for the algorithm and thats good and all, but seeing the text come up word by word with the text at the bottom too is kinda funny to me

Now i have my answer ty Next the ultimate test would be to learn the item again and then all of the tracks just so it know the color and then make it compete against mariokart pro but ik this could be impossible as how difficult it looks like

That Yoshi Falls race was impressive!

Would love to see the AI go up against a human.

Heres an idea what if you teach the AI all 32 tracks. Do a track and if it fails they go onto the next track. Going from the Mushroom Cup to the Lightning Cup.

This is very interesting, good work! Now I wonder if it would be possible to train the AI in such way that it's main goal would be discovering shortcuts, glitches, etc.

Impressive! I've been trying to do something similar but by collecting and labeling data myself (recording my screen + my own controller input and using it as X and Y values). I've never been good at this memory address stuff, but it does seem worth while! Care to share your progress in finding these address values / pointer locations? I'd love to create my own agent and let it just run, to see where it ends up

Do you use AI to pick the stock footage?! (First off if yes absolutely genius for your channel) I loved the fishing clip for "tackle" hahaha

My dream would be a 1v1 between AIs to see how different or similar they learn and if one of them winning the most races

Man, do I wish I knew how to do this. I would love to take an AI and train it to play fighting games. And I’m also curious what the best time your AI gets on those tracks are.

This is fascinating. I noticed you chose tracks that were relatively free of hazards, cliffs, jumps, etc., which makes sense, but it makes me wonder if it would be possible to ever train an AI to handle all the craziness some other tracks would throw at it

So this is why there's just a million people who always play as Funky Kong and all play fiercely competitive...

Is there a github repo for this? Great video as always.

Stil camt seem to stop them bots from hopping around like bunnies, but was REALLY cool to see how relatively quickly it could adapt to the new courses

Have you considered livestreaming the training process of one of these to RU-vid? That way you get to record all of it not on your device, plus people can watch if they wanna and check in on its progress. The only downsides I see is if you're data capped or bandwitdth limited

The final evolution of this AI is playing other kart racers

I like all the stock images that have minimal relation to the words said

I love all the stock footage used in this video for absolutely no reason lol

It's very interesting to see what is essentially known as few-shot learning in other areas of AI, done with reinforcement learning. Curious to see more of that. I am wondering if there are any research papers exploring this

I look forward to racing games in the future using this sort of tech, as well as basically every other game type. It’s going to be a lot more engaging fighting an AI that actually knows how to play the game as opposed to the more simple AI’s that we currently see. Especially if the neural network ones learn as you play.

I think this is the best way to train ai because its not just memorizing the track. Plus it gets a chance to prove what it’s truly capable of, rather than just continuing to race the same track. Because how do we know whether or not its memorising.

Would be neat to see what this trained AI could do on the same tracks, but with items added in. The added chaos of items would be a unique major hurdle to cross.

i think it would be cool to see how the number of pixels or details input (like adding more pixels near the horizon line) affects the training

Hi Tango! Very cool!

It would be interesting if it was possible to make an AI that learns faster, i.e. get the same fitness in much fewer hours. I've had this idea (but no time to implement it myself) that you could train a transformer that just predicts pairs of frames (in the embedding that is produced by the vision CNN) and actions. You see, the transformer learns from literally every frame, even in epochs that fail to produce positive reward, so it could be much more efficient in building a good internal representation of the game. The pre-trained transformer can then be put through reenforcement learning to predict pairs of actions and frames that actually maximize the reward. This is inspired by how ChatGPT was made: They first pre-trained a transformer that just gobbles up all the text there is, and then they use REL to get it to follow instructions. If they didn't have the pre-trained transformer, then it would have been completely hopeless to try to get an instruction-following AI from human feedback alone.

I just noticed the background music in some of the video is the same stock music Charlie slimecicle used to make jortstorm and I’m now hearing it everywhere lol

no fucking way bro is using JORT STORM for the bgm

great, now you can train it to navigate the main menu, start up a new game, and complete the Lighting Cup

Unrelated but this makes me want to go further with some CPU edits I made once. You make me want to release a pack would include upgraded CPU action codes in the KMP files of courses, CPU max speed edits, and if I manage to do it, improve CPU's overall item use params.

You could argue that it already learned how to maximize the reward and in the 1 hour it overfitted with ease to the layout of the new track. To counter this argument train it on a track and then test it on a similar looking track, you could even do a mirror version of the first track first