Using Image Recognition to find Mario 

U right

Nah, spot on here. This kind of image recognition, especially using small snippets on a big view, can take a good amount of time (in computer terms). So in the time between it took the screenshot and clicks, the image has changed without the program knowing. Could theoretically solve that by having it perform the whole process twice, and extrapolating to where it should actually click, but that would be immensely difficult, especially once you account for the case of it going off screen (and thus coming out on the opposite side). And even then, you might get false results when you have it not moving in a straight line, like all the levels where they move in a sinusoidal pattern. Which would in turn be even more obscenely hard to code around

@@genericfakename4372 and adding to that, even if it 100% predicted where the right one was gonna be, there is a huge chance that in that position it would be covered by another icon. I would have to keep following the path and predict a position where no other icon would clash... ouch ... kinda awesome that our brains can just do that automatically and we just take it for granted

@@zewzit True

I think that also, it might be clicking on where the center of the icon would be, if it wasnt covered by other icons. Instead it should click on a random sub-tile that matches 100%, or the sub-tile that matches the closest.

And sm64DS

Nah for me it was SM64DS. I didn't knew that these Minigames are in another game too until I was 22 or so (A few years ago)

@@AlryFireBlade same!

SM64DS came out 1st and had a lot more minigames than NSMB.

@@ugonnankwonta6472 exactly! It deserves more recognition.

wouldbt they fail as its using scren shots to fund them so it only works wheb everythibg stays still

@Test Of Time Gaming bro, what you saying

@@maxwellthesillycat damn i guess four or five typos makes something incomprehensible three of them are even the same type of typo

@@testoftimegaming5219 i mean this minigame also uses moving icons and it seems to be relatively fine with them here

@@testoftimegaming5219 well I don't know actual coding but I feel like a polished "if pink on X,Y coordinates (that don't include the goal area) is detected, click then drag to x,y coordinates (that correspond to the goal)" and then same with gray I guess(?)

Oh. You think 100 is a lot? On sm64 ds I got over 2000 once

@@FiereTheCatBoy Man, we’re talking about a fuldging AI

@@Redinho1 Hunter is a computer speaking to us

@Redino I can count at least 100 times this ai messed up. I only messed up once in a group of 50 levels or so. The only reason I didn't keep going was I played so long my ds died. Also I was bored as hell

@@FiereTheCatBoy That's what a computer would say

Best of luck with programming, and I hope your arm recovers quickly. Take care!

@@BakaTaco Thank you! 😁

Mainly because it is too slow to click it on time, not an issue of finding it

I personally just think it is an excuse to not have to work on it until it's good. or maybe I'm just a perfectionist

I didn't mean as in its better for it, only more entertaining which in the bots case is its main purpose. I don't think he's trying to build a bot that has purpose of playing the game efficiently for research, more make it do something novel to make a video about. Thus it works perfectly since the goal is not to be perfect but to work for the novelty of a bot that can play this game Also when I refer to it making a mistake I don't mean not clicking instantly, I mean actual mistakes of clicking the incorrect icon.

That's the entire point of machine learning though.....

@@thechugg4372 he wasn't using machine learning. Tbh idk if using terms like AI are correct but in reality it doesn't matter that much

They where also in mario 64 ds

Yeahhhh. I remember just vibing in Luigis casino all damn evening

@@jimmyjohnjoejr he wasn’t asking for somebody to give him a way of playing the game. If he played it, I’m pretty sure he’s got the ds to play it again somewhere

@@beans3669 🤓

Daaaamn, I remember destroying my little brother in the bob-omb tennis minigame, it was so much fun!!

You would lose since a lot of the time their hats are completely covered and you can only see their face so you need to match other parts of their face.

Maybe one could add the colour purple for Wario's nose.

well also their chins (when cut off) and mustaches (when theyre covered)

Probably, doubt they used a hard RNG generator. Might just have to use the time converted by same RNG generator that they used, or something similar. (Totally not human like, but who is counting, when using super processing to play match the face)

You don't need to manipulate the RNG, the odds are in your favor as the player anyway.

Don't tell RTGames about that possibility.

I have 8000 coins on that minigame…

Someone ready did It without AI believe It or not, toad , YES, THAT TOAD WITH THE DRINKS is the culprit of It all, look for beating Luigi poker using toad or something like that

SM64DS had this same minigame. I managed to get to 105 before I decided to end it there.

I got 201 and thought I was good man u were insane

Once you get in a rhythm, you have to be the one to stop yourself lol I got to at least 1000 before making myself stop

I got to +1000 and just stopped because I was bored

​@@dile3725 exactly Same with me

I think part of the problem is, that the icon are partially covered. I wonder if the game gives priority to the top most icon or the wanted Icon in case of overlap.

​@@sarowie I played this game a lot when I was a kid and replayed it recently. I'm pretty sure that it prioritize the top most icon so you might be onto something here. Although, I didn't check the code of the game so I can't be sure.

@@SweetSaucer This program doesn't utilize any neural net, so since there is no ML being performed it doesn't qualify as an A.I What I mean by "prediction" is just taking the difference between two integer points, really not that complex. There may be an even simpler implementation but you saying I'm "overthinking" it is strange to me since my solution is really only 3 steps which can be converted to three python functions. Extra screenshots are not needed for determining trajectory.

@@SweetSaucer ML/AI is all about prediction, that's why I gave you the pretext when I went on to misuse "prediction", a synonym could have been "determine" but I agree with you, prediction is just easier to use and common vernacular Every time a screenshot is taken you can just compare the coordinates, that's what I mean. Pic at time 5 and pic at time 7 will have different the location of the face at different pixel coordinates, no extra screenshots are needed. OpenCV can actually do a very good job at locating the faces despite the erratic movements since it's written in C/C++, it can definitely perform the task in the vid

@@SweetSaucer You can give your perspective, I don't understand what made you so angry with me

Except for on ones like the example at 4:34 . If I remember this minigame right that is a static one so the hats never show. There are probably also some overlapping ones that don't move that will also completely obscure the hat.

@@justin66asdf I believe the idea would be to automatically click only if the appropriate red or yellow is present (hopefully within a single frame). If not, then you fall back to matching the image.

@@ZipplyZane I was just coming to comment the same thing

But sometimes their hats is completely obstructed.

@@Mabra51 And? A one-size-fits-all solution is generally unoptimized

Finally someone that mentions SM64DS!

oh crossplay multiplayer like that is a thing??? makes me wanna test it out for myself

Isn’t that just a TAS?

@@thebananaspeedruns9275 A TAS is a human playing the game, but putting in each and every one of their inputs frame-by-frame & via a program, rather than real-time & via a controller. That's why it's called a _Tool-ASsisted_ speedrun!

@@overthinkit_ There are TAS that have real time elements to them. Im willing to bet this AI could count as one if the minigame was timed and had a leaderboard. Also it Tool-Assisted-Speedrun. The last S is for speedrun, not part of the word Assisted

@@overthinkit_ oh hey it’s the guy from quinbobin xD

For that you either need and hyper advanced ai able to recognize paterns types of colisions, enemies, etc... or just mess with the game code and somehow make it send specific signals to an external program

I mean, that is essentially what the 'sub icons' thing was doing. Except instead of a single pixel (which could be covered up), dividing it into groups of distinct characteristics that are less likely to be so.

Counterpoint: Chins.

@@sirsins9598 then have the chins as a final resort, and the colour scheme take place between the 2 since it's more reliable

Colours only helps for 2/4 options though, so it isn't an efficient solution. And as mentioned you may not see that speck of red/yellow if the layout is unhelpful

sho pfp !!!

Use a few frames to test for motion then wait for the loop to come back and match.

That is because you sre looking for red pixels for Mario. I would guess you can also find Wario faster, while performing similarly to the algorithm when it comes to Luigi and Yoshi.

@@andresmartinezramos7513 Mario and warios faces are differently shaped to the point of being notable

It only updates 2 times per second so if the character is behind another it doesn't locate it on that frame. It also misclicks occasionally because it isn't always fast enough.

Not only that, when the movement happens, the Wanted Character isn't always at the front. Any time you click on the wanted character, when an unwanted character is in the foreground on the exact space you clicked, it will think you clicked the unwanted character. I lost a 60 odd run that way about 10 years ago.

@@CodeNoodles would it be possible to make it update faster?

@@McTuber42 More optimized code, use c++, or put everything in a c/c++ library that is performance intensive and call it from python

@@CodeNoodles Makes sense. I'm an extremely beginner programmer so I'm trying my best to learn

If you use glasses, try doing it without them. It's near impossible lmao

@@grylltheonion I don’t wear glasses

Heh heh

Hi

if the colours of their respective hats where blocked by another character, there would be no coloured distinction that the AI could detect. Your strategy might work for the first few stages, but once the screen is filled there is a higher probability that distinctive factors are lost.

@@omicra6874 Good point, I didn't think about that.

It's definitely an AI title, but it's not an AI video unfortunately. Still entertaining tho!

Nvm they changed the title, now it doesn't say AI in the title anymore.

I made no sense until I watched but now I understand what you are talking about. Like if it looks at the hat colour and it matches Mario, Luigi, warrior, wa Luigi or yoshis eyes then skip the top screen search and just look at the bottom screen for that image. Another improvement would be to wait for the bottom screen to not be completely black, if it does not already do that

@@memetech- yeah the completely black screen thing is a hurdle but after you tap the right character the timing should be perfectly consistent of when the character shows up i think

I think I see what your saying. If it looks for the hat color though (excluding Yoshi) what if the hat isn't visible on the bottom screen?

@@CodeNoodles it’s always visible on the top screen. You see the colour, match it to an image in the expansive database of a few Mario characters, and continue where this already does

@@CodeNoodles i'm talking exclusively about the image recognition of the top screen. it definitely isn't the brunt of the image recognition loading time though lol

Who got first most of the time?

They didn't call it an ai tho

@@characookie241 Oh they changed the title. When I wrote my comment the title was something like "AI learns to find Mario"

Vscode

Code Pasta is a good nickname 😆

Lol