Dominating an Online Game with Object Detection Using OpenCV - Template Matching. 

Great video, love to see more like this. Automating games like this is an amazing way to introduce people to OpenCV, it's simple to explain but teaches you important skills applicable to a much wider range of tasks. I'm glad I randomly stumbled upon your channel, your videos don't get nearly as many views as it deserves for such high quality educational content.

Amazing work! Thank you so much for sharing this! I would love to see a video on how to implement convolutional neural networks to do object recognition.

To circumvent the missed image detection from a day/night cycle, you can add a desaturation to both your sample and comparison images to grayscale them before comparison. This usually works for pixel-based art styles, as day/night is usually just an added color filter over the original image. The exception being those games that use specular or bloom lighting effects, which can distort the end-result grayscale in the image. The trade off, however, is that the added step of desaturating the sample image on each frame will slow down the script a bit. EDIT: This method also wouldn't work in games that reuse sprites with different color palettes, for obvious reasons.

This is a really well paced and detailed guide! Great work :)

Great tutorial, really enjoy and appreciate you explain what each line does in detail. Subscribed and liked!

Your videos are awesome man! I really love them :D

3:34 "now my hope is that we detect multiple instances of weed" yes pls!

Thank for the examples and how it all works, liked and subbed.

This is an awesome, very straightforward tutorial. Thank you!

Ah thank you so much, i was looking for a tutorial like this for a very long time :) I did something similiar in auto it just for fun, but I always wanted do something like that in a "real" programming language with more powerful libraries. Thank you so much =) I only need to learn how to use it in 3d games /video capturing. But it is just fine to learn right from the start :)

Liked and subscribed - explained and shown structured and clearly! Thanks!

I love to see you doing these videos! I’d like to see one that is based on real pictures, like “is the car pointed left or right” etc and incorporate some training and AI

You could have also done the grouping by comparing the distances of all rectangle centroids which would essentially create a node graph with every center connected to everyother center and then combining all nodes whose distance is below some threshold distance. Your solution works, I just think it’s good practice to use more general algorithms.

15:15 A more efficient solution here would be to just check for rectangles with intersecting regions. Not sure about python but in c# this is very simple with the Rect.IntersectsWith method (you can also just do it manually, the math is easy). Then you can combine them by inflating one rect with the difference from the other. Also for more reliable matching, there are various imaging algorithms you can use, for my OCR projects I do: grayscale > scaling > binarization > inversion > denoise > bordering, this helps a lot for matching foreground objects over varying backgrounds. You can also start with colour averaging which can make scanning for objects significantly faster at higher resolutions because you can scan chunks of the image and skip processing any that aren't similar. I've used these methods for overlaying real-time translation of text in non-english games.

thanks man. I love your teaching style

haha this is awesome, i've thought about doing it before and after this i might have to. One of the most relatable/interesting coding videos lmao.

Thank you man, this is awesome !!

Oh, come on.. I just decided "Ah crap, just get started..(!) Ima start with the basic webstack and see from there. And then I stumbled over this channel and THIS video.. Great video @ClarityCoders!

this is what I have been looking for... THANK YOUUUU

Very fun to watch and learned a lot. Thankyou.

Couldn't be more clear. Many thanks...

Ty very much, i could not find someone explain this as well as you did ngl saved me

Awesome video! Learned a lot!

Teaching archivement unlocked! congrats man! you are helping people from brazil!

Thank you for making this video, it really helps me a lot.

best tuto about template match i have ever seen！helpful！

I love the tutorial; keep up the great work! Heads up, I am coming for your high score ;)

Wow, this is so cool! Thanks for sharing!

amazing. thanks. so much easy class from you

Nice; Solid introductory video! I enjoyed watching. Wish there was a fast & efficient template matching method that worked with a high dimensional image space. I imagine the game you tested in the vid was only 800x600 or something lol. When working with large images I usually have to search a small section or it will take too long to compute.

Thanks man, you saved my graduation project!

Pretty nice! Found this on my recommended. Subbed :)

Dude, this is so genius. Thx!

Here's an idea to get around the different shades thing: transform all target images to black and white, and same with the recording output going into the program.

Very useful, thank you for sharing!

Great video, very helpful

Pretty clear and very interesting, coming from C and learning Python and opencv for industrial purpose, you helped me a lot , template matching can be used in so many way in my work. you have all my respect and i'll follow you with great joy. Thanks again for your videos.

great job man :D

Thank You for this valuable video

woah you jump from 2k subs to 7k in 5 days, congrats

Great video thank you!

This is a huge inspiration.

Amazing video!

blessed by youtube algo.. found your video and subscribed!

In The setup thank you so much!

this seems really really cool; I want to do that on forge of empires . I'll watch this at some point

ty cuz ive been having a hard ti getting started.

Great tuto!

Great Video!

This is really cool!

Really cool!

Dame! Great job!

very interesting, subscribed :)

Very good video!

I'm loving your content! As a fellow ML engineer, and fan on mini online games, I love this!

👏 Thank for this

ClarityCoders -very very awesome

OMG, thank you!!!!

thanks learned a lot :)

Love this

For the multiple boxes issue, could you lower the threshold till at least 2 boxes appear around the wheat, then remove any 1 box items which should be anything that isn't wheat, and continue from there?

Very good video, can you combine your idea into the online game of chess, and use OPENCV to identify the FEN position code of chess on the web page?

Here after watching CS Dojo's video :)

This is awesome thanks for the tutorial! I just wonder how to implement the code with 5 multiple window (open at the same time)

This is a great tutorial - thanks! Could you also do one explaining the kick ya chop script you have in your git hub? Also I tried to use the kick ya chop script but had to increase the sleep time to around 0.2 otherwise it wouldn't work. This is even after I tried reducing the region to search in. What gives? Do I need a faster computer or something?

Thank you.

@claritycoders Thanks fo the videos! May you guys comment on how do games (their developers) catch such bots? Asking for the ways around obviously)

Amezing tutorial, subscrubed!

You can also find the wheat points without overlaps by running a local maxima filter on the grayscale image- accomplished in OpenCV with a dilate filter followed by a compare filter. You might not need it for this use-case, but you can also additionally filter out plateaus of local maximum value with an erode and a compare (if you want to know what to compare against what, for the sake of brevity here either google it or just try a few combinations; heck, you don’t need Jupyter to figure out the correct operations; you can prototype this kind of stuff with Photoshop/Gimp layers easily). I like image-processing solutions like this because 1. For an end-use you can more easily run the whole thing on a GPU where operations over a whole image are cheap and reducing that an array of numerical results is the expensive part, and 2. You can visually see and tune/debug the dilated image, the dilated-compared image, the dilated-compared-eroded image, and the dilated-compared-eroded-compared image.

Nice video but one question. Did you purposefully excactly double their highscore at the end or was that coincidence?

great video man, terima kasih

Teacher, first congratulations for the channel. I need your help, I have a simple image with a range of 5 placements where the quantity is identified, how do I identify the biggest color type red and say where the X,Y is, and if you can help me, and only for create an alert.

Man that's awesome

expected to see dominating that online farming game

Legends say the guy is still punching.

getting game information from memory: 😴😴🤢🤬🤬 getting game information using opencv: 😎😎🤑🤑🥶🥶

great tut. btw, what is the screen capture software used to make the vid?

thank you

you are wonderful

Jupyter us perfect for a live demonstration 👌

Thank you Mac Miller.

Thank you! Very great guide to OpenCV! But how to optimize it to get it to run 100 times faster? How to use multiple CPU cores and GPU with OpenCV?

young Man!! you have my like button! =)) Ridiculous

Impressive!!!

Oh, this might solve a problem I have. There’s some made up alphabet which looks like it was made with a fixed typeface, and I want to transcribe the text in the images into actual text in order to try to decipher it, and I think I know how to extract the characters, but I still need to find how to detect which character they are. I can’t use standard OCR because these are a made up alphabet in an Easter egg for a game.

thanks🤩

This is AWESOME and it has been extremely helpful to me. There is one thing I'm trying to figure out. Let's say I grab an image and I know I want to look for one "needle" in the lower half of the image and a second in the upper half of the image and I want to know the relationship of the matches to each other. Now one way I could do this is grab two different regions two images. Do the matches, and then use some math to figure out their relationship to each other. But, what would be really convenient is if I can just look for the matches I need to in the lower half the image and get, multiple matches and then look for the other needle in the upper half and get multiple matches and already have the x and y coordinates for the overall image without having to translate them. If it can't be done, it's easy enough to add y offset to the coordinates I get out. But I was wondering.

Amazing tutorial although I do have some ideas to improve some of what was presented, Firstly the rectangles grouping section, I'm sure there's a better way than just duplicating the list, maybe manually comparing the x, y of rectangles and if they are similar to another rectangle we don't use it The other point would be about beating the highscore, i would test for a single pixel's color above the player on each side where the background is static and check if it's color changes (a branch is above) and move accordingly

Hi brother is it possible for you to make a chess bot with opencv as u make for other games or for knife hit game if yes please make a video on it .It will be really helpful .

I really like Spyder IDE.

Holy shit I need to do this.

Could this tecnique work to detect parts of a document (for example a rectangle with a text) in order to read what's inside that section of the document? It would be a scanned document so it could arrive with horizontal lines not perfectly aligned due the subtle movements of the paper durinig the passage under the scanner.

Is it possible to move the mouse to the location of the identified object on the screen instead of a pre-set point?

💯 th like

that small insect walking across his screen got me

great video! could you make something like this for a pokemon game walking in a grass and leveling up?

TNice tutorials video is a great introduction to soft soft for beginners. I would Nice tutorialghly recomnd tNice tutorials tutorial to anyone starting out with soft soft.

Jesus!!!!! Love it

Hey bro, you can google mines game usually available in rummy apps. How to create mines game bot? To make all cells visible by seeing all hidden covered mines inside cells/grid or uncover then through any method or using OpenCv(image recognition) algorithm method to identify exact mines in the cells/grid? Can you make that tutorial?

Hi i am beginner, i am preparing script for a similar game but the question is can i run script like this for multiple game windows lets say arranged in 3x3 matirx (just like tic tac toe) on desktop and would it work or do i need to implement something more?