I tried to make a Valorant AI using computer vision

Подписаться 14 тыс.

Просмотров 1,5 млн

50% 1

I went down a rabbit-hole of trying to make a Python program that can play Valorant using computer vision and some radio shenanigans.
More details, errata, etc.: www.riveducha....
Radio dongle: CrazyRadio PA amzn.to/3GDzEhb
This video intentionally doesn't go into too much technical detail - not sure if that's something people want or not. I tried to present enough so that you can at understand what this bot can and can't do, and also understand some of the problems it's having. And if you don't play Valorant, hopefully the premise is understandable - shoot the bad guys.
If you're worried about this being a hack, you can rest easy. It's not like a wall hack where it looks at Valorant's process memory to get information that's supposed to be secret. The bot's not at the level of advanced Valorant strategy right now, but I have lots of ideas for future development.
Software used include:
* labelImg - used for labeling the data set
* PyTorch - similar to TensorFlow
* NumPy - amazing library for working with matrices
* OpenCV - great library for doing some image processing (in conjunction with NumPy)
* Google Colab and Jupyter Lab - great for exploratory programming, especially when working with images
* PySide2 - y u conflict with torchvision dependencies??
Some people doubted that the OpenAI shell video I made was real despite the mediocre results shown, so I hope that by showing even worse results in this video more people will believe it's real.
Also, follow me on Twitter: @riveducha - / riveducha
Images:
Human Brain clip art: CC-BY 4.0 SykesOffice commons.wikime...
Music:
Corbyn Kites - Shadowing
"Inspired" Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 4.0 License
creativecommons...
NoMBe - Take Me Down to the Fashion Show
Kwon - Pluckandplay

Опубликовано:

26 сен 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 2,8 тыс.

@applesaucw7041 2 года назад

...but why? Why do any of this?

@SeanPizzaSpark 2 года назад

cuz it's interesting AF

@jazzWF 2 года назад

you're more than a year late to ask that homie

@applesaucw7041 2 года назад

@@jazzWF Its a question unbound by time

@cronin4392 2 года назад

Cause computer vision is awesome!

@gautamshetty_ 2 года назад

Because he can.

@iceaf 2 года назад

It's amazing how you can easily replicate my teammates in comp

@zem0ku605 2 года назад

lmaoo absolute gold comment. This should get pinned HAHAHA

@isaacgregoire5318 2 года назад

He should make a neon bot that sprints into the enemy's spawn with spike lmao

@chris3873 2 года назад

@@zem0ku605 what rank are u?

@shadowlp7566 2 года назад

1to1 replica

@jokermann1440 Год назад

Iron confirmed

@khalidjamonday416 3 года назад

This video made me understand why my friends call me a bot.

@reportagebykonstantinos8030 2 года назад

105 likes No comment.After 1 year let's fix it.

@roguegamingvoid4854 2 года назад

Lol

@fngaming4862 2 года назад

@@reportagebykonstantinos8030 agree

@iamHelitha_ 2 года назад

4 comments

@uriwn 2 года назад

LMAO

@real1cytv 2 года назад

Calling Valorant a csgo gamemode is the funniest and most fitting description of the game I've ever heard

@stellasoleclark5586 3 месяца назад

this randomly came in my algorithm 3 years later

@bungakrst 3 месяца назад

same

@skyaftermath2732 3 месяца назад

same

@Goodizer 3 месяца назад

Same

@Traffa99 2 месяца назад

Type shi

@darkshadowsx5949 Год назад

the AI is like a noob and a pro fighting over the controls. you're getting close.

@Darksnipa949 2 года назад

Just got this recommended. Really good work! I am impressed by the performance you can achieve with transfer learning on your "small" annotated Valorant dataset. You still remember how high the performance for different objects was on your test set (accuracy or MAP if you have computed that)? It also really hurt me not seeing your model TURN upon hearing someone behind :D Would really love to see you including audio next, and then seeing some nice 180 flicks in version 2.0.

@Hlebuw3k 2 года назад

reocmmended

@DeSinc Год назад

One thing I thought you could do is make it so a label has to show up for 150ms (roughly a standard pro reaction time) - that way it doesn't shoot at every 1 frame ghost it thinks it sees, but only at persistent threats - and also it makes it seem more realistic and human-like by having somewhat realistic reaction times. You could also have it move the aim slowly over time in trial movements until it's over the top of the marked target and only shoot once it lines up, which would not only improve reliability of the aim but make it seem even more human-like and I just realised this video was from may 2021 and you're likely not even working on this any more oh well

@garyh2y479 Год назад

Desinc on a 1 year old vid

@DeSinc Год назад

@@eHeSTaFIXtatiCkANKpiQU I know, I wanted to write it anyway so I did

@purpl3r0se31 Год назад

@@eHeSTaFIXtatiCkANKpiQU dawg

@Gone-Rogue 2 года назад

Anyone else just get recommended this video 1 year later? Great video btw

@denpoku 2 года назад

Yes

@suguru4260 2 года назад

Not me getting false banned for "3rd party program" then this guys making an ai for valorant 💀💀

@hihi-hehe 2 года назад

When i was doing my project with computer vision, I gained almost x10 performance increase just by downscaling input image by some ratio. Of course, it lowers the accuracy of results, but, sometimes full resolution is MUCH bigger than enough and downscaling isn't going to affect the results at all. So, by downscaling input images you can boost performance for free by finding the optimal level of downscaling.

@Menezeris3029 2 года назад

Why do I honestly think this bot could at least get bronze... Iron is a weird place

@nathangavenski3090 2 года назад

So, I know this video is kind of old, but I just discovered your channel and I'm watching all your videos 😅. I'm a PhD in machine learning, and I saw on the quick code that appears on the video that you are using large images. I don't think that this is necessary. You could downscale the images, use on your model, and recalculate afterwards where the BB is on the real feed. A second thing I would suggest os already use a pretrained huggingface object detection model just to see if it detects the caracteres as a person and use simple code to see the color of the border. This solution should help with the low data amount. You could even create data this way :) I don't have a solution for the spikes and mollies tough. Either way awesome video!

@johnandrewdivina1435 2 года назад

The only flaw I see is that it doesn't know to trash talk

@thewisemonke 2 года назад

This is a prime example of a RU-vidr who needs a shit ton more attention. Well done!

@nauchism672 2 года назад

Bro said he didnt share the code but somehow I see this in all of my ranked games

@pdecassan 3 года назад

I said it before and I'll say it again. These are the *best* videos on RU-vid right now. You sir are on the fast track to 2mio subs if you keep up this frequency and quality. Good luck and well done 👍

@riveducha 3 года назад

Appreciate the support!

@99batran 2 года назад

when Riot don't make bots in custom so you do it yourself

@qaugithaduck5771 2 года назад

Just go into a comp game

@99batran 2 года назад

@@qaugithaduck5771 but then I'll be the bot

@BLettuce 2 года назад

This is gold. Things I think you can do (although it's been a year so who knows what happened) is obviously have it be aware to object presence, as you have said in the video, but also respond to sounds, voice commands (via wheel ore voice chat), awareness to economy, and most importantly, have it teabag other players.

@SelfSimilarJosh Год назад

As someone getting a PhD in Machine Learning, you're doing the work of someone getting a PhD in Machine Learning.

@CrxzyYT 2 года назад

now get 8 more people and make a custom lobby so the bot can learn from actual gameplay experiences.

@andraskmeczo575 2 года назад

The lore of Terminator 7

@roryfahy4568 6 месяцев назад

@@andraskmeczo575shit actually happened in rocket league 😂

@rafay6937 2 года назад

This video is a year old and now is being recommended to everyone

@iCxtt 2 года назад

So this is what my braindead ranked teammates were using

@CaiAllinx Год назад

You need to reduce the size of the images the neural net is provided with. Go black and white and scale down the images, this will let it perform so much faster

@atpray Год назад

But also reduce the resolution.

@atpray Год назад

But also reduce the resolution.

@EpochIsEpic 2 года назад

This project really is incredible - the way the video was captured, the way inputs were sent to the game, the problem solving of getting a used dongle when the exploit was patched, all of it was wild!

@veniszin 2 года назад

why just after 1 year? youtube hello?

@ranaisrivastav363 3 года назад

THIS IS SOO GOOD. As a person just starting off with OpenCV and AI and stuff and an interest in Valorant, this is godly. I do want to see your code just to see how you used all the AI libraries and stuff purely from an academic standpoint but it makes sense why you would not want to.

@dot1723 3 года назад

I wanna see the code as well lol ive made a kinda poop bot for CSGO but it was cool please make a github with the code maybe or something

@acanadianfry2539 2 года назад

6:04 not going to lie that far friendly on the left, i thought that was sova until you pointed it out.. i guess im just an engineered AI

@Wafflecasa 2 года назад

This is why you never take down videos. They could pop off years after uploading

@nathanielepps Год назад

It's funny, I wrote computer vision bots for both PUBG and BDO using very similar tech. I followed nearly the same thought paths as you, used the same strategies / tech, and hit the same roadblocks. The part about being unable to load cudart had me dying, I know that pain. People would ask me why I bothered and I had no answer other than it was fun, so yeah I totally get this video and am glad to see someone else understands how satisfying making something like this can be, even though there is no real advantage to be gained.

@catmium7974 2 года назад

It is so painfull to see ai struggling, knowing he is just not good enought and there is nothing it can do until some human makes a better version of itself.

@hd_y 2 года назад

No what's more painful is knowing there are actual human being that plays like this, viewangle desync (aiming at the ground), doesn't use audio etc etc

@qaugithaduck5771 2 года назад

Eh, if you use one of those learning bots it can. Also I don’t know much about this stuff so lmk if what I’m assaying is incorrect

@talkingbirb2808 2 года назад

whoa, I didn't know pytorch was so hard to download year ago. Now everyone can download it

@RsbidNoodle 3 года назад

This is extremely cool, and I’m super impressed! You’ve given me the motivation to get started on a few personal projects I’ve been considering. I love stuff like this, combining hardware hacking and multiple devices and data streams - managing complexity like that and coming up with solutions for problems in that space is so much fun.

@nachobusiness4355 2 года назад

Play unrated pleaseeee. I need to hear how the ai will react to verbal abuse

@urmom-ch1bb 2 года назад

This takes "My teammate is a bot" to a whole new level

@thebeer5086 2 года назад

hi mom

@GerDrillPlug 2 года назад

hey mom

@Crustayy 2 года назад

It would be super fun to have like a league where it’s only AI you make yourself. 5v5 AI tourneys

@maxence666 2 года назад

Those kind of tournaments exist in Cs : each team has code from a specific dude who programed all the moves of the bots of his team. Very funny to watch

@ErChrys_ 2 года назад

This takes "It dosen't use headphones!!!" to a new level

@antrasit9969 2 года назад

Dude thanks for making this video. You have finally proved my point that this game has bots in ranked, started noticing it since I hit rad

@shikudaci 2 года назад

Wow youtube algorithm took its time

@agniote. 2 года назад

SAME

@TEAMKYR 2 года назад

Oh so that's how an Iron Player is created

@phi1688 2 года назад

Best iron player I’ve ever seen tbh 💀

@AnonymousFloof 5 месяцев назад

It's things like this that make me wish I had the patients to learn coding and neural networks, I'd have so much fun just experimenting and pushing the boundaries of what I could create

@GunAinmNoAodann 2 года назад

"grad student or other sweatshop labor" - lol.

@gawdahm1964 2 года назад

Just got recommended your video today randomly and loved it. I thought you were a much bigger channel, you definitely deserve more views!

@flixgribv 3 года назад

Pretty interesting video, ans it's really well made as well. And it has subtitles! Thanks!

@riveducha 3 года назад

I’m happy that somebody likes the subtitles!

@luigiesposito2481 2 года назад

@@riveducha hello friend, my name is Luigi, would you please help me?

@deto_san590 2 года назад

@@luigiesposito2481 help you in what?

@aminzekavati1803 5 месяцев назад

Bro just created an Iron 1 player. Also this bot oddly resembeled the teammates I get during my rank ups lmao.

@DreamingBlindly 5 месяцев назад

Well you never really know since there are a lot of bots out there that can roughly simulate human actions

@marcello. 5 месяцев назад

@@DreamingBlindly me when i lie

@lifeline0235 2 года назад

"this looks fun, but i might get banned if i test it in a multiplayer lobby" "ever heard of tf2?"

@oxi2118 2 года назад

RU-vid algorithm as unusual as always, glad i found this gem of a channel

@VitalisProd 2 года назад

plz keep training this AI to point it can play at at least Iron level) Waiting for part 2)

@sakana6388 2 года назад

wait, u mean irons are better than this?

@Yoshi-qz4ix 2 года назад

@@sakana6388 yeah, they are

@chaos9790 2 года назад

youtube recommended us all at once

@iceeyz21 2 года назад

@yimkumerjamz1225 2 года назад

Tru

@karim47161 7 месяцев назад

More videos like that with deeeeep technical explanations, i understand in this video so much things that i been searching about and didn't understand

@Abstract_Code 2 года назад

Honestly, I have watched this video about 4 times in the last month; because of how good it is, unfortunately there are not many good videos explaining how to train a custom data set, but your sources in the video's description helped me alot thank you for sharing this information.

@thorbn 2 года назад

This is getting recommended after 1 year lol

@SherrifOfNottingham 2 года назад

the lack of object permanence is pretty realistic for pugs honestly.

@てんちん-h7j Год назад

Very interesting video! By the way, the creator of YOLO had ceased his research to prevent the tech from being used for military applications. I hope it will not be misused.

@FazriGading 2 года назад

I am actually planning on developing a thesis with Machine Learning and AI, and your video just shows up on my feed. Incredible, you just gives me an idea! Thank you so much for that, really appreciate it. Looking back at the older games like Counter Strike 1.6 that has bots, we are totally hoping this game also have like that bot you have made. Actually it's easier to create it on the system itself, not based on Computer Vision, but anyway... this project is smaller than large companies made for robots and self-driving cars like Tesla. Don't compare yours to them, this one-man project really amazes and inspires us on the community.

@muhammadaman9010 Год назад

have a 5v5 with copies of this bot

@TarantulAnt Год назад

You should meet TacticalPumpkin

@hello-fo5yg 2 года назад

Hi! This is so awesome! Im a data scientist and I had some thoughts! There is a lot that could be done to improve the actual CV model, but I want to focus on some other stuff first: For the latency in the detection, afaik a common solution is using something like a Hungarian algorithm to match detects across frames. After you have matched your detects you can use them with a filter like a Kalman filter to model and smooth the trajectory, since you know your latency and velocity, heading etc you can push your reported position into the future as an easy way to get the bot to 'lead' the shots and compensate for detection latency. This is really convenient also, since you can remove unmatched detections, and solve issues with short term (like single frame) false positives. Also, if you lose a detection for a few frames the kalman filter will predict the expected locations based on the object kinematics which may help as well For navigation, things are a lot harder. Navigating strait from pixels is obviously really tough. I think the standard approach would be to use something like ORB SLAM to actually do the localization. If you want to get fancy, you can combine orb slam, mini map and also your key input into an extended kalman filter or something similar There are probably also hackier approaches to navigation using heuristics or dynamic window approach which might be worth looking at!

@riveducha 2 года назад

Agree with all of your suggestions! And thanks for suggesting some good ideas other than "lol python slow" :)

@riveducha 2 года назад

@Ocean Blues I think you have the right idea. One improvement could be to leave some buffer room around walls which would reduce getting stuck on walls and corners.

@cvijetkolivadic5907 2 года назад

@@riveducha I hope you turned off mouse acceleration. I don't remember if you mentioned it in the video. But I guess mouse acceleration would be something that would cause the bot to overshoot and oscilate around the target.

@farguy2615 2 года назад

bro literally made my ranked teammates

@ssam2789 Год назад

No way... this guy finally found a way to have fun in Valorant

@tmn671 Год назад

This is hard AF, I've tried computer vision before. This guy did a great job Well done

@Folded_Soup 2 года назад

I think it would be really cool to have it read chat commands from teammates stuff like"!go A" "!Defend Spike" "!defuse spike"

@TemurenYeageru69 2 месяца назад

The design is very human

@nik7069 2 года назад

My dream is literally being able to do these things. I love the video, keep it up!

@Personal-jr9rn 2 года назад

I see alot if the comments are from hours ago, somehow this old video is suddenly getting recommended to people

@kujubuo 2 года назад

if you look closely the comments are sorted by recent by default in this video

@williamjang1032 2 года назад

@@kujubuo thing is if you change to top comment it still is whithin this month

@BenisBoy14 2 года назад

an improvement method for your labelling: you can add the ability to analyze moving images (for paint-splatters) by introducing a LSTM or similar. this will also remove the false labelling of beams or shells as enemies or spikes because the data in the short-term memory makes it impossible to mistake a bullet-shell for a spike. and the data in the long-term memory might even know which friendly is holding the spike.

@declan_youtube Год назад

You should set 2 of these up and have them 1v1 eachother

@antimatter2417 2 года назад

I'm glad this vid got recommended to me. For enemy detection, you could probably use the fact that the game outlines all enemies in the same bright red color to make the job easier on yolov5.

@lweiii 2 года назад

I’m actually a CS student that is like kinda losing passion. I love playing game and all but after going through 3 years of pain automata theory and algorithm which burnt my brain with a little twist of discrete math. I don’t really see how all these things can help me in any practical way as web development job and other common field actually dont really need much theory So kinda lost. Be like why am i here… why am i suffering for no apparent reason 💀 I recently got into like data science and getting deeper with developing machine learning model. Find it quite interesting since it’s actually highly practical (we need to train data anyways so it cant be non practical) other than machine learning math course. Buttt My uni didn’t offer much unit for this field so pretty blur on what I can do other than training facial recognition models…and it kinda gets boring along the way. In the mean time, i grind much Valorant to get through uni stress THEN RU-vid RECOMMENDED THIS. Just wanna thank you alot as final sem student here and sincerely lost on what the hell i should do for my future. This project is definitely awesome and fun. It’s an inspiration to me for real. I really wanna grind more ml knowledge to be able to make project like this ahahhahaa Would love to see more owo ❤️

@thepurplpineapple9495 2 года назад

This would be super funny in an actual match

@ya1dnif 2 года назад

A full custom game with 10 of these would also be entertaining.

@gothui836 2 года назад

Still better than my comp team

@solidbhatt Год назад

Oh you are the guy who made a brim bot who writes good round my agents!

@ZebiShredz Год назад

lmao

@monkeiman6363 8 месяцев назад

so this must be what all my teamates are

@kaitotties 2 года назад

yo this is actually a really cool video and experiment. thanks for sharing your findings!

@alimustafa2682 Год назад

I would like to thank youtube for combining my interests in one video

@Mrphenixyt Месяц назад

Reasons why reyna instalockers dont speak and then dont play well- they are AI

@SebastianHaban Год назад

There is probably a way to randomly generate training data. The games assets are probably available, so rendering pngs of just character models (with alpha 0 background) at different distances and angles, placing them randomly into background shots of the game and automatically generating the outlining box where the png got placed (that you till now had to manually draw) could give you lot's of training data very quickly wich should improve the results of your AI model by a lot. Just an idea

@enderben102 8 месяцев назад

if only you could train it using a REPLAY SYSTEM

@pludoo 2 года назад

you should setup a custom game 5v5 with these bots, then spectate and see who wins!!

@kaveenchainani127 2 года назад

He’d need like 20 computers lol

@aaaaaa-hh8cq 2 года назад

@@kaveenchainani127 2v2

@MadGaming10 2 года назад

Id love to see a series on this as you keep trying to improve it, it was so much fun to watch

@linuxkerem 2 года назад

hes designed a very evil new generation aimbot and he doesn't know

@ZapWyd 2 года назад

exists already for months, ik ppl who sell those aimbots

@AlizerLeHaxor 2 года назад

its nothing new, people have been doing this since 2017

@ahmedaghadi8281 2 года назад

@@ZapWyd are those based on AI or they just somehow able to read encrypted data of valorant inside the computer and judge the position?

@kecs2 2 года назад

i think u gotta distinguish not just from enemy, but enemy head, body, and legs, so it knows where to aim to.

@adrielle1i23 2 года назад

Inb4 you literally make an aimbot

@kecs2 2 года назад

@@adrielle1i23 haha not on purpose, but it would eliminate a lot of the error -making in the process of elimination for the AI when it notices a close-up enemy or even an in-movement enemy. it would be hard for the AI to notice a head peaking enemy otherwise or somesuch.

@oblivion_2852 2 года назад

@@kecs2 I actually don't think that would make a difference. The biggest difference would be from using multiple frames instead of a single. Even humans have a hard time noticing features of a still image. But if something is moving it's much easier to see.

@kecs2 2 года назад

@@oblivion_2852 ya thats what i mean, multiple frames to highlight a different portion of the body the way valorant divides its damage multiplier: head, body, legs

@eleos5 2 года назад

Honestly I wouldn't be mad if I encountered this in a real game. Because it gets the same inputs as a human and it's not stronger than one, it seems fair enough to me.

@nashooo5903 2 года назад

said no one ever

@carterlove 2 года назад

Right now, yeah, but if this got better it could easily be better than a pro player

@anhduc0913 Год назад

@@carterlove yep. Give it a good gpu and a few month of data and you'll die instantly facing it.

@GenesisHD Год назад

it knows where it is because it knows where it isnt, by subtracting where it isnt from where it is, or, where it is from where it isnt, it finds out where it is.

@AdmiralEclipse 2 года назад

id like to see this model play against itself and learn the game see how good it can get

@Pope_ 2 года назад

Cant belive all of my temates are running this bot

@anguibok 8 месяцев назад

As a tournament organiser, having an odd number of team was bad, that why I would have want to let people creating bot participate (Not cheat, just bot, that use info that only an human can have, as you exactly did.) but, well, I doubted this exist, and your video kinda proved it T.T

@StiekemeHenk 2 года назад

You should try a lower resolution, 720p should be enough for the AI but contains about 45% the pixels of 1080p. So expect quite a boost in reaction/processing time. Maybe there's other things you can do like preprocess the image to reduce the color pallette, maybe less data will make it more accurate and less confused.

@d3v-x64 2 года назад

I worked with cv2 already but this is next level, my dude. I love this video so much and computer vision is extremely interesting. I actually consider focusing on computer vision in my future career. Anyways, thx for this awesome video and great inspiration. U are an beast

@sim789936 2 года назад

Python is extremely slow when working with linear algebra. Java and C can show performance increases of 3000x. Too bad it's a lot more difficult to get the libraries

@riveducha 2 года назад

That’s kind of true but nowhere in this is Python doing linear algebra anyways.

@danielwit5708 2 года назад

All the AI python libraries are written in C and CUDA python just call functions

@Connodogplays 2 года назад

Ah so these are my teammates in my ranked games

@mohamedeshmawy7855 2 года назад

Very interesting good job with that, for the FPS problem I recommend you try to scale down the image resolution to 608x608 (must be multiples of 32) and remove the last YOLO head from the model (responsible for long range detections and is also the most expensive in terms of computations), this will lead to less accuracy over long range targets, but will have a much better chance with close range encounters. Also if you would like, I could help you increase the detection accuracy quite more as well as some performance improvements, I am interested in this.

@Swirlys 2 года назад

Still better than most of my teamates

@Blearky Год назад

Awesome Video. Changing the border color for enemys to a bright Pink or so could probably help with lots of false Enemys. The second one is a bit more tricky you are right. But I believe this could be since the AI get trickked in a way of an illusion like the Necker cube. Since you allways start the round facing correct you'd could use that data to make a null-point and just add the inputs and afterwards delete them again to allways have the correct view. Anyway thanks for the video I look forward for more.

@videepreddy9175 2 года назад

Ah what a way to make an aimbot without saying you're making an aimbot

@theunsaltedpickle7993 2 года назад

thats some god tier crosshair placement

@candareyiz 2 года назад

no crosshair is for gods and this ai tried at least

@cochet1593 2 года назад

it would be cool to see 2 AI's 1v1 each other in valorant

@sweet_n_spicy8818 2 года назад

OMG YES PLEASEEEEEEEEEEEEE

@martinkaclik 9 месяцев назад

Getting the resolution down 2 to 3 times would massively benefit fps and save some computing power

@JulianDanzerHAL9001 Год назад

10:00 would increasing the significance of dead recconing in finding your location help it get to smalelr waypoints as it walks in the direction it thinks the waypoint is based on the lwo res location and givne that direction and a bit of time inevitably assumes it reached the small waypoint based on dead recconing?