Teaching a computer to strafe jump in Quake with reinforcement learning

Подписаться 15 тыс.

Просмотров 44 тыс.

50% 1

In this video, I use RLLib to train a reinforcement learning agent to strafe jump in the classic video game Quake
Chapters
00:00 Introduction
01:11 What is Reinforcement Learning?
02:42 Quake RL environment
04:41 Results
08:22 Comparison with human WR
09:54 Future direction

Игры

Опубликовано:

21 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 59

@Twisted_Logic 4 года назад

Using low difficulty defrag maps might let you focus more on the movement than dealing with enemies and figuring out how to press buttons

@wowie_allie7404 4 года назад

i am so pleased to have found your channel through this video; please keep it up! this is my favorite content i've absorbed recently, as a long time speedrunner of quake-based games, its really wonderful to get raw insight down to basic physics and code to understand why im doing the actions i'm doing

@ratzinstagib 4 года назад

This video is amazing! Your creativity and practicality with the production and visuals is so helpful for viewer comprehension. For example, I had never understood powerbunnying before other than as just a strange engine quirk that made quakeguy go faster, but getting to see the graph displaying the vectors frame by frame totally illuminated the reason this tactic works and allows players to optimize their runs. But of course, the real treat-watching the software complete a quicker 100m demo than crashfort, one of quake/goldsrc/source engine's most well respected and skilled movement players despite some limitations on keypresses and mousespeed. Truly awesome.

@IM1THE2KING3 4 года назад

genuinely hope your channel grows

@AlejandroForeroCuervo 4 года назад

This was very interesting to watch, thank you for sharing it!

@HueyTheDoctor 3 года назад

Your Quake videos are excellent. Many thanks for the informative content.

@unity6926 3 года назад

Incredible channel, just finding it now. Subbed!

@imforpeace9511 3 года назад

The video is oddly enough, very well composed and polished. Subscribed, please provide more content XD

@cattie6554 4 года назад

Fantastic video Matt! Would love to see how far you can take this and really push the engines limits.

@Krunklehorn 4 года назад

Wonderful suggestion about the potential for reinforcement learning to find new techniques! I'm stoked!

@happysedits 4 года назад

This is nuts dude, good job, I love it

@ElguSpeedruns 4 года назад

Amazing video. Super detailed information!

@gerardrey8120 4 года назад

Great job man! I hope to see more

@dannybwai1136 4 года назад

thank you for your work, great job.

@tyzorg 3 года назад

I LIKE THE PART WHERE HE JUMPS

@notgate2624 4 года назад

Awesome video. I wish you had shown how fast it could go without the restrictions.

@doot2359 4 года назад

We want more of these stuff !

@Samopal.VanoZz 2 года назад

I studied ai at university and played a lot of qw, so i find your work very intresting. I have had been writing a bot, but my enthusiasm just exhausted after 4 months of work. But i hardly thought of making the bot move through ai and just took moves from pro-players' demos ( i had 16gb of them). I wanted to use some ai for more common behaviour such as attack or chase, or retreat..camp etc..

@sinus4784 3 года назад

This is sick you should really upload more this stuff is really cool definitely a subscribe from me

@FreedSeed 4 года назад

a new matt video? on easter?! my year is complete

@pynnegames 4 года назад

Amazing work

@EkajArmstro 4 года назад

This is awesome!

@QuakeWorldTeamFortress Год назад

nice video!

@matthewpaquette 4 года назад

This is super neat. I searched “ai bhopping” and found your video. Really enjoyed it. Do you think this is could be repurposed with other games that allow bhopping / strife jumping?

@GeckoEidechse 4 года назад

Any chance you'll make your code publicly available? ^^

@Trapezoideris 4 года назад

Yes please

@MattsRamblings 4 года назад

I've now cleaned up my code and put it here: github.com/matthewearl/q1physrl

@goliathjohnson966 2 года назад

@@MattsRamblings Matt, first of all I'd like to say, I hope you are doing great and have a Merry Christmas and Happy New Year! I have a few questions. 1. Do you think it would be possible to create an AI- bot with reinforcement learning for *Quake Live* Duel? 2. If so, would it be possible to write the code to limit it to only shoot around a certain % of LG and machine gun? Any gun with track aiming. Obviously, it wouldn't be too fair if the AI bot could shoot 100 LG or completely have lock on aim, so it would be interesting to see how it would learn to take fights and learn how to get frags if it could only shoot 30 percent lg or up to 40 or 50. Personally, I think the lower the track aiming accuracy the AI bot would have would be way more interesting to see because it could teach you combat techniques that could work for ordinary humans since only the top .1 % of players were able to shoot super high 40 to 50% LG during duels. If you could create an AI bot like this I would love to buy or rent it off of you so I could plug it into my servers and play it to sharpen my game in duel, because right now I'm only around 2100 elo and would like to get to around 2500, 2600, elo if possible and be at a higher level.

@user-ds1pu2vc8s 3 года назад

this is totally insanity I like it

@danielporto6307 4 года назад

Hello ! Impressive work. I was wondering which algorithm you are using ? Is it ppo2 with baselines and did you tweek the hyperparameters ? Also, which policy and how many timesteps it took ? Which python version and others (gym, tensorflow, baselines/stable-baselines, etc). I would realy like to know more ! :)

@dorian7259 4 года назад

your voice is very relaxing

@btarg1 Год назад

I would love to see this ML approach applied to a full run of Quake or Half Life!

@skope2055 Год назад

awesome

@puilp0502 4 года назад

Interesting project! What kind of visualization library did you use for the diagram in 6:10? It doesn't look like the standard matplotlib.

@N00BT00BX 3 года назад

Cool

@DruidVorse 4 года назад

I wonder how this can discover new tech in AM2R

@realwizardry834 4 года назад

Content like this is fascinating because regular developers created a video game and people are learning how to get the best results out of the rules that the developers have written. Humans working within other humans' expressed boundaries.

@gendalfgray7889 3 года назад

Is it necessary to use learning and stuff? SJ consist of combination of buttons and mouse. Program one strafe jump, program how to chain them together, wouldnt it be easier?

@xIncrement 4 года назад

This is super interesting! I'd love to see more videos like this, being a developer myself. Do you have a recommendation for learning this lib? Also, alternative titel: "Computer teaches Quake novice to strafe jump" lol

@RockZahdok 4 года назад

Check out gym.openai.com/docs/ , if you are interested in the theory behind it search for "RL Course by David Silver"

@magyararon6918 3 года назад

Its crazy that the human wr is the almost the same as RL.

@YaLTeRz 4 года назад

Interesting. You've said it would be interesting to see how a hand-crafted agent would compare; I've got a video of the theoretically perfect completion of 100m here: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-rejtyHCf120.html No demos on that video unfortunately, but I can ask a friend of mine (@Jukspa) to make them using his version of the tools: github.com/Jukspa/TASQuake/

@jimjamz. 11 месяцев назад

Can you post a link to the track map?

@jimjamz. 11 месяцев назад

speeddemosarchive _d0tt _c0mm_ _slash_ quake _slash_ maps _slash_ 100m _d0tt_ zip

@SL4Y3R_ 3 года назад

The link for downloading this map?

@jimjamz. 11 месяцев назад

speeddemosarchive _d0tt_ _c0mm_ _slash_ quake _slash_ maps _slash_ 100m _d0tt_ zip

@goldnutter412 Год назад

Got someone on Q2DM8 once.. bunnyhopping from bottom stairs, through the gap and into the tunnel under the dropdown with megahealth.. As I'm passing through the gap between boxes, I hear someone spawn to the top right.. kept bouncing into the megahealth tunnel and flicked up, behind right, about.. there.. and back to the bouncing.. guessing he just saw railgun come out my ass as I was going past 🤣🤣🤣🤣🤣🤣🤣🤣🤣

@webgpu Год назад

Matt!!! Please pleeeeeeeeeease.. There's this immensely addictive web-game i've being playing called "agario". its simple: just keys w and spacebar + mouse move. it's a super simple and at the same time super complex game: you start as a little blob and you need to eat pellets spread around the field in order to grow, and eat smaller blobs to keep growing, while running away from bigger blobs which are always faster than smaller ones, and finally there are Viruses that are fixed blobs that explode you if you touch them, and you can also shoot pellets (using W) on those virus in the direction of other players so they explode in smaller pieces that you can eat. Players have invented multiple complex movements along the past many years. Could you implement an AI agent to play this game?? (there are already many "bots" in this game, but they are all dumb)