The defense of my PhD thesis on April 11, 2024: a summary of all of my work on Neural MMO from 0.x to 2.0. Advised by Phillip Isola, additional committee Eugene Vinitsky & Pulkit Agrawal. Dedicated to my parents Jose and Patricia Suarez.
Oh cool - didn't expect anyone to see this here! There's a Neural MMO discord. You can chat with me here: discord.com/invite/BkMmFUC For my new project on bringing these advancements to all of RL, you can star github.com/pufferai/pufferlib or come chat at x.com/jsuarez5341 and discord.gg/puffer. Lots of announcements soon!
I can not imagine sitting ina meeting for that long and not double checking if I was on mute or not at least 5 times. Just sitting their not caring at all is the level of not giving any fucks that I want to get to. Still rude tho
The intersection of neural networks and MMOs have always been something I wanted to pursue but there wasn't anything quite like what I had in mind. The closest one to this was OpenAIs work in Dota 2 which after seeing it, made me pursue a specialization in artificial intelligence during my undergrad. After seeing this (by some stroke of fate from the youtube algorithm), I feel like this is the direction I want to go for my graduate studies. Thanks for pushing forward innovations in this space. Congratulations on your defense as well.
I've thought about ECS or EC style approaches but couldn't think of anything like you did! Not as obvious as you might think, requires good interdisciplinary knowledge and creativity. I think you're going to have an incredible career. Congrats on the PhD.
@@alexthemle Thanks! One of the things that helped me realize this was actually using Unity's ECS to write an ultra high-perf renderer for NMMO early in development
@@neuralmmo I'm working on my physics PhD and started self-learning AI out of passion. In particular, I'm really into AI for video games. I've been wanting to do a very similar thing with my career. Maybe we'll cross paths in the future :)
@@nilsolsen8727 I'm guessing you mean story games guided by LLMs? If so, I know a group working on a similar project now and that others are integrating this idea. I think it's the future of story games for sure. Not my area of interest though. I tend to like AI focused on optimizing gameplay. Working on a Pokemon Blue AI right now, for example. Just to see how it learns and tries to optimize the game.
This is the first PhD defense I've ever watched and I have no deep understanding of these topics but I watched the whole thing! Excited for you, rooting for you and Neural MMO!
There was a lot of stuff to handle with a/v... room wasn't quite set up right. I didn't want to hard mute everyone since one of my committee was on Zoom, and I forgot that Zoom switches focus like that
@@neuralmmo if you could make the video just freeze for the frames when ari if interfering with the feed, it would make a much better viewing experiance.
Its nice to see someone did this. as a undergrad student who is really eager in the topic, i would love to have some motivation to pursue a life aroud these topic. But it is hard to do something when u dont have anyone to talk to or lead to in the area. this video gave me some motivation. thank you Joseph.
You don't need people to tell you how to do things. You can just do them. Early years of NMMO were me building stuff in very stupid ways as an undergrad because I didn't know any better. But I don't think any mentors could have made me learn any quicker than just working hard and failing often
@@HesitantOne You never stop feeling lost. You just get to the point where you know more about a very specific topic than anyone else, so at the least, you're the best qualified person to possibly do the work
I’m also a Joseph, nice to see another be recommended by the YT algorithm. I also wrote my paper in the realm of reinforcement learning. This was an interesting watch :)
I would not have done this if I thought it would be anything like cs coursework. Start doing a ton of small side projects. Do stuff you can actually finish in a few weeks, not several months
I don't really leave comments on videos, but I had to leave a like and a sub for such a genius way to train models. This was a great video of the practical usage of video games as a tool, not just entertainment. This reminded me of the anime SAO (Sword Art Online) Alicization - Season 3, where the main character was transported to a world where the government was working on training (AI) = Agents in a world based on real life where there was an economy, combat, skills and this video reminded me of the anime and made me happy to see people use something that was not meant as a tool but for entertainment and that's what made me fall in love with learning and trying to make my own neural networks and AI. Such a great video, and I wish you nothing but the best of luck with the project. I can't wait to see what new updates come to the Neural MMO!!!
Damn! This is absolutely awesome! I've been writing accelerated agent sims since I was 14 with the intention to create something like this one day. This video is one for the history books, great defense!
@@neuralmmo I briefly looked at your code and knowing scientists and post grads I'm stunned. The years have paid off and your earnest passion shows. Looking forward to testing it out more in depth, incredible work again, I hope your flame burns evermore brightly.
I'm not in academia, but is this guy above average in terms of speakers? I get this is his defense, but I'm just really impressed. Well, I guess at this point in his career he really would already be an expert, he just looks really young so it's kind of throwing me for a loop. I'm studying ML as a hobby, never looked into RL at all. The first few minutes of this talk alone captivated me and made me decide im going to look into RL.
This project throw me back to that story about a Quake server where the bots allegedly "learn" and get better at the game as the server went on, eventually they were said to have intellectually agreed that the best strategy is to just stand still and stop playing the game. Putting the uncanny/doubtable accuracy of the story, it great to see our advancements are made from conceptualizing the human conscious. I don't fully understand the majority of the technical aspect (still a dumb undergrad with 0 experience lol), but the ideas you come up with are incredibly intuitive and fascinating. TL;DR: You are inspiring, and congratulations!
Had this same concept cooking in my head since ChatGPT day 1, but I’m broke and my MacBook died while I was in a Warsong Gulch! What a marvel to see it outside of my fantasies. Great work!
A really well done presentation! I've been exploring a RL application for complex multi agent environments, but have ran into many similar type hurdles as you outline here (with far fewer clever solutions). Specifically, i see a use in industrial/operations applications which maybe more traditionally fall under the operations research umbrella. Your presentation reignites that fire and gives me additional resources to explore!
I have made DSS to help me to choose which units most important to upgrade based on some criteria but my lecturer/teacher didn't like it. Watch your great job make me wonder to continue my works or not.
What an incredible topic, work, and defense. You did a wonderful job in a field I love watching evolve. I hope you are not done yet! Subscribing to follow your future work.
Great find from the algo! As a a uni student I definitely need to check out your papers now. I wonder how this would work if you add a communication system to the agents by hooking up llms as a brain (for communication and task assignment), like in the ChatEval paper.
My work intentionally does not consider LLMs in the main loop like that. RL models have millions to tens of millions of parameters. LLMs are 10s of billions. You lose all the advantage of having infinite data by not being able to consume it. We do use LLMs in niche areas, like for the task encoder, where we can precompute once
I think theres something inside us all that wants to do something like this. It's so cool. Were you inspired by carykh's youtube videos from 2017 that play around with this? They look pretty similar to your original setup. I tried to reimplement his stuff on my own since i was a freshman in college and it was the first nn evolutionary program id seen, but i got a bit out of my depth and gave up lol
no hadn't seen it actually. First got the vague idea for something like this in 2014 or 15... tried again in 2015, and then again in 2017. By then, I knew enough to get it to stick
oh, no offence, this is really a good job and presentation actually. It makes sense to me someday Open World Games would have more vivid NPCs (game agents) as proposed, we will see and anticipate, gamers' blessing
@@tkingless That's definitely going to happen and is one of the more fun applications. I have high hopes that RL is going to play a much larger role in AI long term
Hey ! I just discovered your channel and in fact, your entire project and I'm really happy to see someone actually doing what has been proding around in my mind for a long, long time (except I had neither the means nor the knowledge or really the motivation to achieve what you did, what you've accomplished here is highly impressive). I'm a professional game developer, currently working off of Unreal Engine. Seeing this, I get very inspired about everything of course but I'm thinking on whether creating a more, shall we say, "proper" representation layer for an ulterior version would have any value. It would also be an opportunity to bring the MMO's game mechanics into the hands of Human players (you might think this is too simple and primitive to get engagement from human players but just the ability to play against advanced AI agents would be quite something already I think), which might also have some value for training AI, perhaps by attempting to "snapshot" human actions into policies and using that as some sort of training dataset / start point ? Anyway, I'm barely even a layman in all this so I might be saying complete nonsense. But I figured I'd ask, I really want to toy around with machine learning aswell one day, just haven't reached this stage in my life / education yet :)
Love this. I just missed the huge AI boom when I went to uni. We did things such as image recognition with CNNs, some audio recognition with RNNs and some SNN for classifying directional movement etc. Would love to make a similar project. Maybe I will come back to academia to get a PHD but unfortunately I doubt it.
Seeing this in the future coupled with Agent base attributes (like intelligence, strength, dexterity etc.) And then see the interactions between agents really excites me! For example, we have a blacksmith character and a Librarian. Maybe the blacksmith is trying to teach the librarian how to forge a weapon, but the librarians Strength and Dexterity attributes are too low or dont meet the requirement so they fail to learn it in an ample time, match that with a mood or frustration of some sort and they might give up learning it all together. Im excited to see how this blossoms!!
i am currently doing my masterthesis in architecture on the topic of the interaction between digital emergent enviroments and the interactions of avatars, i stumpled on your video through coincidence but its amazing what kind of work you did and what is currently being researched on in the direction of digital enviroments.
layman suggestion: I wonder if adding a "perception" attribute to an agent could improve performance by simply giving lower perception agents less data, as well as rewarding those that train the attribute?
Can you pls upload an updated version where the entire slides are visible? Not prerecorded but just edited on top. If this is to much afford pls link them in the description so it's easier to follow along
Id love to see an update of human vs model improvements, where humans can take an active role in the simulation. I have a feeling that your agents would see an exponential increase in productivity and faster optimal behavior onset. perhaps decode to a graphical representation on a small scale and then squash the humans behavior patterns into weights and use them as training agents against the more randomized approaches. it seems like powerful kindling. either way, great oration and a very cool topic! hope to see you more in the algo
