My Computer Taught Itself to Play Minecraft 

I think a major problem with this approach is that you're giving it the same world every time. The AI is not learning to look for resources, it's learning to how to get to where the resources are in that particular map.

I think that the AI should also be rewarded by completing advancements, so it will actually progress to beat the game and not just mine all the time.

One major issue is that it seems that instead of being smarter, they just get better at beelining to the nearest chunk of high value ores by exploiting generations of discovery. If you repeat this, try varying the map every few epochs to make them more intelligent. You could also add things like the penalty for losing health or hunger.

It would have been more interesting, if each set would have been run on a different seed. With that, you could have had natural negative feedback for digging straight down to caves or lava. Also, to get to diamonds you have to dig down to below 16. It would have been interesting to see if the AI could have developed an efficient strategy to getting to diamond level fast without risking dying. Such as real players use: staircases, spiral staircases and digging two spaces at the time and teetering between them.

Ideas: Higher reward per second for higher health (i.e, +20 points per second with full health, +1 point per second on half a heart, 0 points total for death). Maybe even just leave it at death = 0 points when you die because that's just a flat out failure. Higher reward for more hunger, same as health. Higher reward for wearing armor (correlated with the actual protection value of the armor) SUPER high reward for correctly crafting items, giving it the incentive to learn High reward for combat engagement, but lose all reward on death. I think that should create a really interesting combat dynamic as it tries to avoid low health and death while still trying to kill mobs Basically... Reward/penalize every single aspect of the game according to how a real player feels about each aspect (Player doesn't like low health, armor, or food, so the AI doesn't like low health, armor, or food)

The fact that the bots survived digging down without drowning in lava is just, cursed.

Some general stuff about making this video: the command to let you see the bots more clearly is /effect give @s night_vision 10000 and you should go into gamemode spectator so you don't have to dig tunnels while the bots are doing their thing. Good idea on the x-ray pack tho

One thing you could add is torch placing as it's important in mining. Reward efficient torch placement and high enough light levels so that mobs cannot spawn. Efficient torch placement would be only placing torches when needed (not placing them too close) and also maximizing their coverage (not placing them too far).

I'd suggest rewarding the bot for "Smart mining" like have it go after ore in the normal progression of the game and stop rewarding it when it gets to much of say coal or stone, to the point where it might even avoid less valuable ores like coal in favor of finding something like diamonds

Never actually taught someone would make a Neural network in Minecraft, this is amazing to be higher

The biggest issue I can see is that since you've just used the same map every time it looks like it's memorizing features instead of dynamically responding My biggest suggestions would be: 1. Use different chunks or shuffle through a hundred 2. Use a recurrent network like a lstm so it can use past information to it's advantage

Friggin genius... Could you tell me where and how you learned the material to make this project happen?

Wow, really cool stuff! Keep up the good content, I'm looking forward to your next video. :)

"See if you can find the bug while you watch" Alex: **STARTS FUCKING FLOATING AND GOING THROUGH WALLS**

Really cool and good example. I wouldn’t mind seeing a little more progress on this. Random of 100 worlds, and instead of negative for digging down, large % of worlds should start with lava a few blocks under or a big fall to discourage but not limit digging down. Have them collect 100 of each ore and wood, teach chopping trees, and switching tools. That would be interesting to see a short series with this as a goal. Either way, awesome work!

I'm not even 2 minutes into the video and it's professional as hell. You deserve more subs than what you all ready got mate. Keep up the great work!

This is crazy cool, I could legit watch someone... Commit their entire life to this endeavor. I would be their biggest fan

There's so much more you can do with this. You can add inputs like hunger and rewards from eating food. You can add rewards for getting the most number of points with the least amount of work etc. Thanks for your vid.

Awesome! Just started learning the basics of AI, and that was the most helpful explanation of a neural network I've seen yet. Also, really well made, great production. Thanks for making the video, I enjoyed it thouroughly.

I love how you documented the process and shared it, Please make more like this!!!

This is so amazing, i love neural networks!

Looks like it's on it's way to making a strip mine. Hopefully it learns to place torches before the mobs start spawning.

Just found this channel and finishing watching this episode. Wish there was more since ik interested in this stuff. Hope you'll upload again soon. Awesome vid, and enjoyed it

Thank you for including weights in your explanation of neural networks! I've seen a few videos on this subject, and you're the first person I've seen to actually include them in the explaination

I think programing so that they feel hunger and a fear responce to certain mobs were it can choose to attack or run based on certian specified parameters would help pramote more comples seaking behaviors

This video taught me how neural networks ACTUALLY work. Thanks man!

This video is hella underrated. Very neat idea for a project!

i know basically nothing about programming and neural networks, but this was still entertaining

For your first video this is has really high quality. Loved the vid

i remember watching a video about someone who made an ai that could play Minecraft by only giving it the screen (thought it was compressed) as input, and basically giving score for surviving and removing score when it died. could anyone point me to that video? this video reminded me of it

I think this serie it's a really cool concept and you had all my support

cool, good job! please create more content like this, i was very intrigued :)

ai: walks past tons of iron and coal and proceeds to mine in straight lines Poppro: I'd say this is a great success

The f3 information page also provides a "targeted block" readout, if you allow them to access that Information as an input similar to how you gave them coordinates it could help promote behaviors of looking around and intentionally mining ore when they see it instead of blind tunnelling, which I think would be the logical next step for the ai to do better mining

Explains a Neural net in the simplest way I have ever seen and understandable with an example included !! very good!

That was the best neural networks explanation ever, it finally clicked. I have been making basic neural nets without even knowing. Omg thx so much

Every Human Ever: NEVER DIG STRAIGHT DOWN!!!!!!!!!! Chad AI: Hold My Coolant.

Best approach would be trim the time down to train the ai by coding on already learnt ethics in real life by us and then training forward. Like In real world, jumping up and down or standing in one place for long time without any reason would be considered unhealthy, waste of time in AI world. So just as you coded not to dig not more than 2 blocks down, similarly code few things not to do more than set amount of time, but also allows the AI to bypass such limitations if priority meets the needs.

Great explanation! Would love to see more of what this MC AI is able to do in the future

Will definitly look forwards to the continuation of this series

I know I'm super late to this, but if you want it to explore more organically, I think you should reward it for seeing an ore block and for moving towards said ore block. Making sure that this action can only occur once so they don't just end up spinning in circles for points.

I challenge you to create an AI model that teaches them to survive. Maybe they'll learn to build, fight and farm by themselves if they can access their own inventories and interact with objects? Sounds like a much larger project but simply imagine what an AI companion mob could accomplish alongside players

This is really cool, I would love to see this continued

I hope you continue with this long term... it will be amazing when an AI is advanced enough to build a computer inside Minecraft.

That's the content I'd like to see more on RU-vid :]

In the future, I get to have a friend who plays with me, and never leaves me with a unfinished castle.

This video was super intresting and very well made! subbed!

wow, this is really awesome! what a great first video

This is really interesting as a first step, I would love to see how far this can go. For instance, what if you punished it when it died? What if you told it some basic information, like created an association between mountainous biomes and emerald ore, so it knew (without spending millenia training) that it could find emerald ore in mountains. What about making it guess how far it can jump so it actually tried to parkour? This could go in some really interesting directions depending on where you want to take it. Plus, its literally the first actual description of what the fuck neural networks are I have seen so far. Most just say "it like neurons, next question" (another thing that would be cool is if you composited the weights of all of the networks in a generation, with the total score they earned deciding how much they impact the baseline for the next generation. This would more closely mirror genetics since a single mutation never becomes homogenous throughout an entire species over one generation and by creating composites instead of deciding one victor you allow for it to learn from all of the "failed" attempts slightly.) On a side note, is there any specific reason you choose the network layout that you did? Like how many nodes in each collumn?

This is awesome, are you going to make a tutorial? I've been trying to achieve a neural network in Minecraft, but I haven't had as much success as you lol

crazy stuff dude, great job

this is the first video that *actually* tells me how a neural network works. all the others just throw up the nodes grapic and wave away the "this number goes to this node which goes to this one"

How can such an amazing amount of intelligence and work get put into this project and yet fundamentally mess up. The bots should not mutate from the single best bot. They should mutate from the best 50% or 10% or what ever seems like its working the most. Also, this simulation makes it such that the buts learn what best to do in that specific map. You need to make the bots in Minecraft instead of a simulated program. Even if that's not possible cause it's too slow, that literally how it needs to be done in order for random seed generation and accurate sim world to game world translation since both are the same. I really like this video and the concepts in it. Genuinely good job. You just need to make those changes and you will have an absolute banger of a video.

i love how he just casually writes "how to minecraft" in his search bar

Great job there ! Recode an entire tool for NN etc is huge !

It would be quite interesting to see how the bot learns if you give it milestones. Eg, collecting wood gives it a large amount of points but only the first time. The same for making a crafting bench, etc. I think a bot that can start in any chunk simulation, make a pickaxe, upgrade to iron, and then make a bucket. Seems like the bot invented strip mining pretty quickly, so the hardest part would be the progression ladder.

You should make the rest of the footage available on patreon or you could have a neural network course describing how to replicate this

I would love for you to continue this project a bit to be honest, or maybe if someone can wrote the code in python so i can make do some work with the project. It would be cool to bot only watch the AI mine but also to see it build and even craft, as well as for it to learn to obtain needs for survival and other complex game play features.

Remembering my Armor Stand KI making out of Skript. Could walk anywhere and jump one block up to upstairs. I loved this one.

your explanation of neural networking is probably the first that I actually understand

IF ONLY EVERYONE SAW THIS if given a rly interesting thumbnail and a more visual introduction for us dummies this would blow tf up

This is gonna blow up soon

From a newcomer standpoint, your work is truly impressive!

This fact that you can do this only based on your knowledge and resourcefulness is beyond me. Congratulations on this badass video man.

I love how the bots first instincts are to dig straight down

It will be cool if we forced Ai's to play together and communicate and forming nations

Great video id love to see more from you ^^

Such videos are so interesting to watch even if I have no clue how to do any programming for this.

what if you have the AI watch like 1000s of "minecraft first night survival" or speedruns so it can map some common and rewardful actions?

How does this not have more views?

I like how by the end the network is just solidifying what we already know, strip mining works

I'm just a bit into your video, and I've gotta say; I LOVE IT 10/10 content.

Loved the content, if you need servers for testing I can provide, we need this to grow

Now turn this into a mod, and villagers will finally be useful, they will build their civilization from stone age to modern age, with them working of course, not just giving them materials This is my type of mod too, so I don't feel lonely playing Minecraft lol

Wow. This quality of content from a channel with only 2.5k... Good Job. I'm just amazed

neural network teaches itself to break the first rule of minecraft

Using xray instead of spectator mode is really like slapping mojang

Good job, I think the maps should change every few epochs (or every epoch) so that they don't just memorise the locations of where the ores are, and therefore become smarter.

This is slowly getting into the recommended pages

Are you going to continue the series?

Actually, the seconde set of reinforcement learning with your rule of never digging straight down was not the right thing to do. Your ai learned that digging straight down was the most efficient way to win according to the map you gave it. Implementing an artificial rule to stop it from using this strategy is counter productive, because now you are partially forcing it to comply to how you think the game should be played. Some, in the comment section, have pointed out how you could have had better result without this new rule (which created some new strange and counter productive behavior in your ai). First, players do not dig straight down due to fall damage, lava and cave full of mobs. So you could have implemented a simple death mechanism, when it falls and die 0 points, when it goes into lava 0 points and when it dies from mobs 0 points. Second, depending on where you position your player on the same map, digging straight down to diamond level could be the optimal strategy, sometime (like in the 1st run) there is no lava or caves in between the two Y position. therefore you could have placed randomly your player on the same map and let it learn the different strategies or you could have changed the map every other epochs. Still, a great video showcasing C++ ai programming and java minecraft mod programming!

great video bro, hope ur channel gets more subs

Really appreciate people making great content even with so few subscribers

It's called baritone.

Just use spectator mode instead of digging with xray

This is insane! Very great work. Can't wait for further videos :) Isn't it kind of a genetic algorithm?

That was a hard earned work, earned a sub!

wish they taught me this in school but no i gotta learn about f@*king minerals

I am going to school for software engineering and I hope to do AI classes because this stuff is just phenomenal. The visualizations you had in the video were very well made too. Seeing you use the same IDE I use to code made me really hyped because I could see myself doing projects like this myself sometime in the future. Thanks for the inspiring video Poppro :)

When comes the next part?^^ great job btw. I really would like to see a follow up. Maybe you want to pick Combat behavior as the next project.

Great video!

Thank you sooo much for the source code download link! This helped a lot!

I like this)) thanks for the video

Plz More. It was great!!!

"My computer taught itself to play minecraft" *That's how it all started-*

Man, that was amazing

I must say the editing was great!

This is absolutely a delight to see. I'm surprised at all the criticism and "unsolicited advice" people are offering. Like, let them make their own ANN's. Roko's Basilisk would be proud, sir! **secret handshake** (haha)

Really cool video dude!

i had been feverntly for something like this a year back why did i stumble upon this then and there *sigh