Neural Network Learns to Play Snake using Deep Reinforcement Learning 

Thank you for your kind words. I'm going to be doing more of these. So far this has fared much worse than my other work but I think it has potential.

And also, does it make sense to feed an actual image to the neural network instead of an array filled with [-1, 0, 1] corresponding to states of cells on a board.

The way I set up my network it would immediately scale down the image to the representation you're describing. The overhead here is learning one additional convolutional layer which isn't that bad and it keeps the problem err... "pure" for the lack of a better term (since the aim was to learn from pixels)

The video of supposed to inspire, not discourage 😅. There's a lot of cool things you can do with Snake with many different approaches. I encourage you to still go down this route. If you'd like advice, come to the machine learning discord discord.gg/yHh5UwJ

@@JackofSome It's just that this is the first solution to snake with RL I have seen. ^^ However I wrote this before watching the entire video. Doesn't seem like everything is said and done about snake after all. I'll check out that discord tomorrow. Thanks. : D

I mean, there are many possible ways you could develop this. Using new architectures(such as modifications to LSTM/GRU), or hindsight experience replay without knowing the goal state, etc

That's ... That's the easy stuff to me. I did a silent stream prior to these ones where I built all of that in under an hour. There's lots of tutorials out there on how to make games and capture screen data. It really doesn't belong in a discussion of RL algorithms. That said _some_ aspects are covered in my RL streams

@@JackofSome Could you share the code you used for this video?

That is really really nice. I watched the video it looks great. Do you have a repo i could look at?

@@JackofSome I don't have a nice repo to share now...I 'm busy finishing my master's degree at the moment, but I'm going put everything in a repo as soon as I'm done. It is a distributed deep q-learning algorithm with prioritized experience replay and n-step updates. The other videos on my channel are trained using the same codebase.

我有類似疑問

Thank you so much for watching. I do deep learning and computer vision professionally. Teaching myself deep RL for fun and also because I think it'll be important soon.

Yes. OpenAI does that with DotA and deepmind did it for StarCraft. Both very impressive projects though they both had a bit more information into the game than just the frames and input (deepmind plays by just looking at the image though). There's also some projects people have done playing Mario or the chrome dinosaur game. The data requirements make all these tasks really daunting though (e.g. OpenAI trains on 5000 CPUs for 8 months...)

@@JackofSome Yes. Something we could do? For example we feed the game frame by frame and then the AI analyse the frame to recognise the Text and then AI does actions . . . actions which reduces the distance between text and agent get rewards and like more information is required for better results agent have it's position in the frame/game and be analysing every frame he's able to get the position of the text. That example was in general terms.. Now something you could relate . . . Mario game.... there is that a new demon and our Agent/Player is standing and that demon is standing not moving, the Agent performs actions and distance is reduced only when right action is performed after performing series of right actions the distance is very much reduced and then agent will fire and demon gets killed . . . The goal is to kill demon . . .the reward I have thought of are * Rewarded when distance gets reduced * Reward till the time agent is alive [As if it continues to reduce the distance it will collide with the demon and die] *Reward to achieve the goad '''to kill demon. NOTE: demon has a shape and "demon" is written on it so the AI recognise the text from the frames and gets to know there is a demon. Might have missed something . . . . Hope you understand it and I hope we could do anything about the same . Thank you ❤️

+1 for approach food, -1 for go away from food, +10 eat food, -100 die. It is a CNN. Calculate the deltas for each episode and then update the weights after all 4 snakes die.

I'm familiar with the hamiltonian cycle and the video I mentioned at the start also used a similar solution. This video isn't about solving snake, it's about learning how to solve snake where the input is the game image.

@@JackofSome The comment was meant as starting point for detail background info for your viewers.

I did. Haven't heard back.

Also, for different sizes of the snake grid, would you change the complexity of the model?

I wasn't changing the complexity, but I don't think that should matter. The model I was using was probably more complex than it needed to be. 84 comes from the image size used in the original Atari paper. No real reason to pick that number other than that.

Yes

Hey that looks fantastic. I don't have a mac or iPhone unfortunately otherwise I would have tried it out. What method are you using for the autonomous behavior?