Great work ! I would love to see a longer form video with a more in depth exploration into how you make your environment and how you create your agent. Subscribed !
Wow nice!! I notice the walk is much better than almost other that I could find. Are you using imitation learning for that? Or pure reward n punishment? If it's pure reinforcement learning, I wonder how you can achieve quite great walking motion. Great vid!!!
Thanks! It's just reward and punishment here. The walk comes as default when you use the Walker agent from MLAgents in Unity, so it is a good starting point.
I would love to! This has been my dream project for 3 years. The problem is that these projects take a lot of work and frustration to make. Also, this was, weirdly, my worst-performing video.
Awesome! Love the humor that you put in there, and the agent converges pretty well. Is it PPO or SAC? Would definitely be up for more (some ideas: CNN-based RL or even better Multi-agent RL with similar complexity).
Wow nice. 40 muscles and still it learns. Itsy also funny how it developed the main pushing leg and a supporting one behavior. Yes it's limping, but still. What would be the strategy to make it aware of the moving obstacles? Now it just tries to simply run fast, I suppose?
Yeah, this was an enjoyable project to look at. The moving obstacles were challenging to work with: on the swing, the AI does understand how to jump from the edges of the platforms to have a better chance; on the sweeper, he just learned to run as fast as possible, which could be better. I need a way to teach them timing: they should stop before the obstacle and wait for the right timing before moving forward. One way to do this is by adding as input information: how close am I to the start of the obstacle, where is the obstacle in its movement cycle. This information would allow the agent to understand that some timings are better than others. That's one solution. There have got to be better ones...
Thanks! I did a Master of Engineering at CentraleSupélec in Paris, but the AI classes there were too theoretical. I learned RL primarily through the AI club we had at that Uni.
I have always had a question about mlagents: they randomly select actions at the beginning of training. Can we incorporate human intervention into the training process of mlagents to make them train faster? Is there a corresponding method in mlagents? Looking forward to your answer.
Excellent question. What is commonly done to choose human actions instead of random ones at the beginning of training is called "Imitation learning." MLAgents does provide documentation on imitation learning, but I have never explored it, and it is probably complex to implement: github.com/gzrjzcx/ML-agents/blob/master/docs/Training-Imitation-Learning.md
@@alexandresajus Thank you very much for your answer. I have looked at the link you sent and found that it is an old version of mlagents, which is different from multiple settings. For example, the new version does not have Brain and Academy's Broadcast Hub. So, what should we do in the new version? Thank you for your answer!
@@keyhaven8151 To be honest, I don’t know since I never tried imitation learning. Try to look up « imitation learning mlagents » online, I’m sure there are tutorials. Or use the older version of MLAgents
