Тёмный

Improve your Unity A.I. | Configuration 

Bot Academy
Подписаться 11 тыс.
Просмотров 6 тыс.
50% 1

In this video I'm going to show you how you can improve your Unity AI. You will learn the ML-Agents configuration parameters for the Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) algorithm. Those are two commonly used state-of-the-art reinforcement learning algorithms.
Please let me know if you have any questions. I'll try my best to help you out.
PPO Paper:
arxiv.org/abs/1707.06347
SAC Paper:
arxiv.org/abs/1801.01290
IMPALA Resnet Paper:
arxiv.org/abs/1802.01561
Download my Unity A.I. example:
github.com/Bot-Academy/BallJump
Find me on:
Discord: / discord
Twitter: / bot_academy
Instagram: / therealbotacademy
Patreon: / botacademy
Credits:
11.55 - End
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Music: Ansia Orchestra - Hack The Planet
Link: • Ansia Orchestra - Hack...
Music provided by: MFY - No Copyright
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Contact: smarter.code.yt@gmail.com
Chapters:
00:00 Intro
00:17 PPO vs. SAC
03:09 Parameter: trainer
03:21 Parameter: summary_freq
03:57 Parameter: max_steps
04:39 Parameter: time_horizon
05:36 Parameter: init_path
06:41 Parameter: threaded
07:26 Parameter: summary_freq
07:26 Neural Networks in a Nutshell
08:01 Parameter: num_layers
08:20 Parameter: hidden_units
08:54 Parameter: vis_encoder_type
10:37 Parameter: normalize
11:26 Recommendations & Next Video
12:10 Outro

Опубликовано:

 

26 июл 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 46   
@fahadmirza8497
@fahadmirza8497 11 месяцев назад
this video is exactly which i was looking from many days ago
@willdegra317
@willdegra317 3 года назад
Just logged in to say you have one of the best tutorial series ever. You have a clear way of explaining, you know when to break things into other parts and you don't do unnecessary stuff just to pad your video length. Thanks for doing these, it is very much appreciated.
@rajeevjawaji4472
@rajeevjawaji4472 4 года назад
I really like your tutorials I hope you keep making more everyday
@BotAcademyYT
@BotAcademyYT 4 года назад
Thank you, I will! Especially next year when I've finished my thesis.
@ireidemarfernandescosta8158
@ireidemarfernandescosta8158 4 года назад
Unity 3D is my favorite Engine, there are two things I wish I had in it; 1st - The correct way to orbit in the viewport is to just press the (mouse scroll), in SketchUp, this function already comes by default and in 3ds Max, you have to configure it, so it is much more practical to just press the mouse scroll to if orbiting, this is the way it is in several other software, unfortunately in Unity 3D ñ this is the way you have to press (Alt + left mouse button), you have to occupy both hands and this creates a certain discomfort, In other words, you do two jobs and you could only do one, so you want to do two jobs and you can only do one, not to mention that it will take more work and less productivity, this seems to be an insignificant thing ñ it is what it seems because this is one of the functions in which it will be most performed. 2nd - The correct way to zoom in on a certain part of the object, just position the (mouse arrow on a specific point on the object where you want to zoom in) and scroll it (mouse scroll forward), this is how q is in SketchUp and 3ds Max, so it’s much more practical to just position the (mouse arrow on a specific point on the object you want to zoom in) and scroll it (mouse scroll forward), this is how it is in several other software, unfortunately in Unity 3D ñ this is the way to do it you have to press (Alt) Position the (mouse arrow at the point you want to zoom in) and scroll the (mouse scroll), you have to occupy both hands and it generates a certain discomfort, that is, you do two jobs and you could only do one, you want to do two jobs and you can only do one, not to mention it will be more work and less productivity, this seems to be a insignificant thing ñ is what it seems because this is one of the functions in which more will be performed a, share this idea until you reach the developers of this excellent software.
@BlackSheeeper
@BlackSheeeper 4 года назад
like it
@RaptorBot
@RaptorBot 4 года назад
You teach in a unique way. Can you make Godot tutorials, please?
@BotAcademyYT
@BotAcademyYT 4 года назад
Godot is on my list, but it might take a bit. When I‘ve finished my thesis around Feb 2021 I can fully focus on RU-vid :)
@geckoboi2029
@geckoboi2029 3 года назад
3:38 read what the file is called
@tuliomoreira7494
@tuliomoreira7494 3 года назад
Amazing video. I was wondering, is it possible to train a neural network (ppo) where the output is a set of mixed discrete and continuous action values? I did find some papers on it, but I don't know if we can try it in Unity
@BotAcademyYT
@BotAcademyYT 3 года назад
Thanks! Good question. It's definitely possible with tensorflow. I am not sure if there are specific problems in a Reinforcement Learning setting when using mixed action values. It seems Unity doesn't support it directly (yet). I think somewhere in the documentation is a doc how you can use the ML-Agents python connector to train your own models. I'd start there.
@tuliomoreira7494
@tuliomoreira7494 3 года назад
@@BotAcademyYT I'll check it out! Thank for the reply!
@rajeevjawaji4472
@rajeevjawaji4472 4 года назад
Hey if your environment reaches the max steps your agent will automatically stop training right. Cause when i was making my own bot after some time every-time i resumed training it would instantly stop.
@BotAcademyYT
@BotAcademyYT 4 года назад
Yes. When u config it with 500k steps and it trained 500k steps u need to increase the max step size if you want to resume training
@rajeevjawaji4472
@rajeevjawaji4472 4 года назад
@@BotAcademyYT thanks alot
@aamirbadershah887
@aamirbadershah887 11 месяцев назад
can we have a tutorial on as to how you made the first 7 seconds of your video?
@rajeevjawaji4472
@rajeevjawaji4472 4 года назад
Is sac better for making Ai players for an MMO
@BotAcademyYT
@BotAcademyYT 4 года назад
that depends on the complexity and whether you have access to the environment. In general I would say - yes ... especially if you're creating a bot for a game that's not yours (cause u can't sped up the training).
@alvaromachucabrena97
@alvaromachucabrena97 4 года назад
Great video, thanks ! How can I edit the parameters of my neural network in Anaconda using Jupyter Notebook?
@BotAcademyYT
@BotAcademyYT 4 года назад
Thank you :) You mean without using a config file? If so, check out the mlagents folder. There is a trainer folder with a script for ppo and sac. I think that's where the neural networks are defined / created. Otherwise, just change the network parameters in the config file :D
@alvaromachucabrena97
@alvaromachucabrena97 4 года назад
@@BotAcademyYT Thanks a lot ! I'm waiting for the next videos using ML agents ! Youre great !
@alvaromachucabrena97
@alvaromachucabrena97 3 года назад
Do You know How Can I get the weights and biases of the Neural Network?
@BotAcademyYT
@BotAcademyYT 3 года назад
Uhm. Through the python connector as far as I know. But haven’t done it yet so I can’t tell you more. I‘d ask this in the Unity ML-Agents forum. Chances are good that someone knows it there.
@alvaromachucabrena97
@alvaromachucabrena97 4 года назад
What kind of neural networks does PPO use? Do you use a recurrent neural network, a combolutional neural network, or a multilayer perceptron?
@BotAcademyYT
@BotAcademyYT 4 года назад
You can choose between a normal one (fully connected), CNN and Recurrent via the config arguments. (vis_encoder_type, use_recurrent)
@alvaromachucabrena97
@alvaromachucabrena97 4 года назад
@@BotAcademyYT Thanks a lot !
@aryan_kode
@aryan_kode 4 года назад
hey i am using ray perception 3d on a ball. i have turned on the use child sensors on the ball agent. but i dont want the rays to roll along with the ball. could you suggest something
@BotAcademyYT
@BotAcademyYT 4 года назад
Hey Aryan. Very good question that I honestly can't answer. My first thought would be to check the ray perception code and see if you can lock a specific axis so that the rays are not rotating. I'd recommend asking this in the ML Agents Forum - probably someone there encountered and solved the problem already and can tell you how. forum.unity.com/forums/ml-agents.453/ If I figure something out, I'll let you know.
@marshalryan5049
@marshalryan5049 4 года назад
does locking the ball's rotation rigidbody helps? since the sensor probably relies on the agent's position and rotation
@postsbyleon
@postsbyleon 3 года назад
Iheard a lot about curiosity and understand what it means. But i did not figure out how to implement it. Would it be possible if you do a video about this topic, or do have a source where i can learn it, cause i can not find anything
@BotAcademyYT
@BotAcademyYT 3 года назад
I don’t know how it is implemented tbh. If you want to see the exact implementation, I’d spend a day skimming through the mlagents implementation. It has to be somewhere in there.
@postsbyleon
@postsbyleon 3 года назад
Would it be possible to make "fair" Ai. I thought about a simple 2D space game where i can shoot others or mine asteroids to buy some upgrades in a spacestation. The job of the AI would be to be an NPC that mines, shoots others and buys its own upgrade. Could i make it fair so it does not hits like every shot etc.. I read about this in the unity forum and they said it would not be possible.
@BotAcademyYT
@BotAcademyYT 3 года назад
That’s tricky. I’d have a different AI (‘brain’) for each task. General AI is way too hard at the moment. Probably you can implement something that the ‘brain’ changes depending on what’s happening and then use a small NN for the shooting task, so you have a ‘brain’ that’s not soo strong. But all of this gets problematic when two tasks have to be performed simultaneously
@postsbyleon
@postsbyleon 3 года назад
@@BotAcademyYT So I have a "Main" brain that controls the whole Agent. I am still reading the docs an looking in the internet cause I am still new in this technology. But it is really interesting
@alvaromachucabrena97
@alvaromachucabrena97 3 года назад
Using vis_encode_type = simple , Am I using a Multilayer Percepton?
@BotAcademyYT
@BotAcademyYT 3 года назад
Well, first of all, vis_encode_type will only be used when you have visual observations. Otherwise it will be ignored. You have a multilayer perceptron as soon as you have one hidden layer. Which is always, cause num_layers is at least 1 and specifies the number of hidden layers. What vis_encode_type does is adding two convolutional layers before the multilayer perceptron that you specified with num_layers.
@alvaromachucabrena97
@alvaromachucabrena97 3 года назад
@@BotAcademyYT thanks a lot !
@alvaromachucabrena97
@alvaromachucabrena97 3 года назад
@@BotAcademyYT What means when you say " When You have visual observation". If I use like input of my neural network the position of my enemy, is that a visual observation?
@BotAcademyYT
@BotAcademyYT 3 года назад
Visual observation is when you attach a camera object to the agent so that he receives pixel information as input (so basically images as input)
@alvaromachucabrena97
@alvaromachucabrena97 3 года назад
@@BotAcademyYT Thanks u very much !
@pixboi
@pixboi Год назад
The parameter part is confusing, you're saying the exact opposite to what the documentation / default config says. Might be because this is 2 years old
@user-sy2et4dl6y
@user-sy2et4dl6y 3 года назад
Hello, I'm Korean who is watching your video Can you add English Subtitles like other videos for me?? Thank you :)
@BotAcademyYT
@BotAcademyYT 3 года назад
Hey & thanks for your comment. All videos should have english subtitles that were automatically generated. Please let me know if you cannot see them for some reason.
@BotAcademyYT
@BotAcademyYT 3 года назад
Just realized that it is indeed missing. I'll try to add them during the next 24h. Thanks for letting me know
@BotAcademyYT
@BotAcademyYT 3 года назад
just added the subtitles. Enjoy the video :)
@user-sy2et4dl6y
@user-sy2et4dl6y 3 года назад
Your video is very good!! Thank you for your help
Далее
Understand your Unity A.I. | Tensorboard
14:04
Просмотров 6 тыс.
Improve your Unity A.I. | Hyperparameters
16:15
Просмотров 3,9 тыс.
Curriculum Learning | Unity ML-Agents
18:03
Просмотров 8 тыс.
How many AI Agents can Unity handle?
6:18
Просмотров 1,7 млн
Unity ML-Agents 1.0 - Training your first A.I
11:55
Просмотров 113 тыс.
What are AI Agents?
12:29
Просмотров 81 тыс.
Unity3D+ML-Agents Custom Algorithms and Environments
22:15