Unity ML-Agents 1.0 - Training your first A.I 

Does the server still exist?

@@soareverix yup

Can you pay attention to dc server that you are streaming because there is no admin and no order as well it creates complicated situations.

instablaster

Thanks ❤️

Nice tutorial but didn't work for me on Windows machine. Please make a tutorial how to install properly on windows machine.

Jorge Barroso, i solved the second problem just importing the ml agents 1.5.0 - preview version.

@@lpthurler Yeah that is was I was meaning, for some reason I couldnt install the 1.5.0 version in the package manager and installed it manually. I didnt know it was only me.

which unity version did you have?

@@gagamagaj2136 2019.4.3f1

Alternative Solution: if you have the ml-agents package from the package manager (which is currently a bit older than the latest release: 1.0.5) - than you could just download an older release from github with the older version (in my case it was 1.0.2) and use its examples folder (projects/assets/mlagents) - helped me with the „2.“ issue

Oh man, Release 2 was a year ago, but it's still the latest verified version that can be downloaded from the package manger

Were there any further releases? Some may consider it cringe, but I'm asking ChatGPT to clarify certain methods that I can't understand from the documentation and it told me that the newest version of ML Agents takes ActionBuffers parameters, instead of float [ ] parameters. This could obviously just be ChatGPT being wrong, but this video and the tutorial I am following are 2 years old after all.

Heyo! Thanks ❤️❤️❤️

Still not working for me, I get no message, I'm simply prompted to enter another command. I'm using the ml-agents0-release_6, do you think that's the issue?

You are an absolute legend! how did you figure that out? it cured that problem I was having. Hopefully this comment I'm writing boosts this comment up in the list so more people can see it. Thank You!

@@pixelpowergo1607 Thanks! Glad it helped someone :)

Hey! Thanks for the correction, I wish I could change the Video 😄

@@SebastianSchuchmannAI It's a really well made video lol

@@SebastianSchuchmannAI I think there may be some ways to do live editing or re-uploading. (people often need to change videos to make corrections or remove copyrighted content). Def look into it when you get a chance! (there's also the popup box option). Not a really big deal though; that one error is the only imperfection in an otherwise perfect video!

Does this mean if I only pass in positions then velocity won't be calculated?

This is my Plan !

There's a built in record feature if you're using Windows using the game window Win+G

(Just had to edit the config file. Got it!)

Thank you very much. You have a great channel, I can probably learn a lot from you. Always open to collaboration!!

I fixed that, thank you!

(using cloned repository) You now need to install the mlagents package and the extensions package separately. From Unity package manager, click + (top left of window), then select add package from disk, navigate to \ml-agents-release_12\com.unity.ml-agents.extensions and select the package.json

try this "mlagents-learn --run-id=MyFirstAI" if that doesnt work do this first: "pip3 install --upgrade mlagents"

Its a third party asset: Behavior Designer

@CraftyClawBoom if you want the agent to learn from player dont do reinforcement learning instead learn imitation learning, you can learn it from channel called code monkey but yes you need to know the basics

I had the same issue but it was becouse i had a folder named ml-agents-release_2 inside the ml agents folder so i had to do the command "cd desktop/ml-agents-release_2/ml-agents-release_2" to get it to work

@@alexhammer2802 I finally know the reason. My CPU is too old that current tensorflow does not support. :(

oh, I forgot a stage between 2) 3), which is the procedure of creating new env using 'conda activate "~~" '

I just imitated your approach but my computer shows a a lot of errors. C:\Users\ymc\Desktop\ml-agents-release-0.15.1>mlagents-learn config/trainer_config.yaml -run-id=whatever Traceback (most recent call last): File "c:\users\ymc\anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in from tensorflow.python.pywrap_tensorflow_internal import * File "c:\users\ymc\anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 28, in _pywrap_tensorflow_internal = swig_import_helper() File "c:\users\ymc\anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description) File "c:\users\ymc\anaconda3\lib\imp.py", line 243, in load_module return load_dynamic(name, filename, file) File "c:\users\ymc\anaconda3\lib\imp.py", line 343, in load_dynamic return _load(spec) ImportError: DLL load failed: During handling of the above exception, another exception occurred: Traceback (most recent call last): File "c:\users\ymc\anaconda3\lib unpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "c:\users\ymc\anaconda3\lib unpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\ymc\anaconda3\Scripts\mlagents-learn.exe\__main__.py", line 4, in File "c:\users\ymc\anaconda3\lib\site-packages\mlagents\trainers\learn.py", line 12, in from mlagents import tf_utils File "c:\users\ymc\anaconda3\lib\site-packages\mlagents\tf_utils\__init__.py", line 1, in from mlagents.tf_utils.tf import tf as tf # noqa File "c:\users\ymc\anaconda3\lib\site-packages\mlagents\tf_utils\tf.py", line 3, in import tensorflow as tf # noqa I201 File "c:\users\ymc\anaconda3\lib\site-packages\tensorflow\__init__.py", line 41, in from tensorflow.python.tools import module_util as _module_util File "c:\users\ymc\anaconda3\lib\site-packages\tensorflow\python\__init__.py", line 50, in from tensorflow.python import pywrap_tensorflow File "c:\users\ymc\anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 69, in raise ImportError(msg) ImportError: Traceback (most recent call last): File "c:\users\ymc\anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in from tensorflow.python.pywrap_tensorflow_internal import * File "c:\users\ymc\anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 28, in _pywrap_tensorflow_internal = swig_import_helper() File "c:\users\ymc\anaconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description) File "c:\users\ymc\anaconda3\lib\imp.py", line 243, in load_module return load_dynamic(name, filename, file) File "c:\users\ymc\anaconda3\lib\imp.py", line 343, in load_dynamic return _load(spec) ImportError: DLL load failed: Failed to load the native TensorFlow runtime. See www.tensorflow.org/install/errors for some common reasons and solutions. Include the entire stack trace above this error message when asking for help. why does it give errors like that? Do you happen to have an idea?

@@Uebermensch03 I had the same error. Dont know about your other problems, but i resolved this with installing another version of tensorflow via pip. In the end i used tensoflow 2.0.0

The documentation of ML agents is what you need to look at for troubleshooting. For me, things started working once I used a virtual environment, which they explain how to set up. Here is the documentation. github.com/Unity-Technologies/ml-agents/blob/master/docs/Getting-Started.md

Sounds fun! It might be possible to do with ML-Agents, it's hard to tell without having tried it. My gut feeling says this sounds more like an Autoencoder problem, it's a type of neural architecture that might be suited for this kind of task. I would say in general that it's less of a Reinforcement Learning problem, so I would look into PyTorch or Tensorflow to implement an Autoencoder. Good luck!

@@SebastianSchuchmannAI Awesome thank you! I'm new to the AI world and your tutorial was really easy to follow. Keep up the great work!

do you know any fixes?ive been trying to fix it for 2 days

I'm struggling with this error for over 2h hours now. No solution from Mr Google could help me. Have you figured out a solution for this?

@@UitzUitz nope I gave up

@@maarten9222 This helped me get it running forum.unity.com/threads/mlagents-learn-is-not-recognized-as-an-internal-or-external-command-operable-program-or-batch-fil.909716/ Maybe it also helps you.

I think you have valid concern. The problem with having the users change the reward is that the training process is seperate from the engine and therefore seperate from the built application. If you want a nicely packaged application like unity can deliever, only inference is possible right now. You would need to create your own package that somehow includes the python and the unity part, which is probably possible in some way. Additionally, evolutionary algorithms are not included in ML-Agents though implementing one isnt too challenging if you are not striving for maximum efficiency. In summary, I would recommend prototyping with ML-Agents because its easy to work with but be careful not to invest too much.

Yup it's RL. So where's the state? I would say there is the environment state which encapsulates everything about the environment and its internal, private state (The game objects in the scene, their logic and so on). Then there's also the agent state, which is a partial representation of the environment state, so everything the agent observes about the environment via its sensors like raycasts or cameras. Does that make sense?

this tutorial has more info on how to do that. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-RANRz9oyzko.html&t. I made it a month ago so everything is up to date.

dunno if you solved this but, go to Package Manager and add by Name com.unity.nuget.newtonsoft-json

I was more referring to the ML-Agents Framework but of course it is even more true for Machine Learning in general. I think Hyperparameter Tuning is one of those complex topics, where an understanding of the algorithms (PPO/SAC) helps a lot, as well as curiosity, gail and of course all the basics of Machine learning and reinforcement learning in general. This course by OpenAI is a great resource for that: spinningup.openai.com/en/latest/spinningup/rl_intro.html But I have in no way mastered it, so take my advice with a large grain of Salt.

Same here Can anyone help?

@@mr.puffin3947 you're either using python 3.8 or 32bit python