Deep Reinforcement Learning Tutorial for Python in 20 Minutes 

Thanks so much 🙏! Got another run of RL tutorials coming up soon!

Heya @Bogdan, thanks so much! I'm building up to more sophisticated examples of RL. I'll be doing a lot more with different environments in the coming months!

Thanks so much for your kind words @M K! Truly appreciate it!

Thanks sooo much @Raihan!

Thank you so much @Mohammed!!

Thanks so much!! Whatcha working on?

Thanks so much @Alex, you've found it now 😊! I've got way more reinforcement learning and game AI coming in the coming weeks!

Glad you liked it @Bruno! Definitely, got a special one on Reinforcement Learning coming up!

Anytime! A heap more rl videos coming, it's going to be a big focus this year!

Thank you so much @Tom 🙏

Yooo, thanks so much! I've got a bunch more RL stuff coming soon!

@@NicholasRenotte Cool!

Seriously, I'm recommending this channel to my data science class

@@freydunthanos3155 yesss, thanks so much!

A Go fan. Didn't expect to see another person of culture here.

Thanks so much @Bruno!

Thanks sooo much! There's some more reinforcement learning stuff coming this week, hopefully a video on Atari and (assuming my GPU doesn't catch fire) one on CARLA!

@@NicholasRenotte looking forward to seeing that

@@islam6916 awesome stuff!!

YESS! Thanks @Omar, glad you enjoyed it!

Thanks so much @Chop, glad you enjoyed it!

Thanks so much @Ed, glad you're enjoying them!

Hey thanks co much @Maxime, glad you enjoyed the video!

Heya @WhataDay, check this out: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-bD6V3rcr_54.html

Thanks so much @Tom!

Thanks so much @Andy, definitely plenty more coming!

Nice, got more RL stuff coming in the weeks coming @Suvankar!

This worked for me aswell i downgraded protobuff which downgraded tensorflow aswell. After i upgraded tensorflow to the correct version and everything worked. I think the origin of the problem is not having the correct version of TensorFlow in the first place

Anytime!

When I try to run "dqn.fit(env,nb_steps...)" command I am getting ValueError : Error when checking input : expected flatten_2_input to have shape (1,4) but got array with (1,2) can you please help me out??

Yesss! Thanks for checking it out @Varun!

Thanks @Pranav, definitely will do! Got it on the list!

Thanks so much @Housseyne!

Heya @McRookworst, for CartPole we don't use a pixel vector (moreso used in the Atari envs). In CartPole the four values represented in from the observation space are: [position of cart, velocity of cart, angle of pole, rotation rate of pole].

dude I can't import the agents and policies, basically keras doesn't have rl.policy or even rl.agents, what should I do?

# new version with terminated and truncated episodes = 10 for episode in range(1, episodes+1): state = env.reset() #initial for each episode terminated = False score = 0 while not terminated: env.render() # render the CartPole action = random.choice([0,1]) # 0,1 left or right observation, reward, terminated, truncated ,info = env.step(action) score+=reward #based on our step we get a reward till it's done print('Episode:{} Score:{}'.format(episode, score)) Docs observation (object) - this will be an element of the environment’s observation_space. This may, for instance, be a numpy array containing the positions and velocities of certain objects. reward (float) - The amount of reward returned as a result of taking the action. terminated (bool) - whether a terminal state (as defined under the MDP of the task) is reached. In this case further step() calls could return undefined results. truncated (bool) - whether a truncation condition outside the scope of the MDP is satisfied. Typically a timelimit, but could also be used to indicate agent physically going out of bounds. Can be used to end the episode prematurely before a terminal state is reached. info (dictionary) - info contains auxiliary diagnostic information (helpful for debugging, learning, and logging). This might, for instance, contain: metrics that describe the agent’s performance state, variables that are hidden from observations, or individual reward terms that are combined to produce the total reward. It also can contain information that distinguishes truncation and termination, however this is deprecated in favour of returning two booleans, and will be removed in a future version.

@@FrancescoPalazzo26 I had the same problem. In my case I could solve it by importing the modules from a different path, which is 'tensorflow.python'. so the import commands look Like 'from tensor flow.python.keras.models import sequential'. Hope this solves your problem!

episodes = 10 for episode in range(1, episodes+1): state = env.reset() terminated = False score = 0 while not terminated: env.render() action = random.choice([0,1]) n_state, reward, terminated, truncated, info = env.step(action) score+=reward print('Episode:{} Score:{}'.format(episode, score)) run these for new versions

yoooo thanks mate

can you please guide me solve the problem I am getting while working on this example TyperError: _set_agent() missing 1 required positional argument: 'agent'

Heya @Norio! Try deleting it then running the cell that creates the model again. I show it here: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-hCeJeq8U0lo.html

@@NicholasRenotte didn't work for me. But thanks for your help :/ I now use Tensorforce and don't have any problems :D

@@Officialnorio awesome work! What did you think of Tensorforce, I checked it out earlier on but switched to stable baselines a little later on!

@@NicholasRenotte Sometimes my code threw some weird output but changing the agent-type fixed it. Tensorforce is pretty easy to use and does its job (so far) pretty well. I am using Tensorforce for my bachelor thesis about MTSP solutions :D

@@Officialnorio awesome, will need to give it a second chance!

Should be able to reload the weights by running this when you open up again: dqn.load_weights('dqn_weights.h5f') Then to chaneg the number of episodes just change the number set to nb_episodes e.g. For 30 episodes run this: _ = dqn.test(env, nb_episodes=30, visualize=True) For 40 episodes run ghit: _ = dqn.test(env, nb_episodes=40, visualize=True)

Thanks for tuning in @Laha, good to note!

Thanks so much for helping out the fam @Tom!

Thanks so much @Rafael!

Thanks so much @Ahmed!! 🙏

Yo @Gustavo Lorentz! Ya, you'd need a custom environment. You could also try out some of the pre-built games from OpenAI though as a kick start, LMK if you want me to make a video on doing it with games! Also, looks like there's a ton of third party envs you could use as a baseline though:github.com/openai/gym/blob/master/docs/environments.md#third-party-environments

@@NicholasRenotte that sounds like a good video, I'd watch it! I'll try building my own custom environment. Thanks for answering :)

@@Guskiller1 anytime, I've got it on the list for some future videos now!

Thanks sooo much @Bilal, check this out as well :) ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-Mut_u40Sqz4.html

softmax is great for classification, but the experiment shown in the video is more of a regression problem. In this case, it makes more sense to use linear. Doesn't mean you can't use softmax, but your dqn will most likely don't work as you would expect it.

@@xnyu254 here you have two states as the output (either go left or go right). It _is_ a classification problem, and not a regression one at all.

Take a look at the full rl course on the channel, progress it output via Tensorboard.

Anytime @Imed, glad you enjoyed it!

Fr😂😂

I don't think it's available through Keras-RL but if you work with the environment directly you can get it based on the generated action. Want a video on it @Igor?

@@NicholasRenotte I was able to improvise by making him print the action in the env.render(), so I use visualize = True when testing and it returns the action taken in each step. I guess it kinda works. Thanks!

@@igorperessinotto5774 awesome, glad you got a workaround!

Heya @Abrikarim, working on something in this space now. Will share it once it's ready :)

@@NicholasRenotte Thank you. Can't wait for it, because am doing my PhD on multi-agent deep reinforcement learning, so am excited for this.

@@abdikarimibrahim7078 woah, awesome space to be working in!

@@NicholasRenotte yeah, thanks alot

Woah, thanks so much! Awesome to hear about your journey, any particular use case you were looking at with a capsule net?

@@NicholasRenotte You are welcome Nicholas , all the credit is to you , because you explain us ,what you are doing and why you are doing it , so thanks again :-)) Regarding the capsule Net , can we use this example ( lucenaresearch.com/2019/07/15/capsule-networks-deep-learning-for-stock-forecasting/ ) for time series classification just for BUY and SELL pictures from TRADING CHART with the OS and CV2 importation folder. Looking forward for your reply .... Greetings

@@mrwallstreet3438 definitely, might need to do some research into it. But I'll take a look!

@@NicholasRenotte , thanks very much , looking forward to that , since then , i will keep learning with your videos , thanks again , bro !!!

Thanks so much @Mustafa!

Thank you so much @Premith!

Sure is! Check this out @Fatemeh: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-bD6V3rcr_54.html

I don't think that is possible and also I don't think that will achieve your desired outcome. You can define the action space as just about anything from discrete options to box shapes (x and y) or even combinations of the two from my limited understanding. Could you give an example of your desired action space in a given moment?

Definitely, I've got the code 80% of the way there! Should be out in the coming weeks!

Did ya try it? Try things.

If you sample your states what does it look like? Are they non-integer values?

Thanks a bunch @01bit!

I'm having the same issue

Did you ever find a solution to this issue? I'm having the same problem.

Install the package 'rl-agents==0.1.1'. It works for me.

@@GeraLdario It worked!

The variable states is probably not an integer scalar array. are you testing the same eviroment?

Heya @Olivier, try running del model, then rerunning the cell that creates your model.

@@NicholasRenotte it worked for me, but why it acts in such a strange way?

@@adrianchervinchuk5632 I think there is conflict between tensorflow and keras. Seems to happy pretty frequently.

I checked your other video. Deleting the model and reloading the kernel works. This comment is for anyone with same issues

Awesome work @Sriram, yep that's the solution the majority of the time!

@@randomizer272 thank you for your answer, i have the same problem. But my knowledge is still quite limited so i don't know how i delete my model and reload the kernel. Would be nice if you could explain it a little bit more. Thanks in advance!

@@ts717 You can just do a new line del model and create the model again. It worked for me fine. I will attach the video in which he explained about this error. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-hCeJeq8U0lo.html

in the section "def build_model" you have to change the line : model = Sequential() into : model = tensorflow.keras.models.Sequential() i checked and it seems that's because python can misinterpret it with keras, and not tensorflow's keras (but i have no clue why) this worked for me

Heya @Richchizzl, this is due to a DQN being an off-policy reinforcement learning algorithm. When you train, there are actually two models being trained in tandem, one is consistently being trained based on the latest values (think of this one as a fast learner) the other is being used to generate the next set of states (think of this one as a slow learner). Every 10,000 steps the model weights from the fast learner are copied over the the slow learner, this helps ensure you get more reliable training.

Thank's a lot for your reply! Do you know, if it's possible to change the mark of 10,000 steps within the keras framework? I'm trying to implement the connectx kaggle challenge and it seems like that a slowly increasing reward is completely shaken after the update of the slow learner...

@@richchizzl5020 hmmm, did you try changing that parameter? TBH, I'm now using stable baselines over kerasl-rl as it actually seems, well, a lot more stable. You can set the target model update frequency for a DQN pretty easily using the target_network_update_freq parameter: stable-baselines.readthedocs.io/en/master/modules/dqn.html I did a bit of a crash course on setting up experiments with it here: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-nRHjymV2PX8.html could swap out the algorithm used there for a DQN and set the paramater there.

Awesome @Frank, let me know how you go!

Hey Frank, I am also using Colab but facing a lot of difficulties in rendering gym. can you please help me with that @Frank

@@hardikkamboj3528 hmmm, I normally avoid colab for that reason. Rendering anything outside of the notebook is problematic. You can try training on Colab then rendering on your local machine.

@@NicholasRenotte thanks a lot mate. I have started reinforcement learning, following your videos. Awesome work mate, really appreciate it

@@NicholasRenotte I will try it