#Reinforcement Learning Course by David Silver# Lecture 1: Introduction to Reinforcement Learning #Slides and more info about the course: goo.gl/vUiyjq
Just finished lecture 10 and I've come back to write a review for anyone starting. *Excellent course*. Well paced, enough examples to provide a good intuition, and taught by someone who's leading the field in applying RL to games. Thank you David and Karolina for sharing these online.
I've finished both of them, and I'd say that this one has a better and much more solid content, although the one from udacity is much more light and easy to follow, so it really depends on what you want :)
0:01 Outline Admin 1:10 About Reinforcement Learning 6:13 The Reinforcement Learning problem 22:00 Inside an RL angent 57:00 Problems within Reinforcement Learning
what a wonderful time to be alive!! thank god we have the opportunity to study a full module from one of the best unis in the world. taught by one of the leaders of its field
1:10 Admin 6:13 About Reinforcement Learning 6:22 Sits in the intersection of many fields of science: solving decision making problem in these fields. 9:10 Branches of machine learning. 9:37 Characteristics of RL: no correct answer, delayed feedback, sequence matters, agent influences environment. 12:30 Example of RL 21:57 The Reinforcement Learning Problem 22:57 Reward 27:53 Sequential Decision Making. Action 29:36 Agent & Environment. Observation 33:52 History & State: stream of actions, observations & rewards. 37:13 Environment state 40:35 Agent State 42:00 Information State (Markov State). Contains all useful information from history. 51:13 Fully observable environment 52:26 Partially observable environment 57:04 Inside an RL Agent 58:42 Policy 59:51 Value Function: prediction of the expected future reward. 1:06:29 Model: transition model, reward model. 1:08:02 Maze example to explain these 3 key components. 1:10:53 Taxonomy of RL agents based on these 3 key components: policy-based, value-based, actor critic (which combines both policy & values function), model-free, model-based 1:15:52 Problems within Reinforcement Learning. 1:16:14 Learning vs. Planning. partial known environment vs. fully known environment. 1:20:38 Exploration vs. Exploitation. 1:24:25 Prediction vs. Control. 1:26:42 Course Overview
Excellent moment around 24:10 when David makes it crystal clear that there needs to be a metric to train by (better/worse) and that it's possible - and necessary - to try to come up with a scalar metric that roughly approximates success or failure in a field. When you train something to optimize for a metric, important to be clear up-front what that metric is.
Wow, this is incredible. I'm currently going through Udacity and this lecture series blows their material from GT out of the water. Excellent examples, great explanation of theory, just wow. This actually helped me understand RL. THANK YOU!!!!!
@@JousefM I agree, those explanations by GT professors were confusing and less clear, the entire DS nanodegree which had ML, DL and RL was painful to watch and understand.
1:10 Admin 6:13 About Reinforcement Learning 21:57 The Reinforcement Learning Problem 57:04 Inside an RL Agent 1:15:52 Problems within Reinforcement Learning
I just finished the course, and the people in this comment section are not exaggerating. This is one of the best courses on Reinforcement learning. Thank you very much DeepMind, for making this free and available to everyone!
This is one of the clearest and most illuminating introductions I've watched on RL and its practical applications. Really looking forward to the following instalments.
@1:07:00. Instead of defining P_{ss'}^a and R_s^a, it's better to define p(s',r|s,a), which gives the joint probability of the new state and reward. The latter is the approach followed by the 2nd edition of Sutton&Barto's book.
Thanks a lot for the great lectures! I enjoyed watching every one of them (even #7). This is a great complement to reading Sutton/Barto and the seminal papers in RL. I remember looking at the Atari paper in the late 2013 and having hard time to understand why everyone is going completely crazy about it. A few years later the trend was absolutely clear. Reinforcement Learning is the key to push the performance of AI systems past the threshold where the humans can serve as wise supervisors to the limit when the different kinds of intelligence help each other to improve via self-play.
really high quality, I'm impressed at David Silver's (or somebody else's?) choice to offer this content to the general public free of charge.. what an age we're living in :DDDDDDDDDDD
For Me : David Silver is God ❤️ What a Man ! What an Explanation. One of the Greatest Minds who changed the Dynamics of RL in the past few years.Thanks Deep mind for uploading this Valuable course for free 🤍
Rare opportunity to listen to Christian Bale after he is finished with dealing with criminals as Batman. On a serious note, overall great series of lectures! Thanks, prof. David Silver!
Take swimming as example: learning is part that you directly jump into the water and learn swimming to survive; planning is that part that before jumping into the water, you read books/instructions on how to swim (obviously sometimes planning helps, sometimes not, sometimes counter-helps).
I have never heard a punishment described as a negative reward at any point during my 71 orbits of the Sun. You can indeed learn something new every day.
@@pratikd5882 Yes, I did that. The exercises were good, but I'm not an AI guy but a simple programmer. I managed to do the exercises but I think that explainations were very concise. So in 15 minutes they explain what you get in 1 hour on those lectures. I think that is very summarized. But it's good they have exercises. So I don't think after doing that I'm actually able to do much
Do you have any suggestions about which one to start with , the Lecture series here or the RL specialization by Alberta University (on Coursera). I need to apply RL on my own project work. By the way I did the course on Machine learning by NG Andrews and I could follow the pace it was good enough for me and besides the programming exercises helped me alot than I could imagine. But I am not sure if so would be the case with RL by Coursera as well. Can you guide me on this?
On Slide "History and State" @ 34:34, does the order of Actions, Observations and Rewards matter? If yes, then why the order isn't Observations, Rewards and Actions; the reasoning is that the agent sees the observations first, assesses the reward for actions and then takes a particular action? Please clarify if the chain-of-thought went awry at any place. Thanks.
I want to discuss: "All goals can be described by the maximisation of expected cumulative reward" "Do you agree with this statement?" My thoughts why it could be controversy is that you can never specify the reward such as you will never have unexpected side effects/behaviour of the agent. Any other inputs/thoughts?
The above comment was meant to be in the context of your life. Your brain is a cumulative of all your prior experiences and the choices/decisions which you make will be an a action taken by your brain(which is a markov state). So what I perceived from that statement was that, "you need to forget your past and move on".
Hi, am the example of the mouse pressing the lever, would that mean that the representation of the agent state will determine how well the agent learns?
anyone can confirm if this is still relevant in 2022? I would like to study RL. It seems that there is a more recent series but with a different professor on this channel.
