Timestamps for mobile users: 0:00 Intro, goals 3:15 OpenAI gym discussion and exploration 23:45 Model class 37:00 Replay buffer 55:00 Train step/loss calculation 1:17:30 Debugging the train step 1:29:00 Setting up main for the final training 1:34:00 Sanity check, adding logging 1:43:00 Updating replay buffer to be faster 1:47:00 Epsilon greedy implementation 1:57:00 Training iteration 1 1:59:00 Oops ... fixing loss 2:01:00 Loss being wonky, tuning hyperparams 2:05:00 Training iteration 2 2:13:00 Writing code to test the model
Excellent video! Well motivated and much more informative than most videos on the topic. Watching you debug and go through code optimization was an invaluable lesson here. Thanks for posting.
Thanks. Glad it was helpful. I wanna do more but keep not finding the time. I think demonstrations of debugging are sorely lacking for educational material in all computer science and that was why I focused on that here so much.
I have watched many videos of deep learning in Python, but this is by far the most educational I have watched! Thank you for uploading this content, and I look forward to watching more content from you.
Could someone please explain why in the loss function, we are multiplying by the one_hot actions and done_mask? I do not understand: loss_fn = nn.SmoothL1Loss() loss = loss_fn( torch.sum(qvals * one_hot_actions, -1), rewards.squeeze() + mask[:, 0] * qvals_next * 0.99 )
A walkthrough of your entire desktop, the window manager, Emacs, desktop env, operating system would be really good. I'm looking forward for it. Also, discord?
Silly question. @34:26 how are you able to insert text from the minibuffer into the main buffer like that? Seems like such a useful behavior in emacs. Cheers for the VERY useful tutorial!
Terrific tut! One quick question, though: why using one_hot to encode the actions, while we sum the qvals along rows in torch.sum(qvals*one_hot_actions, -1) ? =)
@@JackofSome Thanks for this. Well I need to ask you one thing that my loss is getting reduced properly but avergae reward isn’t going above than 28 even though I’ve used the same hyoer parameters as yous
Sorry. This is the correct link. github.com/safijari/rl-tutorials If you look through the commits you can find the state of the code at the end of each stream
@@JackofSome Thanks I'll check it out. Would you mind telling me that is there any possible reason of average reward not going above than 60 even though I've used the same parameters as yours