DQN in Pytorch from Scratch stream 1 of N | Deep Learning 

Timestamps for mobile users: 0:00 Intro, goals 3:15 OpenAI gym discussion and exploration 23:45 Model class 37:00 Replay buffer 55:00 Train step/loss calculation 1:17:30 Debugging the train step 1:29:00 Setting up main for the final training 1:34:00 Sanity check, adding logging 1:43:00 Updating replay buffer to be faster 1:47:00 Epsilon greedy implementation 1:57:00 Training iteration 1 1:59:00 Oops ... fixing loss 2:01:00 Loss being wonky, tuning hyperparams 2:05:00 Training iteration 2 2:13:00 Writing code to test the model

Finally someone who can explain dqn 🤣 I can't believe that I have been searching for this for 2 years 🤣

Excellent video! Well motivated and much more informative than most videos on the topic. Watching you debug and go through code optimization was an invaluable lesson here. Thanks for posting.

Excellent session! Very engaging and full of detail. I've learned a lot. Thank you!

I have watched many videos of deep learning in Python, but this is by far the most educational I have watched! Thank you for uploading this content, and I look forward to watching more content from you.

Thank you for the tutorial, there were many tips (wandb, ipdb, dataclass...) that I can incorporate into my own practice. It was my first DQN :)

Could someone please explain why in the loss function, we are multiplying by the one_hot actions and done_mask? I do not understand: loss_fn = nn.SmoothL1Loss() loss = loss_fn( torch.sum(qvals * one_hot_actions, -1), rewards.squeeze() + mask[:, 0] * qvals_next * 0.99 )

A walkthrough of your entire desktop, the window manager, Emacs, desktop env, operating system would be really good. I'm looking forward for it. Also, discord?

Yoo great series man.

Silly question. @34:26 how are you able to insert text from the minibuffer into the main buffer like that? Seems like such a useful behavior in emacs. Cheers for the VERY useful tutorial!

sir I got an error in the train_Step that torch.Tensor is not created because it got float32 not a sequence any solution ???

I havn't finished the video but are you doing markov decision process? I see there is a section with elision.

if you use visual code, then no need to use ipdb, just select the code and press shift+enter

Really great stream, one I'll refer to the most when I start implementing DQN. PS. Idk why everyone wants to be RU-vid partners lol, probably bots

Buffer? The Corleone Family had a lot of buffers.

Hey I want to make a bot for a 3D game what data should I feed it

Terrific tut! One quick question, though: why using one_hot to encode the actions, while we sum the qvals along rows in torch.sum(qvals*one_hot_actions, -1) ? =)

do you have the sample code, so I can just copy and paste. Python newbie here.

Is this in github?

My loss is getting increased after updating target_model, I've even tuned my parameters too

what's this ide you are using??

Would you mind providing the link to Github?