Idea - Could you do a reinforcement learning where you learn MNIST numbers. A random path is optimized for best fit in viewing the pixels in each sample. Like moving a camera or moving an MNIST number in the camera view. Move the number so a simple model gets the best recognition. Maybe a spring constant function. One the resets its value but gets the right value when there is input to be processed.
I'm really having a bad time understanding how the DRL model needs to be trained in regards to the Bellman equation. Simple Q-Learning is easy to comprehend with the q -table and continues update to the q-values per actions. But in Deep RL i can't seem to grasp how the model needs to be updated. My first thoughts was using DRL like a normal supervized learning problem. first creating a q-table from using Q-Learning and than using a deep model on that data and so trying to train this model for other data that the Q-Learning has not seen yet. But obviously this is not the case. Do you have any recommendations on what to read for me to understand more slowly what is supposed to happen regarding the model update phase of the training?
Next time may be you should create a model with Subclassing API which allows you to create any kinds of custom losses, metrics, etc instead of using a poor sequential model wrapped in one function lol. And for such models have you tried to use Batch Normalization or selu activation with lecun kernel norm to get much faster convergence?
Unsure if your comment is regarding the old Keras API or the new post tensorflow 2 API. TF2 makes this a breeze (and I will continue to use sequential models as they continue to be useful).
Really useful and I'm also struggling with DQN for solving mazes. Currently, I'm not using a target network since a lot of tutorials don't contain a target network as well. Does that really matter and will try to add it now. Thanks a lot
ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-_7D8W-uUSxw.html I made it unlisted because of the unmitigated disaster it was. Trust me you don't need it 😅