09L - Differentiable associative memories, attention, and transformers 

Thanks, Alfredo! :) Two hours of lecture passed like a few moments

Timestamps: 00:00:00 - Motivation for reasoning & planning 00:09:11 - Inference through energy minimization 00:18:08 - Disclaimer 00:19:02 - Planning through energy minimization 00:32:59 - Q&A Optimal control diagram 00:39:23 - Differentiable associative memory and attention 01:01:03 - Transformers 01:08:14 - Q&A Other differentiable attention architectures 01:10:32 - Transformer architecture 01:27:54 - Transformer applications: 1. Multilingual transformer Architecture XML-R 01:30:16 - 2. Supervised symbol manipulation 01:32:14 - 3. NL understanding & generation 01:36:51 - 4. DETR 01:46:47 - Planing through optimal control 01:55:37 - Conclusion

Have been feeling like in a philosophy class :)

thanks for sharing Alfredo, you are awesome!

The answer to the question yann asked by ChatGPT: Germany shares a border with several countries, including Austria, Belgium, Czech Republic, Denmark, France, Luxembourg, Netherlands, Poland, and Switzerland. It is difficult to say which of these countries has the largest commercial exchanges with China, as this can change over time and may vary depending on the specific goods and services being traded. Furthermore, without access to current information, I am unable to provide a definitive answer.

For masking, is there a strategy to remove words instead of random masking, as if the object of interest, eg: curtain @1:29:19 were to be removed from both English and French, wouldn't it make the prediction task much more difficult, as a lot of objects could be substituted in its place.

Yann gets sad at 1:26:09 while he is talking about attention mecanishm might take the place of convolution at images :/