This is GOLD!! The DL course in my university is completely different from this. This course just gives you a new perspective. Thanks to you Alfredo and whoever else was responsible for making this public :)
I'm an outsider to ML, CS and even maths and thought I'd never understand this (EBM) yet decided to persist. Glad I did, thanks to your work I feel I'm about 60% there. Just have to play with the math myself but my intuition is solid, which is what matters to me. You're a legend.
I watched a video from spring 2020 about this. I watched this video now. And only when I took a pen and paper and painted everything neatly I understood everything. I'm getting old
@@alfcnz not just a unit but an absolute unit (r/AbsoluteUnits). 🙀 Was today the day I get to return the favor and teach Alf something after the countless days of him teaching us?! 🤞Seems too good to be true 😺
Coming from a physical chemistry background turned to work on ML, I am increasingly finding the two worlds coming together. No wonder why I called my company Softmax😅 Cool stuff + I did not know you studied in Trieste too!
Hopefully, it means a softer-max rather than a softer-argmax! Electrical engineering background here. So, applied physicist? Yup, born and raised in Trieste. Escaped to London. Changed my personality. Left for the States.
(I think youtube lost one of my comments? Sorry if this comment is duplicated) Hi Dr. Canziani, thank you so much for sharing this course with us! Many DL course I encountered so far is a jumble of different model architectures with no overarching insight. This course showed me for the first time how (seemingly) different DL models relate to each other and this was extremely valuable. I just hope to say that I cannot thank you enough for your time and dedication. And lastly sorry I do have one small question / request: would it be possible for us to access the notebook for this tutorial as well, so we can play with the code a little bit? Thank you so much and looking forward to hearing from you!
I'm glad you found my work of some value. The notebook you're referring to was not yet available (there's a difference between teaching notebooks and notebooks I create for my own to generate illustrations). Nevertheless, since I've already written this book's chapter, the teaching notebook is now available. It will be released around Christmas together with the first draft of the book. So, a few months and you'll have it!
Thanks for this amazing video Dr Alfredo. I have a question, during the training of the model, do you use actual-softmin with value of Beta = 1 to do the minimization process and backpropagate through that? and if i want to use Beta = infinity would you do the same with actual-softmax or simply take the minimum distance point for z and backpropagate through it? I could not find training notebook for the same video.
You're welcome 😊 I did use both, β = 1 and gradient descent for the zero temperature limit. The notebook was not a teaching one (I made it for creating the illustrations). The teaching one now do exist and will be released with the first draft of the book.
Hi Alfredo thanks for the clear explanation on a not so intuitive topic :)) I've got a question which might be a bit silly to ask lol At the end of the video you've mentioned that there are 2 things to research/work on: 1. How to find the right decoder 2. How to determine the constraint of the latent variable May I ask for point number 2, apart from the dimension of the latent, should the subset of value of latent also be learnt? Or actually by constraint you mean constraint of all kind no matter it is dimension of z or subset of z or other weird stuff. For example 59:37, should the range of z be learnt so that it is between 0 and 2pi? I understand that in fact z can range from [-inf: inf] in this case coz eventually it is fed into a sin and cos function.
I think I might have figured that out. It should be all kind of constraint, like for VAE we learnt the mean and variance of a normal distribution. For other type of architecture it can be some other weird stuff...
The VAE adds some regularisation over the code, such that there's no infinite low energy space. So, point 2 says that you need to figure out a way to limit the information content of z.
Hi Alfredo, firstly thanks for letting people learn from these outstanding resources! Is there a way to get the homework/exercise materials of the course? Thank you! Grazie (:
Probably naively, I would assume that as a 1d z corresponds to a 1d zero-energy space, a 2d z would correspond to a 2d zero-energy space... But at 1:02:44 you said that if z is 2d, there is no known constraint on the zero-energy space and that the zero-energy space would be flat. Is there an intuition on why is that? Or maybe I misunderstood?
*If the ambient space is 2d,* a 2d latent would let you reach all locations, effectively giving you a flat free energy, unless you limit the volume of low energy by using a regulariser.
@@alfcnz Ah I see... If I understand correctly, the ambient space is the y space which is 2d in this case (y1, y2). And this would imply that in other cases, z can be high dimensional so long as it has less dimensions than y (the ambient space), so that the zero energy space is a subspace of the ambient space.
This is great, thanks for the video. Do you have a public place to take a look at the exercises left as homework and the notebook you are using for the example presented (I'm not a NYU student)?
Hello, Professor Canziani. First of all, thank you for sharing this great video. I have a question about Free energy with beta (Video around 8:58). You and Dr.Lecun mentioned the lowest energy will survive. When we multiply any number with infinity, how they survive? Sorry for the dumb question. But to understand this, I revisited lecture video several times.. Thank you! Best regards, CK
Multiply the integral by exp(-β E_min) and the term inside by exp(β E_min). You'll have that the minimum value of E will get a contribution of exp(β E_min - β E_min) = exp(0) = 1. All other terms will be exp(-β × (>0) ) → 0, β → +∞.
@@alfcnz Thank you so much! Now I totally understand the point. And the relationship between infinite free energy and beta free energy when beta goes to infinity. Thank you again!
Hi alf, I have a question from you: can I share my answers to the homeworks on GitHub so that others could use it if they needed, or do you think it is better to not have any answers around so that everyone could think deeply? anyways, whatever your preference is I would obey because It was nice of you to share the homeworks and you certainly didn't have to do it. tnx for your kindness.
Plz, don't share the solutions. Not only this would give me troubles for the next editions of the course, but would make online students not think deeply about the questions. We post the solutions every time the students receive their graded assignments.
I don't quite understand the difference between conditional and unconditional and their relationship to unsupervised and self-supervised training... I think I missed the formal definitions of un- and self-supervised learning was introduced in the lectures :/ Must go back to slides again...
Wait, is the only difference between the conditional and unconditional case the fact that we have a new predictor? Before we're learning the latent factors from y only (unconditional), now we're learning how x can help predict y_hat as well (conditional)? But why would it be classified as self-supervised?
Conditional: there is an x. Unconditional: there is no x. Supervised: conditional and y is human-provided. Self-supervised: conditional and y is just part of the _raw_ data. Unsupervised: uncoditional.
@@sanskarkumar1614 hum… Like practical stuff / engineering? I've never done a Kaggle competition, so I'm not really sure what problems they typically offer.
