Thank you, Alfredo, great video. I have been reading about the energy models for a few weeks already and still have a nagging question: Does energy function is a generalized loss function? I keep thinking that I can reframe any traditional neural network loss as an energy function. What do I miss here?
Thank you for posting making such amazing videos. I have a doubt. Whenever you are talking about optimizing the energy function to find the minimum value you are saying that we should be using gradient descent and not stochastic gradient descent. In my understanding, in gradient descent we calculate the gradients using the whole dataset and then make an update while in stochastic case we take random data points to calculate the gradient and then make the update. So I am not able to understand what is the problem with stochastic gradient descent
The energy is a scalar value for a given input x, y, z. You want to minimise this energy, for example, wrt z. There’s nothing stochastic here. When training a model, we minimise the loss by following a noisy gradient computed for a given per-sample (or per-batch) loss.
At 17:56 you say there are 48 different z's, then later you have only 24 of them at 29:43, and then later yet one can count 48 lilac z points in the graph at 42:45. What's the reason of changing the number of z points back and forth?
Good catch! z is continuous. The number of distinct values I pick is arbitrary. In the following edition of these slides there are no more distinct z points but they are shown as a continuous line. So, there are infinitely many z. Why 24 and 48. 24 are my y. I used to generate them with 24 equally spaced z. When I show the ‘continuous’ manifold, I should show more points than training samples. So, I doubled them. Hence 48. It looks like I didn't use the doubled version for the plot with the 24 squares. In the following edition of this lesson (not online, because only minor changes have been made and these videos take me forever to put together) and in the book (which replaces my video editing time) there are no more discrete dots for the latents.
A “loss” measures the network performance and it's minimised during training. We'll see more about this in the next episode. An “energy” is an actual output produced by a model and it's used during inference. In this episode we didn't train anything, still we've used gradient descent to perform inference of latent variables.
So inference means we're given x and y and we want to predict an energy score E(x, y)? I thought lv-ebm was supposed to produce predictions for y, better go back to the slide :/
Ok so I think what's going on is, during training y is the target for which we should give low E for; during inference, we're choosing a y that gives the lowest energy and y is an input. Mind is blown :/
@@alfcnz I am sorry Professor Canziani. I want to tell you, your videos are the best learning source for people who want to study deep learning but can't afford it. Oh your videos and also deep learning by Ian Goodfellow. It is a very good book. Thank you for all the efforts you put in sir :D
Thank you for your great explanation and make your course material available to all. I have a small doubt at 45:25 where you compute energy from all inferences from z samples. Is it the right way to use euclidian distance for computing distance from the reference point (y) to all the points(y hat) in the manifold. ? Will it is more appropriate if points from the bottom half of the manifold resulted in more energy than the first half?
E is a function of y and z. Given a y, say y', E is a function of z only. What I'm computing there is E(y', z) for a few values of z. In this example, for every z the decoder will give me ỹ. Finally, the energy function of choice, in this case, is the reconstruction error.
@@alfcnzActually my question is reconstruction error is the best choice for EBM ? (Funny ideas: construct KNN graph with y_hat manifold and y (observation) and find the shortest path from y to all other y_hat; instead of computing energy between two points cant we measure the energy between two distributions which are formed by y and y_hat in EBM? )
