Your bottom-up (instead of top-down) approach that you mentioned in the beginning of the video would be really great to see for all kinds of differrent concepts!
I'm in the middle of a $2,500 course, BUT → RU-vid → your video... 👏🏻👏🏻👏🏻👏🏻👏🏻 Thank you for starting with the "why", and appealing to my brains desire to understand, not just do.
That was great. I have struggled to understand certain aspects of KL Divergence, and this is a great way to think about it without getting bogged down in symbology.
Best Math Teacher ever. So clearly explained the design and thinking process of how the algo comes out. Many video just explain the formula which confused me why we should do this way... Thank you!
In the 'Motivation for log,' you said that taking a simple average is not the right way to go, and then you try to find a function that makes f(4) and f(1/4) have opposite signs. That means you are trying to make two very different distributions have the smallest distance possible (canceling each other out), which is contradictory to what we expected. We expected them to be large.
I recently got interested in learning machine learning and stumbled upon the stable diffusion, the current state of art open source image generation ai. That's where I encountered the KL divergence. The more I try to understand it, more complicated concepts and formulas are thrown at me. I managed to find some videos that explains how to derive it, but none of them explained why the hell logarithm is present in it for gods sake! And here you are, explaining every missing details from other videos and blog posts in a way that the person who knows very little about the subject can understand in a very satisfying and easy to follow way. Hats off to you, sir. I wish every teachers are like you.
Thank you for the great explanation! I totally agree that math is not given from above, but invented by people. And showing how the invention can be done is the best way to teach the new concepts. Thanks a lot!
the thing you said in the first minute, is something ive been saying for a while now. As students we arent told what problem drove scientists or engineers into constructing new formulas or ways of thinking.
I am a masters student in data science and machine learning and I have to tell you that this is the best explanation one can get for concepts like this...Hope you make more videos on these types in concepts.
me not know some of the fundamentals after listening to your explanation made a lot of sense, and I felt I understood the concept well. I am willing to watch your videos more often.
Math should all be taught this way, and to go one step further we should teach people how to make sense of math themselve in the long run Thanks for the explanation for kl divergence though ;)
Thank you for this, the best explanation of KL divergence that I have seen. Love how you approach it building gradually, really inspiring for how to learn math.
Thank you as always for sharing your brilliant teachings, Ritvik. Could you please do a video on the Gram-Schmidt process and how orthonormal basis matrices are relevant to data science?
Great video. Two pieces missing for it to be perfect are in my opinion. If you could just calculate for sum of log (p(x)/q(x)) and show us what's wrong with that number. Exactly as you did with simple p(x)/q(x) why it isn't good solution. Finally in last slide if you could give numbers. You tell about quantification of something that is visually clear, but missing the numbers is kinda missed opportunity to explain how it works :) Again - thank you a lot for explanation. Great work.
When you said that this is an asymmetric measure, my first thought was: "Okay, so probably KL(P||Q) = -KL(Q||P)". However, that's not the asymmetry you were talking about, and, as it turns out, the KL is non-negative. Nonetheless, I'm wondering if there is some neat relationship between the values of KL(P||Q) and KL(Q||P). Can we easily transform one into the other? Btw, I love how you approach learning/teaching math :). Stating the problem first helps a lot.
