3:44 Intro, Gaussian Distribution, Probability Density Function (PDF) 7:38 GMM Intro 9:08 Covariance matrix 10:15 GMM Definition, K Gaussians 11:30 How to apply GMM for classification 12:30 Problem statement, Fitting a GMM model, Maximum Likelihood Estimate (MLE) 13:58 Similarity to Kmeans clustering algorithm 16:13 Expectation maximization (EM) algorithm and difference to Gradient Descent 18:15 When to apply GMM, anomaly detection, clustering, object tracking 19:30 Coding example with Python 25:10 EM algorithm workflow in practice, Log Likelihood 27:54 EM algorithm visual / walkthrough 36:30 Summary great video, many Thanks :)
I watch 4-5 vídeos of you per day. I'm Learning generative models for drug Design Siraj. Watch your videos not only motivates me, also makes my life & study fun and cool.
In case you have bad results using Gaussian mixtures, keep in mind the EM optimization only has local convergence properties, just like gradient descent: it can get stuck. Restarting the the density estimation with other initial parameters might solve it ! :)
warning: when he finger styles his hair, get ready for hardcore info dump. PS: 3blue1brown series on linear algebra has THE BEST vid on eigen vectors/value pairs, no joking.
suggestion at time 6:45 minutes, the y values aren't the probabilities of the x values, intuitively the probability for a single point on the gaussian will be 0.
Thank you! Your videos helped me a lot... I was so lost and confused about this topic that I was on the verge of giving up. Checked out your tutorials that gave a lot of useful information and insights. Thanks a tonne! :) :D Keep up the good stuff
Siraj, I think it would of been helpful if you showed the resulting clusters that you get from the gaussian mixture model approach in your data. You showed how to model your data using the gaussian mixture, but I am unclear on how we get the specific clusters (say 2 clusters) from that?
Siraj never fails to inspire, and I agree with his point strongly - we are the most important community in the world today. We all have a common goal, of making the world better with the best tech we have to offer. I for one am working on a universal translator not just for spoken languages, but for sign, braille and more. ML and NNs has moved my research forward by at least a decade.
hey siraj ! EM is a heuristic with no guarantees for global convergence. there have been recent algorithms based on method of moments, random projections etc. which provably recover the gmm under some assumptions
The quality of the audience is reflected from the content:) Thank you for sharing and helping understand complex subjects in an approachable way. (and not dumbing it down:)
When my friends ask me how to start with machine learning and AI, I tell them Siraj is the way to go! Thanks for making the AI community so cool! Yes we are the COOL GUYS!
Hey Siraj! Just found your channel and it doesn't cease to amaze. I am learning a lot about AI and ML with your vibrant and enthusiastic expression. My 2 cents would be to talk a tiny bit slower but it is up to you. Congrats and Keep up the Good Work!
I got pretty confused around 33:33 with the E step. You've computed wp1 and wp2, which is cool, and then normalised them so their sum is 1 [wp1/(wp1+wp2) + wp2/(wp1+wp2) = (wp1+wp2)/(wp1+wp2) = 1], which makes sense. You then add the log of this sum to self.loglike. But the log of 1 is 0... Which is where you lost me.
You can use gradient descent. it's a standard maximization problem (likelihood).. the variable here is denoted by theta, where theta (for gmm) is the mean, variances (co variance matrix) and the probabilities for every gaussian. nothing stochastic when you have the given data points, a no more complex function then loss of a network.
I have some questions: 1. In the end, what we achieved: probability distribution of people whether they keep playing the game? 2. May it cause overfitting if we set too many gaussian distributions? Regards.
hello, I know this video is a bit old (in internet years :D) but I wanted to leave my positive feedback. I found your video because I am preparing for an exam and your energy gave me that burst of motivation I needed just now. Also, your method was very didactic, you explained something very complex in an understandable and enjoyable manner. Thank you so much! Congratulations, best wishes to you!
3:45 Siraj, in my information theory class, I was told Gaussian distribution as the distribution which assumes the least about the data (maximized differential entropy for a given variance) so maybe you can include that in your explanation when someone asks why we assume Gaussian distribution apart from the central limit theorem.
I have the problem with the gaussian mixture models, I don't know how generate outliers uniformly in the p-parallelotope defined by the coordinate-wise maxima and minima of the ‘regular’ observations in R?
Great video, I tried running your code on my terminal and it's giving the error that 'GaussianMixture' object has no attribute 'loglike', would you happen to know why an error like would occur, or anyone by that matter. Thank you so much
Siraj this is Awesome!! Brother... Man you gave awesome reference links. Exploring them gave full knowledge on the concept. Rewatching the video after that made Complete sense.. Hope i find a Job at ML and DL and support you on Patreon
You're the real man! Why didn't you come to Indonesia? We also have ML/DL community here. :) Anyway, thanks for your elaboration of GMM, it is indeed helpful and easy to understand. Cheers!
He makes mistakes... If only that was the only one... Referring to Variance as Variation... Doesn't know how a Standard Deviation is calculated... omg.
Would be nice with timestamps, since it is quite impossible to find the bit of information about Gaussian mixture models that I was actually looking for...
So the probability density function looks more intimidating than it really is. Thanks for explaining it. If you had to choose between a semester of linear algebra or statistics, which would you choose?
I keep getting this error : AttributeError Traceback (most recent call last) in 10 try: 11 mix.iterate() ---> 12 if mix.loglike > best_loglike: 13 best_loglike = mix.loglike 14 best_mix = mix AttributeError: 'GaussianMixture' object has no attribute 'loglike' I am not sure what to do in this case. Any ideas? Thank you
Hi Siraj, I appreciate your videos and I love your content. I' am working on a project on cross-matching using active learning, what advice would you have for me? I' am trying to build something scalable but not so computationally intense.
At 4:35, it appears that the score is nonnegative. Although a Gaussian distribution is a close approximation in this case, could a log-normal distribution also be used in a Gaussian Mixture Model? Are there advantages to selecting a Gaussian distribution instead?
Hi. Great again Siraj. You're the best on that online apparently. Should we have a video about non-parametric estimation or Higher Order statistics, perhaps ICA?
Hi, Im following this channel for a while now and love that you create different series. can you make a small series of basic examples next, so it's easier to learn and get started. With one of your first videos I've just created an sklearn programm that had 50 examples of fruit and car names and with KNN I've got pretty good results. but they are not perfect. now I want to use deep learning for that and would love to see a series where you give different simple examples like this to compare and get started using the different libaries and algorithms. And yes you created some beautiful similar content before but it's not exactly that. Best Wishes
I am trying to use your notebook and getting this error -- any ideas?? I am getting an error for #checking the fitting process AttributeError: 'GaussianMixture' object has no attribute 'loglike'
Hey Siraj, I have vectors with 10 components, thus 10 features. I labeled the vectors by 4 classes. I wanna use GMMs to calculate the probabilities for a new incoming vector belonging to each one of the classes. What do I use? Do I have to create a GMM for every class? If yes, how to model a GMM to a 10 feature vector? Or could or even should I use Multivariate Gaussian Distributions instead?
You guess a theta ( model params) , then that gives you a probability distribution of the hidden variables. With that known, you maximize the joint probability distribution of X and the hidden variables. That gives you a new theta. Repeat the 2 steps above: use the new theta model params instead of your guess.
Hey thanks for the video, However i noticed that your solution is rather hardcoded for a mixture of 2 distributions. What if we are dealing with a more complicated data set and we do not know how many distributions will be mixed? Is there any deterministic approach to find out this number?
Siraj I have a question/problem. I have two data inputs which is to be comparatively trained by a learning model. It's not a multiple set of data but only one. It's a set of pair of inputs. I have been reading pairwise svm. How do I do that? Is there a better model.
Loved the explanation. If I have to model 6 features instead of 2, and use a sliding windows approach on my dataframe (I need to find the anomalous windows), how can I modify the weights and the rest of the code? Just looking for direction.
It's always great and informative to watch and learn from your video. But my question is a non technical, but do provide a solution plz... Question : I saw your github profile, and I'm curious what filters you applied on your profile pic(dp) ?? :p ps: I already told you this question is going to be a non-technical one and Yes !!! you have been on my youtube's subscription list from the very beginning. Cheers !!!
Hello Siraj, I am working on a project to extract the total bill from restaurant receipts. Is there any way that I could use CNN or any other deep learning techniques to achieve this. I am new to Ml and would greatly appreciate your suggestions.
So... 38 minutes to predict something and he just forgets about the prediction part? I'm sorry, but the justification on 36:02 is not enough for my satisfaction.
