Let's explore the math behind principal component analysis! --- Like, Subscribe, and Hit that Bell to get all the latest videos from ritvikmath ~ --- Check out my Medium: / ritvikmathematics
Thanks infinitely for all your videos, you're literally the best at explaining these concept in a clear and excellent way in order to continue with what we have to study/ do! Huge respect man.
Dear rivitmath, Thank you so much sir for your clear explanation. Even being in my last year of college, I am still struggling with the basics of statistics. With your help, I have been striving exponentially in class and looking to graduate from college in this semester. Your videos have been so so so helpful and i wish you an amazing health to continue with your content. I wish you could have been my professor in college. Thank you for putting out the high quality contents. Words can't describe how much I appreciate you, sir. Thank you. You have changed my life.
I've been wrestling to get all intuitional and computational components for doing pca for a while, and seeing it all come together here helps tremendously! Great as always, 10/10 video :)
Thank god I found your channel. I am studying masters degree in computer science in a prestigious university and cost me a lot of money but your channel is very useful to dig deeper and understand many things. Stay on the good work!
12:20 Quick note on why going down the list of eigenvalues is legit, the covariance matrix is a symmetric matrix, and it can be shown that if such a matrix has more than one eigenvalues that are not the same, the corresponding eigenvectors will be orthogonal.
I had to go thru the prerequisite videos to clarify my concepts first, but after that this PCA explanation is amazing! I think you are equivalent to 10 college professors out there in terms of teaching skills. I hope you get that proportion money and the college professors feel ashamed and work harder to catchup to your standards. Again, amazing!
Thanks Ritvik, I went through multiple resources to figure out this exact questions " why does eigen vectors and eigen values of a covariance matrix represent the direction and strength of the biggest increase in variance" . Thanks your video clarifies it beautifully. One question still though, I understand the equation we use to maximise but why do we need the constraint(uT u =1)?
Great, great video I really appreciate your effort and good methodology to teach. I have a question on the projection math. on your projection video you obtained P=XUU but here you used P=U*XU. Maybe this is a silly question but I would really appreciate if you can tell me why this equivalence is possible. Many thanks
I'm loving your content, you're showing a part of math that is not usually shown. The part where you actually use it, where you make your choices and why are you choosing them. Like it's nice to understand the equations and why it gives you a 0 on the sweet spot, but it's also nice to remind that it not only works but it was build to work with that intention. So in the end you still need to figure out how do you get your problem to fit in one of those, what can you choose in these big generic operations to fit it into your problem.
What you've called the closed form of the covariance matrix is actually the biased estimator of the covariance matrix \Sigma. And if you divide by (N-1) instead of (N), you get the unbiased estimator of \Sigma. Awesone video! Thanks :D
There's a property of transposes around 6:45 that you could have mentioned, and I got tripped up for a second. The reason why you can write u^T*(xi-xbar) as (xi-xbar) ^T*u is because (AB)^T =(B^T)(A^T) It's a cool trick, but not obvious
Good morning If you have difficulty understanding the statistical models and programming them with the R software; You have difficulty understanding where the main components come from when you do principal component analysis; You need to discover the statistics for functional data in particular the analysis in functional principal components; you have no idea how to model by functional linear model ... You like clear and detailed explanations. Click on this link amikour.wordpress.com/nos-formations/
Thanks for the amazing video! can anyone please explain why the projection is u1T . Xi * u? In the projection video it is ( Xi . u ) u. Are they equivalent?
Ahh such a clean explanation. I really appreciate! I will have practical statics for astrophysics exam soon, and I was having some problem with the theory part. All your videos were very helpful! I hope I am gonna get a good grade from the exam. :)
Good morning If you have difficulty understanding the statistical models and programming them with the R software; You have difficulty understanding where the main components come from when you do principal component analysis; You need to discover the statistics for functional data in particular the analysis in functional principal components; you have no idea how to model by functional linear model ... You like clear and detailed explanations. Click on this link amikour.wordpress.com/nos-formations/
Hey Ritvik, It would be great if you can generate some problems for viewers to solve. Watching is great but if you can supplement with actual problems then it would drive the points into viewers head. You can then further post solutions on your medium site. Hopefully at least 4-5 problems per each video. I've watched many videos on DS subjects but something in your teaching method is making it simpler to understand. Thanks.
@ritvikmath 5:02 I don't understand where is this formula of projection (proj(xi)=ut xi u) coming from. The projection video does not say that. What the projection video exactly says is that the proj(xi) = (xi dot u)*u. No transpose there! Where did you get that transpose from? And the dot product is missing ? Another question, at 5:50 why do you take only the magnitude of the vector?
Good morning If you have difficulty understanding the statistical models and programming them with the R software; You have difficulty understanding where the main components come from when you do principal component analysis; You need to discover the statistics for functional data in particular the analysis in functional principal components; you have no idea how to model by functional linear model ... You like clear and detailed explanations. Click on this link amikour.wordpress.com/nos-formations/
Just finished the LA section in the Deep Learning book and I can tell this is going to help supplement and fill in this gaps of understanding. Good vid.
Good morning If you have difficulty understanding the statistical models and programming them with the R software; You have difficulty understanding where the main components come from when you do principal component analysis; You need to discover the statistics for functional data in particular the analysis in functional principal components; you have no idea how to model by functional linear model ... You like clear and detailed explanations. Click on this link amikour.wordpress.com/nos-formations/
We find the equation of the variance of the vector, on which we are going to project the data, and then tried maximizing it, because, the vector, for which the variance will be highest (max eigen value), is gonna retain most of the information of the data, after dimensionality reduction.
Brilliant explanation of why eigen vector is the one from maximum optimisation, never saw such great explanation before. Wish your course is in Coursera. I do not think any text book explains the eigen value as Lagrangian Multiplier and eigen vector as maximising variance. Thanks so much.
Good morning If you have difficulty understanding the statistical models and programming them with the R software; You have difficulty understanding where the main components come from when you do principal component analysis; You need to discover the statistics for functional data in particular the analysis in functional principal components; you have no idea how to model by functional linear model ... You like clear and detailed explanations. Click on this link amikour.wordpress.com/nos-formations/
Hmmm i noticed that if two categories are strongly correlated, the plot will look close to a straight line. Going to multidimensional space, that "line" looks like the vector u1 in the video, on which the data are projected. Does that mean PCA will perform better the more correlated two (or more) categories are?
Hi ritvik, thanks for the video. Can you please tell me how the vector projection formula is being used to calculate the projection of xi on u here? The formulae in the two videos seem to be quite different. Would really appreciate if you could help understand the underlying math
This is a great explanation, thanks a lot. It'll be great if you can also make a video showing a practical example with some data set, showing how you use the eigenvectors projection matrix to transform the initial data set.
Good morning If you have difficulty understanding the statistical models and programming them with the R software; You have difficulty understanding where the main components come from when you do principal component analysis; You need to discover the statistics for functional data in particular the analysis in functional principal components; you have no idea how to model by functional linear model ... You like clear and detailed explanations. Click on this link amikour.wordpress.com/nos-formations/
nicely explained! but I noticed you didn't mention the need to standardize the original data for PCA. Is standardization a little trick to make things faster or is it needed in the underlying math?
Hi Ritvikmath, thank you for your super informative videos! I took all courses on this topic but I was wondering if you could expand it with factor analysis and correspondence analysis. It would be interesting to know how different methods work and relate to each other because it would provide a deeper perspective. Thanks
@ 4.54 - you are referring about projection video - on how you arrive projections formula. There is no such mention of U transpose in that projections video.
Fun video. Thank-you. And thanks for all the pre-req videos. Question: I've seen other videos that describe PCA vectors as orthogonal, but using eigenvectors they would not necessarily be orthogonal, right? What is the correct way to think about the orthogonality of PCA vectors? Thanks. * I think I answered my own question. The eigenvectors in question are of the covariance matrix of the related variables. This matrix is symmetrical so the eigenvectors will be orthogonal. Correct?
I HOPE U WILL REPLY 🛑🛑🛑 Aren't we suppose to standardize data before applying PCA and if we do standardize data then mean = 0 At 8:12 the second part of the equation will get cancelled right? So the equation changes
Thanks for such amazing videos. Have 1 question: In the projection video you derived projection as X. V / ||V|| * u here you took started with u1T Xi u. What is the difference? Will be helpful if you can point me to some resources !
It's really a great explanation and one question I got is from the video of vector projection it is clear that the vector onto which we wanna project has the value is (u.x)u where (u.x) is the magnitude and u being the unit vector. Here comes my question in this present video(math behind pca) you used (u^T .x)u as the vector magnitude of the vector which is projected on to. What is the difference in using u and u^t(u transpose)? Can you please answer me?
Good morning If you have difficulty understanding the statistical models and programming them with the R software; You have difficulty understanding where the main components come from when you do principal component analysis; You need to discover the statistics for functional data in particular the analysis in functional principal components; you have no idea how to model by functional linear model ... You like clear and detailed explanations. Click on this link amikour.wordpress.com/nos-formations/
Shouldn't you also constrain u_2 to be orthogonal of u_1 if you want 2 dimensions? Such that each dimensions principal component will be orthogonal to each other. Or is that just an SVD and PCA relationship thing?
