Machine Learning Tutorial Python - 19: Principal Component Analysis (PCA) with Python Code 

Do you want to learn technology from me? Check codebasics.io/ for my affordable video courses.

Awesome videos - Simple explanations. A balanced approach to teaching with a right mixture of theory and practicals and not overwhelming the learners . i loved the approach - After seeing numerous ML training videos from across the spectrum , this is far most the best one i have seen . Thank you for taking time to create these videos .

The last few minutes were BANG ON! This is what i wanted to hear. Thanks!

super helpful for newbies not scaring them off with too many statistical terms and getting overwhelmed. thank u so much

You are the best! I am doing PG in DS but still, I watch your videos for better understanding. Kudos! Keep it up!

This is a really easy to understand and thorough explanation of principal component analysis. Many others I watched were either too technical and math theory oriented or to basic in showing how to use the function but not what it does. This is a great balance of understanding and practicality.

My long await topic!!!! Thank you for posting this PCA lesson

Thank you, PCA concept is clearly explained . Need to understand in actual real life scenarios, what we consider, the performance or process time

Thanks sir the great work, your explanation makes ML easier for sure 🙏

I would really appreciate for your hard work in making these videos and decoding the complex to easy..

Hello Code basics. I usually enjoy your videos as I learn a lot from them. Can you make a video on association rules, apriori algorithms and any machine model that deals with the determination of interrelationships amongst variables? Thank you

it's like the 10th video i'm watching on PCA and the FIRST one I understand, thank you so much!

@codebasics Thank You for Teaching and helping us!

Another very informative video. DHANYAVAAD ! :)

great explanation on PCA. It's an abstract concept to grasp. well done

Amazing explanation, I understand PCA now.

I was trying to understand PCA, this video helped me a lot

Ahhhhhh I've been struggling to understand this and this cleared everything up, thank you

It was an amazing explanation of PCA without much mathematics and eigen value and vector which scares me. Interesting learning 1. we can know variance explained by each PC which helps.

Masters in Data Science in the UK and still loves watching only your videos :-)

Impressed with the clarity of explaination

Wonderful, as always - thanks for making this video, it has helped me a lot ! Regards

Thanks for this amazing tutorial. Hope you could do one video about when to use feature selection and feature extraction, or even combination of them.

Very informative and practical.

You are absolutely amazing.

Thanks for sharing, it's highly appreciated

You kill it, amazing!

thanku ..pca explained very easily

Thank you Sir. It was really helpful.

Thank you very much for simple and great explanation. I got higher accuracy in SVM=86.74 %, after PCA I got accuracy in RF=73.06

Thankyouu so much for this amazing video

Thank you code basics❤

great looking for more videos dhaval bhai............../

This is the best channel on RU-vid. You are simply amazing. You just saved my career. Thanks a million

You are the best!!! 👏

I am watching this vedio 2nd or 3rd time. @Dhavel you best among equals...👍👍

Great video!

Great explanation on PCA. @codebasics: I must have missed it though, but after running the PCA to identify which features will give the results, I missed where one might get the features.

nice explanation !!

Thank you 🙏

Thanks sir 😊😊

Nice points.

Nice video Thank you

Superb

thank you so much

Sir also make a video on how PCA actually works and what type of information we can gain from the loadings like how are these features affecting the label

nicely done

at 17:35, shouldn't we be doing pca.fit_transform()on our scaled dataset (X_scaled in our case) rather than on X?

Thank you so much

Hi, I just want to ask why we use StandardScaler instead of MinMaxScaler in the lecture (not exercise). Thank you so much for your help!

Please also explain the math behind it. You get questions on math behind PCA in interviews. People ask the derivation of PCA

Thank you for the video. I have one problem, though: Shouldn't we first split the data into training and test sets, then scale each set separately using StandardScaler(), RobustScaler(), etc. ?

Thanks ❤

Thank you

Nice Video sir. Please make a video for the math behind PCA. Thanks in Advance Sir

Your videos have helped me to complete my MSc research. Thank you for being a great teacher. I do have a question, during the explanation, you said we should always use PCA on the scaled data but during implementation, you used the unscaled data. Could you explain this?

Thanks

I would add that reducing number of columns should help with overfitting

Highest accuracy: SVM - 85.83%, after PCA (3 PCs), accuracy was 83.87%. For all 3 models, accuracy slightly (

Highest accuracy 0.8729 with SVM (linear) and with PCA n_components = 11.

Hi. I have a question. why when you perform PCA, for input, you did not give (x_scaled), you gave x?????

Hi Good tutorial, i think you must use X_train in pca.fit_transform and not X. Thanks

Hi. I might not be making any sense here but I wanted to know if same thing could be achieved with entropy and information gain. We know information gain tells you the feature with the most information or importance as a number . Therefore, in theory, we can remove all the features with really low information gains. I think it would this would work with discrete data better. I don't know if it already exists. If it does, what method does this. If it doesn't, can I know if this solution is practical.

Sir please also make a video on how to find relations using descriptive technique.

I HAVE SOME QUESTIONS: 1)if you use ur PCA data that has been scaled before doing any train test split...wouldnt it cause Data Lakeage? 2) should not the target be dropped?

I am getting around 80 % score when using PCA(0.99999) in exercise, which is higher than the score before using PCA, and also getting a better score without removing outliers.

Thanks for the explanation. I have one question: When you are doing PCA, you are taking the whole data and after that you are doing train test split. Isn't bit unethical? Again, if I do pca on train data, is same will the same pca can be applied on test data? Is there anyway to store the transformation of PCA to apply that on test data?

Thanks v m! can you pls help explain Eigen values and Eigen vectors from the data science perspective? thanks again

amajing

Hi, I have seen your videos on PCA they are really informative and your explanation is really cool. I have a doubt, we apply PCA on the dataset and later we go for model fitting for example if it is a classification problem we go for classification model. Here my doubt is, after building a model we validate it with test set, after that if I have new observation i.e new row in the dataset, how to predict my label?

Sir can you please tell which features it reduces How can I get those particular selected (reduced) features as output?

I have a query to ask. You said we should pass the data to PCA after scaling but you passed the original X instead of X_scaled. Why ??

sir i have a question how one can predict target values when PCA create all new columns based on its own calculations

Very nicely described ! I have a question: In your example PCA(0.95) reduces the dimension to 29. But, how do we know which dimensions it picked? I am asking this because I want to use PCA to determine the principal drivers in determining the targets.

After completing the assignment, i got a best score of 0.85 with svm:rbf kernel and after PCA my best score reduced to 0.68 still svm:rbf kernel

Here is my two question for u 1- why u dont scaled the target column means (y) 2- for score as accuracy why u dont compare with actual and predicted u give for score is x_test and y_test why not y_pred and y_test

Hi Sir, very informative video. I have a problem I uploaded a local dataset and when I use code dataset.keys () which shows Index(['Unnamed: 0', 'Flow ID', ............ Now at pd.DataFrame(dataset.data, columns=dataset.feature_names) then it shows an error even I changed data into Unnamed as well but ut occurs the same problem. AttributeError: 'DataFrame' object has no attribute 'data' waiting for your kind reply. Thanks.

Shouldn't X_scaled be passed inside pca.fit_transform() method? Because if you're passing X, there's no point we did the scaling right?

Can you tell how to do varimax rotation?

hello, if I want to visulize this data in 3D, then what will be z axis ?

I have question, we can trained model using pca , then how about imbalance dataset? We not need to do imbalance?

How can I get selected features in dataframe along with its feature name instead of number 1, 2, 3,etc.?

Hi Mr. Dhaval, I am so thankful for sharing such a good informative video. Like "ogobuchiokey2978" even i wanted to know, if there is a specific reason of not selecting X_scaled while fitting into PCA? In the above demo, if I fit raw X, I get 29 new PCA-features but if i fit scaled_X i get new 40 PCA-features. Similarly through your exercise, if I fit scaled_X I get 10 features (only 1 attribute is reduced) with Accuracy of 85% and if i fit raw X, i get 2 attributes, but accuracy dips down to 69%(Random Forest) I believe this depends on the data as well.

hello! i am confused about scaling of data before pca...what should i do if we have 20% categorical feature in my dataset?do i need to scale them?

Should you scale the data before PCA?

feature_names column is now not reflecting in Load_digits() dataset

I have a question. After going through scaling, and then PCA, the features are now all different to the original of values between 0 and 16. When I have a new digit to classify and been provided with the same 64 features, how do I make this new prediction?

sir you told that we should scale before pca but you are applying pca to non scaled data in code

Please I have a problem with some work. I was given a list of words let’s say about 200 different words. And I’m meant to create a code that generates 3 random words each together. Eg wordlist=[a, b, c, d, e,................z] Output should be = a, d, z c, o, x And so on Please how do I do it?

hello sir, i am 27 now and masters is in progress sir i have teaching experience. but now i want to start my career in Machine learning or data science ? so it is possible? every company needs new fresh comers so what should i do?

The dimensions it should get reduced to, isn't it another hyper parameter? For 2 dimensions yeah you can visualize. For > 2, visualization is not possible. How can you decide what dimension you should reduce your dataset to?

After reducing the dimensions is it possible to know which columns are selected after applying the PCA in this example we got the 29 features is it possible to know what are those 29 out of total data frame features

is this the completer ml course?

How do we know which columns got removed?

Hello, I have newly started working on a PCR project. I am stuck at a point and could really use some help...asap Thanks a lot in advance. I am working on python. So we have created PCA instance using PCA(0.85) and transformed the input data. We have run a regression on principal components explaining 85 percent variance(Say N components). Now we have a regression equation in terms of N PCs. We have taken this equation and tried to express it in terms of original variables. Now, In order to QC the coefficients in terms of original variables, we tried to take the N components(85% variance) and derived the new data back from this, and applied regression on this data hoping that this should give the same coefficients and intercept as in the above derived regression equation. The issue here is that the coefficients are not matching when we take N components but when we take all the components the coefficients and intercept are matching exactly. Also, R squared value and the predictions provided by these two equations are exactly same even if the coefficients are not matching I am soo confused right now as to why this is happening. I might be missing out on the concept of PCA at some point. Any help is greatly appreciated.Thank you!

Nice job. Thanks

Why PCA rotate the axis? What is a significance of that?

How to interpret PCA result in regression?

How to see which columns are this?

Sir, I did all the things in exercise. I have a question. If I did the PCA, I'm loosing the feature names. How do I get back them?