Very relatable when u said most ml developers are .fit and .predict people lol. I started learning ml a month ago and same way thought ml is easier than I thought but when I started learning more complicated algos I'm learning the importance of stuff like this, be it feature engineering or selection or another thing I got to learned today: class imbalance. I'm learning off of internet and projects so it's a little hard road but youtubers like u are blessings 🙌 . An idea- Can u maybe make a vid explaining a real life ml problem and how would u go about it, like churn prediction, sales prediction, etc
may my God bless you sir for this distinctly and succinctly explained lecture on RFE and RFECV. It was revealing and understandable for an average mind. Pls open a school sir, i'd gladly be your student.
No one touches these topics the way you're doing. It makes concept more clear. Please can you also make a video on parameters we pass in each algorithm. E.g. init,n_init,n_clusters,max_iter in kmeans clustering. It confuses a lot. Due to this many people don't touch those parameters
Thankyou for an informative lecture. As you mentioned when we apply RFE using Decision Tree algorithm we get different features importances than when we apply RFE using Linear algorithm (coefficients in this case). My question is how do we interpret it i.e. how do we conclude which feature is the most important one in such a case?
Thanks. Keep going. 👍🏼 I wanted to know exactly, based on what concept or what parameters it will eliminate?! Hope I will get the answer if I learn little more. Another clarification required: RFE inside algorithms working varies from each other but output features must be same right?! If that varies which algorithm to trust?! Good explanation.
I have query. U put decision tree classifier or logistic regression for rfe to select features. After using rfe do we have to apply classification algorithm to generate model using different algorithm. Can we use svm, rf, knn algorithm.
Thanks for explaining .. I am getting these two error : 1. __init__() takes 2 positional arguments but 3 were given 2. 'RFE' object has no attribute 'ranking_'
Hey Aman, As you said, you will upload videos about RFE and RFECV with python examples. I looked everywhere in your channel but couldn’t find those. Could you please give me the link to those videos?
Thank you for the RFE video. Your explanation was very good! I have a question: I have an already existing linear equation with 7 variables (feature) including the target feature. However, I want to elimination 3 less important features. Please, how can I custom fit this equation (algorithm): that is I want a user defined algorithm for the RFE. This will help me to eliminate the less important features.
Great explanation, thanks! but i have a question... after using this RFE, we got useful and useless features. what are the parameters used to get those features? cv scores? or something else?
Let's suppose you create an instance with the name "rfe". Use rfe.support_ to get the useful and eliminated features. It returns a Boolean array where True means that feature is kept and False means that feature is eliminated
I really appreciate the explanation though, I cannot agree with your point about the feature importance in linear regression is measured by their coefficients. I think we should use p-value to measure the importance instead.
Sir, In wrapper exhaustive feature selection method we find the optimal subset of feature using certain Machine learning algorithm, so in finding optimal subset can we use any algorithm or there are only specific algorithms that should be used only? I’m confused in implementing this
First off, thank you for a great explanation. I do have a question though. When does the RFECV algorithm know when to stop removing features? With RFECV, does the algorithm stop removing features once the CV score goes down ? Thanks.
Amazing content Aman sir!! cheers.. I have a doubt, on what parameter the RFE is eliminating the features? Is it the r2 score or the adjusted r2 score? How is it different from forward and backward elimination?
Kindly do not confuse RFE with search algorithms that search over all possible subsets of features. RFE does something similar, but not check all possible combinations.
RFE eliminates by taking all the features in the model , ranking them based on coefficients then removing the features whereas in the case of BFE , for eg one feature is dropped then possible combinations of features to model then it checks the accuracy of model , then next iteration based on selected features... Hope it explains
Thanks Aman for this very informative video! Can you please tell on what basis feature importance is decided by RFE algorithm? is it based on p-value and VIF?
My understanding is that it is based on the algorithm that you input into it. So in the case of this example, a Decision Tree algorithm will use the feature importance which is (using the TowardsDataScience definition) "the decrease in node impurity weighted by the probability of reaching that node". RFE is like a secondary step to add on to it. Like backward stepwise regression, which isn't a standalone thing, you still need to decide which statistic you are basing your steps off of. I could be off though and someone can correct me if I am wrong!
Many Thanks for your excellent style to teach important basics of data science. Would you please give the link of your video showing RFE/RFECV usage in Python; I am looking for that video particularly. Thanks in advance.
Thanks Khalid. I think I am unable to find it now. Probably I will search once more or meanwhile this link will help machinelearningmastery.com/rfe-feature-selection-in-python/
Would you please clarify one thing; I am supposed to use RF algorithm for my prediction problem. Do I still need to screen the predictor variables by RFE/RFECV before training the model?
Hi Aman, Can you please make a video on propensity model.Have been looking for it for a long time but couldn't get good resources online. Please help me in understanding it using python
Yes Malavika, these models are mostly used in Risk Domain in Finance or in marketing analytics type of scenario. I have some friends working on these models, I will get an idea and try to present.
According to this, we do REF on the training data. That means we do REF after splitting the initial data set into TRAIN and TEST samples. if so, what happens to the recursive features in the TEST set?
