How do random forests work? Decision trees video: • Decision Trees Decision tree pruning video: • Decision Tree Pruning Overfitting video: • Overfitting
I'm studying Data Science at MIT, you really can't imagine man how much "ritvikmath" is helping me, and a couple more channels, before I start any topic I like to tackle it first or just take a general idea, and you can't imagine how much your videos helped! Short, concise, and to the point! Thank you man 🙂 Just one notice, It might be a good idea to choose an easy to remember / clear channel name, sometimes when I'm talking to someone, it is almost impossible to remember the name of your channel, just a clear name with spaces! Thank you again 🙂
Great video. 1) Spoke well and explained the concepts clearly 2) Threw and caught the marker every time, with no interruption in speech while doing so. Bravissimo!
Very well organized and well put together. Simplified enough for the medium, but included just the right amount of detail to guide one in their further pursuits of the topic. Thank you.
I'm trying to learn some ML content as it relates to classification to quite a large degree, and just want to say that this video on Random Forest is one of the only ones that actually made sense to me as a layman! Thank you
Wow great explanation - I am hooked on these videos. Get the main points in a short timeframe - would be nice to have a video on Tuning RF and other ML algorithms. And the pre-req videos are very useful to have the right background to understand this one. Thank you!
Loved the interpretability of the random forests idea! Very clever / useful. I'm guessing that you would want to reshuffle the dth feature for each i to avoid the effect that the shuffled data accidentally correlates with an important feature.
Thanks Professor , your explanation is very good. I am really enjoying your videos and they are helping me to focus on DS. I have seen many videos prior they only mention Idea 1 - Bagging and say it is Random Forests. But you have mentioned Idea 2 - Random Subspaces as well. Just to confirm on it , do the Random Forests use both the ideas ? Do you mean that Bagging + Random Subspaces = Random Forest ? If possible can you explain how to code it ? Thanks for your time on videos ! Many of your videos are good , even your Bias-Variance video is also super.
Ritvikmath, i would like to complement you with the clear direct explanation video's. you make it easily accessable and clear with practical examples. please keep it up. Kind regards, Jan Pieter Wagenaar
I think that it's since we're trying to focus specifically on the importance of each feature to the model. We're avoiding adding the additional variable of how well the model generalizes and therefore works on the test data so we can see the features' contribution to the model's accuracy under ideal conditions.
u r incredibly amazing ,but i have 2 questions : 1- What is the meaning of when i use all features the tree will be correlated to each other, i know what is the meaning of 2 features are correlated ,but what is mean when i say 2 trees are correlated ????? 2- when i need to determine how much a specific feature is important now , i trained the model using 80% of the dataset and now do i get the accuracy of this (80% dataset) of the dataset and after that shuffle my specific column and get the accuracy again of 80% of the data after shuffling then subtract them ? or i'm using 20% for both ? but u said in the video u r get the accuracy of the data that made that tree so u almost talking about the 80% , it make no sense for me using 20% of the dataset
Yo I heard the RF is bad alone and needs help when: 1) a strongly predictive linear feature exists in X. You gotta help the RF out by either feeding it the residuals from running the linear model on that feature first, so each model in the ensemble can do what it does best, the linear doing linear things and the nonlinear RF doing nonlinear things. Or else just preprocess to create an additional feature which is just the output of the linear model, and give the whole augmented feature set to the RF now. 2) 2nd order associations are expected to be important, because despite its subsampling of feature space, the RF is actually NOT good at automatically finding 2nd order predictive associations in X. THus we should help the RF out by doing some feature engineering of the 2nd order terms in advance into the X and then give it to the RF NOW. Further it might help still more by telling RF to stop using the typical 0.5 ratio default of subspace sampling and instead just focus on exactly 2 columns at a time, no more, no less, forcing it to look much closer at all the 2nd order associations that you expect should be found by the RF. These are hear-say and hypotheses. It would be cool to see how to do it in sklearn's pipeline on a dataset like "jewellery" which is used for demo code by the pycaret library. Jewellery has a strongly predictive feature "carets" or "weight" in its X. But they just look at trees alone in their model search, so I think it can be improved by helping out the fancy nonlinear tree models as described above.
Can't we get the feature importance for free, without permuting, by looking at the accuracies of models trained with and without certain features (in the random subspace step)?
