Difference Between fit(), transform(), fit_transform() and predict() methods in Scikit-Learn

Подписаться 1 млн

Просмотров 91 тыс.

50% 1

Hello All,
iNeuron is coming up with the Affordable Advanced Deep Learning, Open CV and NLP(DLCVNLP) course. This batch is starting from 17th April and the timing will be 12:30pm to 2:30pm IST on Saturdays and Sunday and it will be live sessions.
Prerequisites: Python And Basic Machine Learning
The course fees will be 3000 inr+18% GST.
Download the syllabus and fill the form to reserve the seat
ineuron1.viewpage.co/DLCVNLPAPRIL
Incase of any queries you can contact the below number.
8788503778
6260726925
9538303385
8660034247
9880055539
-------------------------------------------------------------------------------------------------------------------------
⭐ Kite is a free AI-powered coding assistant that will help you code faster and smarter. The Kite plugin integrates with all the top editors and IDEs to give you smart completions and documentation while you’re typing. I've been using Kite for a few months and I love it! www.kite.com/get-kite/?...
All Playlist In My channel
Interview Playlist: • Machine Learning Inter...
Complete DL Playlist: • Complete Road Map To P...
Julia Playlist: • Tutorial 1- Introducti...
Complete ML Playlist : • Complete Machine Learn...
Complete NLP Playlist: • Natural Language Proce...
Docker End To End Implementation: • Docker End to End Impl...
Live stream Playlist: • Pytorch
Machine Learning Pipelines: • Docker End to End Impl...
Pytorch Playlist: • Pytorch
Feature Engineering : • Feature Engineering
Live Projects : • Live Projects
Kaggle competition : • Kaggle Competitions
Mongodb with Python : • MongoDb with Python
MySQL With Python : • MYSQL Database With Py...
Deployment Architectures: • Deployment Architectur...
Amazon sagemaker : • Amazon SageMaker
Please donate if you want to support the channel through GPay UPID,
Gpay: krishnaik06@okicici
Telegram link: t.me/joinchat/N77M7xRvYUd403D...
Please join as a member in my channel to get additional benefits like materials in Data Science, live streaming for Members and many more
/ @krishnaik06
Please do subscribe my other channel too
/ @krishnaikhindi
Connect with me here:
Twitter: / krishnaik06
Facebook: / krishnaik06
instagram: / krishnaik06

Опубликовано:

11 апр 2021

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 140

@jammerules80 4 месяца назад

Thank you for the clear explanation. I spent $10K learn ML-AI from UC Berkeley and yet I could not understand this concept before this video. Job well done!

@mariuszwiesiolek9340 2 года назад

This was fantastic, I really got the essence of not only when, how, and why to use fit(), transform(), fit_transform(), predict() but in the context I was looking for!

@nigamaveena4211 3 года назад

I always get confused about fit() ,transform(), fit_transform()....thank you sir... you are like a saviour to many people like me...

@cocgamingstar6990 Год назад

Still not got

@pratikghute2343 Год назад

@@cocgamingstar6990 see bro, first of all we use .fit in two scenarios first one is at time of scaling and second one is at models training. (scaler.fit_transform(xtrain) and scaler.transform(xtest) that is part of Data preprocessing step and the second scenario we use .fit is at model training (model.fit(xtrain)) there we use fit to fetch the parameters like slope and y intercept.

@survivor9367 3 года назад

Actually I am searching for this in on other videos. As it is not available in you lr play list.. You just updated.. Thank you so much sir

@devinpython5555 3 года назад

summary of this is (intuition) for train data: fit creates formula for all the features in dataset ,transform will transform data with created formula. for test data: formula already created just transform it accordingly.

@srujankumar637 2 года назад

But on test data fit( ) has not applies then how it gives transform( ) value,,, Because mean(mu) and st.dev has to Calculated for test data by using fit ( ).

@srujankumar637 2 года назад

Or data distribution is almost same for both train and test data so that's why mean and st.dev is same for train and test data... And once we got values for train by using fit () that will be transformed for test data ????

@alessandrodf888 3 года назад

Batman, Superman and Krish Naik

@jiaxinxie8794 2 года назад

This is clear, in-depth, comprehensive, and helpful! Thank you so much!

@abdulrahiman8111 2 года назад

Hi Krish. Your video was really informative and helped me understand the requirement as well as the difference between fit(), transform(), fit_transform() very well. Thank you

@gomathic9557 3 года назад

it is very useful video krish, now i got a clear information about fit and transform thanks giving this useful information krish .

@rohitkatkar3736 2 года назад

I was not clear with, why fit_transform for train data and only transform for test. Now i understood this concept. Thank you!!!

@harikrishna-harrypth 3 года назад

Krish, you are a LEGEND!!!!!!!!!!!! Thanks much for making these enlightening tutorials!!!!!!!

@jamalhasanzakarneh9837 3 года назад

Thank you Krish; it is just another beautiful video of your very helpful videos

@akshayg.p8201 Год назад

Krish sir , I got lots of idea on fit(), transform(), fit_transform() and predict() methods. Thanks a lot.

@notmimul 2 года назад

God bless you!!! Your videos make everything simple.

@AromonChannel 3 года назад

Thank you so much krish naik! i've been trying to understand this and you explain it in the very easy way, so we can easily understand it, thank you!!!!!!!!!!!!!!!!!!!!!

@biswajitnayaak 2 года назад

Crystal clear and detailed . Awesome. Keep it up

@Annisa-yc5zp 2 года назад

Hello, you're such a good teacher! This helped me a lot. Thank you!

@nguyenthituyetnhung1780 9 месяцев назад

So clear explanation that i can also understand process of machine learning. Thanks a lot

@01kumarr 2 года назад

Just in one sort u cleared all doubts. Thanks 👍

@optimusagha3553 2 года назад

Simple and straightforward! Thanks!!👏

@suvamgupta2914 2 года назад

Hats off sir!! Your explanation is of God level 💯 Thank you sir ❤️

@ashraf_isb 2 месяца назад

old videos are gold, thanks for this krish

@shashikarathnasinghe1241 3 года назад

Thank you soo much ,, was struggling to understand this concept .superrr well explained

@eitanshirman9072 2 года назад

Thank you so much for such a brilliant explanation!

@theforester_ 2 года назад

awesome video thanks very much. big shout out to all indians out there helping out the world. big greetings from brazil

@zaindeen4490 3 месяца назад

Thank you so much krish sir. It was quite informative! I was searching for this kind of video but wasn't able to find it Thanks for all of your great efforts ❤

@shaelanderchauhan1963 3 года назад

Thanks a lot you have contributed a lot to this community

@doumansarouei6994 2 года назад

Thank you so much for all of the valuable content you shared!

@aishwaryanarkar2954 3 года назад

THANK YOU KRISH AMAZINGGGG BLESSINGS TO YOU

@soajack 3 года назад

Clearly Explained ! Thanks a lot !!!

@ajaykuruba1738 3 года назад

Hi Krish It would be really helpful if you create a playlist on tensorflow serving and tensorflow lite.

@parikshithh4991 3 года назад

Beautifully Explained

@pavanviswanadhapalli3512 Год назад

compared to all other channels ], your classes are so detail and very understandable, so sir please can you make a complete vedio on pca...? please sir

@bkpusprajkumar8744 3 года назад

Thank you so much, sir for this lecture.

@gangeshwarinetam413 2 года назад

thank you sir ..I always get confused but now its clear. Thank you soooo much

@Harshpatel-uw2dw 5 месяцев назад

it amazing video i had come through a great understanding and very easy to understand the concept thank you sir

@robertoespinoza199 3 года назад

thanks so much for the value of your videos 💯💯

@saheedajayi7352 2 года назад

Again, Thank you Krish, well explained.

@rishabhkumar-qs3jb 3 года назад

Awesome explanation :)

@bhargavikoti4208 3 года назад

Finally😁..Thanks for uploading

@saurabhbarasiya4721 3 года назад

Thanks for this

@sabaamanollahi5901 2 года назад

Excellent Explanation !!!!

@jlxip 3 года назад

Thank you so much, this helped me a lot :)

@ajayjaadu42 Год назад

Sir you explain so good .Thankyou for this

@akshitagoel3099 2 года назад

Well Explained. Really very informative. Thankyou so much :)

@muhammadzeerakkhan6300 3 года назад

Great explanation and intuition (Y)

@heliyahasani6859 2 года назад

I love you man you are a game changer god bless you please load more videos

@ayaansk99 2 года назад

Its very helpful video sir Thanks for guiding

@pranaypakhale 3 года назад

Can you please make video on different types of transformation viz standardscaler, minmaxscaler etc and when to use which

@KiranGunda-ph7df 3 месяца назад

Superbbb explanation brother...

@oyesinghji7910 2 года назад

hi krish, can you make a full video of how to do deployment full process video, including all steps.

@darshanayenkar Год назад

you have cleared my concept

@BytemeMaybe 8 месяцев назад

amazing explanation, thx bro

@bluejadoo6912 Год назад

thank you for clearing my doubts sir

@hiral9591 2 года назад

It's amazing👍

@shivu.sonwane4429 3 года назад

Fit_transform use on training data but transform only on testing /new data Applies the same transformation to both set of data which creates consistent column and prevent data leakage it means learning something from testing data this is not allowed

@naeymaislamph.d9976 2 года назад

Excellent!

@priyadarshanr9950 2 года назад

Sir , I can understand that it formats the test data in the same format of train_data , but how does transform function helps to overcome overfitting,

@paulkang2806 2 года назад

if you are fitting, and transforming for the scalers and normalization, and you fitted (mean, stdev) for the training data, and say if you are applying it to the test data, isn't that something related with data leakage?

@hanscesa5678 2 года назад

So where should you use fit(), transform(), fit_transform() during a K-Fold Cross Validation? Before CV or During CV?

@shivamshinde9810 3 года назад

very helpful!! Thanks!!

@louerleseigneur4532 3 года назад

Thanks Krish

@sanyamsharma3940 2 года назад

You are amazing !

@adipurnomo5683 2 года назад

13:46 sir, what the real world application when we don't use test data instead we use unseen data. Is the data from unseen data need to be normalize before put into model?

@arshad1781 2 года назад

Nice 👍

@tonnysaha7676 3 года назад

Thank you very much sir🙏

@anirbanc88 Год назад

superb

@vanditha07 3 года назад

Thank you so much!!

@1111Shahad 2 года назад

Thank you Krish

@mangkhongsai9029 Год назад

Thank you so much...

@adityasharma5876 3 года назад

Hi Krish please make a video on difference between map(), flat_map() and apply() in tf.Dataset

@rkkcode 2 года назад

Thank you .

@akashkumar-bq7cl 3 года назад

hi krish ,what will happen if i apply fit_transform to my test data as well?what will be the outcome?why shudnt we do it?is it because new mean and sd will be calculated for the test data?but we need the same mean and sd and formula of the train data to be applied to the test data aswellright?is that the reason we use only transform?just did not get this part and the rest of the video im so happy that so much content in just half an hour that too for free,GOD BLESS YOU PLEASE HELP

@manishaundale7458 2 года назад

If the train data and test data unique values are different then how can we apply label encoder with fit and transform?

@sugavananv 16 дней назад

Best video! One question: Where is y_test used?

@tatendaVIDZ90 2 года назад

this is beautiful

@subhashvarma4551 3 года назад

sir, if we apply the same mean in transforming the test data as in train data, this may be the case of data leakage where we are leaking information of train to test. which might not be preferable in the real-time scenario as future data should be totally anonymous to the train data. we should also perform a fit transform on the test data in such cases. Need your thoughts on this.

@mouleshm210 3 года назад

No bro, we should be cautious only on the data leakage from test to train data where, future data parameters like mean or min/max values must not be leaked while doing preprocessing, thats why we do only transform() in test data.

@naiduvinay4911 2 года назад

Thank You, understood

@adipurnomo5683 2 года назад

Classifier algorithm whose using distance usually do normalize the datasets before put to model

@preetamchakravarty Год назад

Can you state the screen recording software and the settings you have used for this recording? Thank you.

@frankdearr2772 2 года назад

Hi, I understood about well what you told, but could you tell me WHY y_train is not scaled like X_train ??? For me that is because values are like false or true , if the y_train values were different like 10, 5 , 41, 5.8, etc , I think I will have to scale y_train ?? Please show me the way for that small question about your video :)) Thanks for your great video about that topic Laurent

@kavanadeshpande9690 Год назад

Hi, as per my knowledge, scaling of dependent feature is not necessary when we have less cardinality for classification problem. For regression, if we scale the dependent feature then automatically Mean Square Error will also get scaled.

@frankdearr2772 Год назад

@@kavanadeshpande9690 Thanks, great information. That give me the right way to go ahead. Please have a nice day :) Laurent

@frankdearr2772 Год назад

@@kavanadeshpande9690 Hi, thanks a lot for your answer.. I understand better now :) Please have a nice day Laurent

@kumarprince5054 3 года назад

Thanks

@Trendz-w5d 2 года назад

Thank you sir

@arpankhadka8671 12 дней назад

You said for test data we only do transform, we don't do fit. But can we do transformation without fit?? For standardization mean and SD is calculated by fit according to what i understood from your video. Please explain it.

@nagamohan1412 3 года назад

Hi krish, I am Naga Mohan. I want to use data science or data analyst technology for my fathers agriculture land but I don't how to start actually I am so much confused. I have no data. I don't know how to create my own data for my farm land. Can you please give me tips. How to start the project and how to create the data. We have 2 acres of paddy land and 2 acres of banana land

@milliesadie486 2 года назад

thank you

@munawarabbasi9683 Год назад

Thanks for making it a complete halwa.

@nellitharun8466 3 года назад

Sir unable to access your github filescode IAM learning python from 12 April 10:00am

@rezapourbahreini4473 2 года назад

thank you for your tutorial. There's one serious issue that I want to address here. As far as I know, we're not allowed to do anything that results in leakage from test data to train data. So when you do a fit_transform on a train_data and save the parameters in the scaler, it's okay to do scaling on the test data based on that very scaler, but not the other way around!! Because there would be a leakage for mean and s.d from train data to test. This way always the result would be better but it's because of the cheat that is happening and the model really. So be careful with the order of steps you go through when scaling train and test data.

@swethanandyala Год назад

I too feel the same...we have to fit and transform on the test data also..to avoid data leakage

@omsonawane2848 Год назад

what if we fit on whole data and then split and transform train and test data. This way test data will not depend on training parameters. also no data leakage will occur

@krishnabhadke6161 2 года назад

nice sir

@bivasbisht1244 Год назад

amazing

@SUMITKUMAR-qi6mz 3 года назад

I am having experience of 1 year in customer service in BPO but I want toh become Data scientist . But I'm having difficult toh get job in same because they are asking for experience in data science. Pls help me how to portrait my resume to get job

@yashub9580 Год назад

sir can you please tell me how to resolve this error "Deprecated distribution is specified in `adstock__tv_pipe__carryover__strength` of param_distributions. Rejecting this because it may cause unexpected behavior. Please use new distributions such as FloatDistribution etc."

@kiranvanukuri9382 3 года назад

Plz make video on image recognition in jupyter note book and deployment technique with deep explanation

@unmeshmandal3071 2 года назад

What if first I scale the whole X table and then split using train_test_split?

@subhamsaha2235 3 года назад

Sir, you didnt tell one thing is that if we are applying fit and transform to X_train which means (for standard scalar) fit(calculating mu and sigma) then transform(applying z formula to every value), and ONLY transform to X_test which means mu and sigma are not calculated then how is it transforming the values? I think something else is also there in fit which is used to teach the model? Kindly clear my doubt. Thank you

@saikiranreddykondapalli279 2 года назад

while transforming test data we are using actually the mue and sigma values of trained data and comparing the transformed test data with predicted data .(this is what he actually mean).but it is wrong to do we cant use mue and sigma values of other data.so it is always better to split only after all the data set is fit and transformed.the it is quite valid to check predicted and actual test values

@harshagrawal5613 3 года назад

100℅ clear

@soukainahanafi1685 3 года назад

I understand that but with a polynomial model we use fit_transform and not only fit .It' hard to understand . ##this is the example that I'm working on pr = PolynomialFeatures(degree=5) x_train_pr = pr.fit_transform(x_train[['horsepower']]) x_test_pr = pr.fit_transform(x_test[['horsepower']])

@gunamrit 2 года назад

magician!

@merveozdas1193 2 года назад

In which platform did you tell this lesson? you can use your pencil properly.

@shadmanansari5750 Год назад

Hi, You mentioned that Fit_transform() is applied on Training data and only Transform() is applied on Test data, So, in case of StandardScaler, Fit_transform(Train) will have mean and std dev of train data, and then we are using same mean and std dev on 'Test data' Should'nt we apply Fit(on entire data) to calculate mean and standard dev of entire data, then transform(train) and transform(test)? Please clarify