@@sarmadali5110 if we use fit on test data then the model will also learn the test data and so it will overfit the test data and we will not be able to find that our model is good enough or not on unseen data
Gazab, maja aa gaya... Awesome content sir. Wish yeh channel mujhe pehle pata chta, I could have done some wonder. Thanks,,,This content is much much better than paid course
gender is a nominal data still if i treat it like ordinal data, i will get a column of '0' and '1'. But suppose, i divide 'gender' column into two columns using ohe, but since both columns are dependent on each other, I drop one column. So, now for 'gender' I have a single column with zero and one as two types of entries, indirectly I have treated nominal data like ordinal data, end result in both cases is same.
in this car selling price data set , i think brand , fuel type and also owner columns consist of ordinal categorical data .one might not consider fuel type to be ordinal but if we practically think then fuel type also effects the selling price of car, example , electric cars are comparitively expensive , and there is some similar trend with other fuel types as well depending on the company
@@campusx-official what if we change categories with less frequency(some threshold) to 'others' category and commit this change for our dataframe, initially. And then we can use OHE from scikitlearn?
Hi sir, quick doubt. Let's say i did OHE on training data (Jan -April data), did train test validation split and all worked welI. Now, took another data (June-Aug) for predicting using trained algorithm above but the problem is i don't have same no of categories for a column(OHE) and it threw error saying expected 30 cols but got only 25 cols. In these scenarios how should I approach. And also one more question on Missing data implementation, lets say in the same unknown data(June-Aug) one column has all NaN values but in train data we had data nd did Labelencoding. So in here what should I impute missing values of unknown data with.? @CampusX
sir what about the dropping of the first column in case of one hot encoding with top categories. There can be also a problem of multicoliinearity right?
Why he did TRAIN TEST SPLIT at 18:30 , before applying ONE-HOT-ENCODING, and after that he applied ONE HOT ENCODING only to XTRAIN , NOT XTEST THIS WILL BE A PROBLEM
sir can't we put owner into ordinal categorical data and use ordinal encoding because we can give it priority wise as first owner first priority,second owner second prioroty and third owner last priority
fun fact, I noticed you were speaking hindi in min 14 😂, still by just watching what you do was still helpful!! thanks! EDIT: you are swithing langauges LOL 😂 when I noticed the hindi in my mind I was like: mmmm weird I did not notice, then on min 18 you switched back to english, it drove me insane for no reason. Good video anyways! keep it up.
@@campusx-official No problem at all, of course I am missing some explanations, but my priority is to see the code, if you don't mind me asking, I have cathegorical features where some features have like 50 unique values and other feature have like 7000 unique values. I can't afford to just take in consideration 10 or 30 more frequent values, because I need to be able to predice any target based on those unique values, do you think it's ok to use one hot encoding for this?
sir in previous videos you said that we fit and transform on trained data so my question is that pd.get_dummies method which is used to apply onehotencoding is also applied on trained data? or just the dataframe
TypeError Traceback (most recent call last) in ----> 1 ohe = OneHotEncoder(drop="first",sparse=False) 2 ohe.fit_transform(X_train[["fuel","owner"]]) TypeError: __init__() got an unexpected keyword argument 'drop' please help me to solve this issue.
xyz = np.hstack((car[['brand','km_driven']].values,car_train_new)) a = pd.DataFrame(xyz). Sir ek doubt hai, if Data set is small then this approach is good. How can we get column names and add to the data frame a so that we can see the transform data with column name.
