No video :(

Merging multiple datasets for Machine Learning Project | Challenges in merging multiple datasets

Просмотров 23 тыс.

% 285

Hello Friends,
In this episode, I am going to share details about,
When you should merge datasets
Why you need to merge datasets
challenges while merging datasets
how to handle these challenges
what are all alternative to avoid merging datasets
code: github.com/datamagic2020/pandas-tutorial/blob/0f15ce4b84166ceafcb3495eacd7547fce0f9322/Merging%20Multiple%20Datasets%20into%20one%20Dataset.ipynb
Stay tuned and enjoy Machine Learning !!!
Cheers !!!
#mergedatasets #datapreprocessing #machinelearning
#datamagic
Connect with me,
☑️ RU-vid : ru-vid.com
☑️ Facebook : datamagic2020
☑️ Instagram : datamagic2020
☑️ Twitter : datamagic5
☑️ Telegram: t.me/datamagic2020
For Business Inquiries : datamagic2020@gmail.com
Best book for Machine Learning : amzn.to/3qCe0Rf
Machine Learning for Absolute Beginners: amzn.to/3mMSRUO
Machine Learning For Dummies: amzn.to/32K7Ms6
Hands-On Machine Learning with Scikit-Learn and TensorFlow: amzn.to/3mOf0SL
The Elements of Statistical Learning: amzn.to/3Jysegf
Machine Learning in Action: amzn.to/3mNE7Ff
🎥 Playlists :
☑️Machine Learning Basics
ru-vid.com/group/PLTmQbi1PYZ_E1iTkBrZWK_htO0hY4vcGK
☑️Feature Engineering/ Data Preprocessing
ru-vid.com/group/PLTmQbi1PYZ_EnBmO1-E0Z81ArnE-zSR1a
☑️OpenCV Tutorial [Computer Vision]
ru-vid.com/group/PLTmQbi1PYZ_GrjMHiGCYa0WyDZfxu-yTz
☑️Machine Learning Algorithms
ru-vid.com/group/PLTmQbi1PYZ_EMlRyZDMlza6BkDO_fHlBG
☑️Machine Learning Environment Setup
ru-vid.com/group/PLTmQbi1PYZ_FcSnxMLrlRaPaNy0b5gwId
☑️Machine Learning Model Deployment
ru-vid.com/group/PLTmQbi1PYZ_FTveld2Whf1c5RfBMpxE38
☑️Machine Learning Projects
ru-vid.com/group/PLTmQbi1PYZ_HpgksoXsjnO4cwVixa08df
☑️Kaggle Tutorial
ru-vid.com/group/PLTmQbi1PYZ_Hy697WY2mNKl6LFezwbz8y
☑️Microsoft Lobe Tutorial
ru-vid.com/group/PLTmQbi1PYZ_GqPm9fAQyJaJwBhLVQx8rX
Thank you for watching !!
Please Like, Comment, Share and Subscribe!!!

Наука

Опубликовано:

15 мар 2022

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 44

@IgarokSpider-ht2ve 11 месяцев назад

Best channel. Pls make a video on having 9 datasets for 9 different time series price assets. Separate it in 3 groups: (dataset 1,2,3), (dataset 4,5,6), (dataset 7,8,9). Train each group for price correlation with different AI models. Then in the end ensemble to one meta-model.

@popoolaolawale7564 2 года назад

Thanks for the brilliant explanation. Please, kindly explain further how the large dataset can be used as train data and the small dataset as test data. Without the need to merge the datasets. It will be greatly appreciated.

@popoolaolawale7564 2 года назад

Thanks.

@DataMagicAI 2 года назад

Its simple just load both dataset in two different dataframe. Drop unecessary columns from both dataset. Use one for training and another one for testing. Still will try to create short episode on it soon.

@Futureyouth-be1bo 4 месяца назад

i have two datasets those are flights can i merge them or i can use one for training and one for testing

@adrienykt874 Год назад

hi. can we merge 2 datasets which have the same number of columns but not necessarily the exact headers? if yes can you please upload a video showing it. thanksss

@manojbattula8100 6 месяцев назад

Is it possible to combine two different deeplearning models. For example yolo and CNN.

@hudata 6 месяцев назад

would love to know that aswell !

@shaminakaushar8421 Год назад

please can you make a video on merge medical image dataset and A textual dataset of patient symptoms or diagnoses. i wanted to make a project on : Generate medical images (e.g., X-rays or MRI scans) based on textual patient symptoms or diagnoses for educational or diagnostic purposes

@DeepDiveintoSuccess 2 года назад

Thanks for Great Contents. Is it possible to make episode about getting data from different source with out copying in our computer such as direct connect from sql and github and kaggle . Thanks

@DataMagicAI 2 года назад

I haven't tried but I guess its possible. For example: I can access data from sql table with , pandas.read_sql_table() You can refer below API doc for more details, pandas.pydata.org/docs/reference/api/pandas.read_sql_table.html

@Work-n8h 2 месяца назад

can u make video on federated learning tutorials from basic?

@ChristianahOyewale-w6y Год назад

Please can you make a video on training a deep learning model using a dataset and testing it on a different dataset? Please note that both datasets are from different representations

@DataMagicAI Год назад

Typically we won’t do it. It will give poor performance. We should have all representations during training so our model can learn best out of it.

@hundumamarzo7618 Год назад

Thanks it very Nice Presentation But How Can combine Different Features /Column not records with Same common Feature ? E.g I want to predict Performance of Employee with different Features like in banking based on Quantitative parameters Such as transactions sold , Number of New Account etc by Salesman Wise. here want to prepare two data set to Transaction Data set and Account data set , final want two merge this salesman wise .

@DataMagicAI Год назад

From transaction data and accounts data find the feature which represent the unique employee.9For example it could be an employee id something). based on this employeed id merge the total transactions sole and to new account created. Again it might your data is having so many records of transaction or new account created in this case first data binning need to be done on hourly, daily or weekly basis as per your business requirement. If you are doing some realtime project contact me at datamagic2020@gmail.com I can consult you further.

@regal7548 5 месяцев назад

Sir. So whenever i download a dataset from kaggle , i get like 6 or 7 datasets included in them. Each having different variables. When i thought of removing null from a dataset, theres like a huge number of null. So i cant remove them. For example an airbnb dataset from kaggle. Kindly help me. Im stuck at the point where i dont know what to do with these datasets. 😢😢

@alndr4u 2 года назад

merge two dataframes side by side, after merging right side value shows nan. Plz help

@DataMagicAI 2 года назад

Issue is your column names are not same in both datasets

@vlogersadda7625 2 года назад

Sir. Thank you but i want to know What if merge data set if i have multiple target.

@vlogersadda7625 2 года назад

i have Heart disease Data set and Diabetes data set can i combined it in one?

@DataMagicAI 2 года назад

If you have multiple different target values it's fine but you features should be same.

@anuragfunde1569 2 года назад

this lecture was amazing one .really helps but i also wants to know can we merge two models trained on 2 different datasets into one model using multimodels concept...if yes its possible then can you please upload a video on this

@td0rmx Год назад

I also would like to know this @Data Magic (by Sunny Kusawa)

@farzifables_17 Год назад

Hello sir, i have a doubt on merging of many dataset i have six dataset then how to merge all because in same dataset the data is totally different only gender is similar then how to merge categorical data? If i merge the effect tha different columns its shows null value please tell me how to solve this kind of dataset! 0:26

@DataMagicAI Год назад

If you don’t have common fields in dataset then don’t merge. Might be dataset for same purpose but it’s parameters are all together.

@feelgoodspace Год назад

thanks a lot

@DataMagicAI Год назад

Cheers!

@soubhikbandhyopadhyay8160 Месяц назад

How to feed null data using time series ml model

@DataMagicAI Месяц назад

Plz watch out our time series analysis playlist. There are few libraries which imputes null values by them selves. You can also fill this null values with valid values.

@vineethreddy2960 Год назад

Hey, what if two dataset columns are differing entirely?

@DataMagicAI Год назад

If your target values are same in that case you can merge the both dataset features for same target values. If targets are also different then these two dataset are for different representation don’t try to merge it.

@1of999 2 года назад

Hello, I have been trying to look for datasets from Kaggle that have a common column or columns to merge and perform EDA. I have had a rough time in finding related datasets on Kaggle can you please help

@DataMagicAI 2 года назад

Below are the two height weight dataset from kaggle. You may check it out. www.kaggle.com/datasets/yersever/500-person-gender-height-weight-bodymassindex www.kaggle.com/datasets/tmcketterick/heights-and-weights Always keep in mind, whenever you are selecting common columns to merge dataset, make sure it's values at same scale. For example. If weight is in KG in one dataset then same should be another dataset. Lets assume if one dataset have weight in KG and another in Pounds. Then you need to convert both dataset weight values either to KG or Pound and then merge the dataset. Hope it will help you to understand better.

@vlogersadda7625 2 года назад

i have Heart disease Data set and Diabetes data set can i combined it in one?

@DataMagicAI 2 года назад

If both dataset have same features or at least most of the features same then you can combine thise same features.

@rahmamohammed8541 2 года назад

hello how do i combine different dataset eg i have sepsis dataset and jaundice dataset to develop a prediction model and how do i encode them after combining .some variables may not indicate the other disease how do i solve this problem please

@DataMagicAI 2 года назад

First of check whether combining these two datasets make sence to solve business problem. Is this datasets for same purpose and sharing common features....if yes then only combine it...encoding feature values is not a big deal.

@rahmamohammed8541 2 года назад

@@DataMagicAI Yes they are for same purpose to develop a prediction model ,they share some features but there are other features they don't share how do i combine then encode them in csv file ? Thank you

@ManveenKaur0902 Год назад

hi, i am doing a project on impact of climate change on indian birds. i need to combine 2 datasets , ie , climate change in india (temp,precipitation) and the second dataset would be how its affecting indian birds in those regions. do i need to combine the datasets or some other method can be used for my ml project?

@DataMagicAI Год назад

If your both dataset having common feature like date and region then you can merge it.

@Sandyyy143 9 месяцев назад

Bro i merged two data set...can i have that as a copy ??

@DataMagicAI 9 месяцев назад

Yes. Save back and store it somewhere.

@Sandyyy143 9 месяцев назад

@@DataMagicAI I merged the dataset by seeing your video...but it Is stored in a variable...now I need that dataset as an external copy...how can I get this ?