How to Clean Up Pandas Data and Fix Missing Data 

Here we should how to clean up Pandas data, in particular what do with about missing data, as well as what to do with invalid data.
Consider a survey. If you send people a survey you cannot control in all cases what questions they will answer. And you cannot anticipate what data they might put that is invalid. For example, they might leave off some data. Or they might enter a number in a question that is only supposed to be text.
So here we show you how to :
1. Generate some random data that is purposefully not clean
2. drop duplicate rows
3. get rid of rows that have missing values
4. convert missing values to something else, like a fixed value of the average of all the other values in the column
5. delete outliers, which are obvious typos. For example, here we enter some salaries as 1 million while everyone else is around 100,000. So that's most likely a mistake (since these are employees and not company owners or the CEO).
6. apply a custom function to every row to do whatever special checking you want
check for missing values
7.show different ways to check for numbers or strings and how to convert those when they are of the wrong type
Code is here.
github.com/wer...