Grate video as always. If you guys are using pandas library for your data preprocessing you can use their get_dummies() function that will convert whatever categorical columns you want into one hot encoded. import pandas as pd df = pd.read_csv(“file path”) one_hot_encoded = df.get_dummies(df[“categorical column”]) It’s just a rough sketch but you can read more about it in pandas documentation. Plus if you’re let’s say using TensorFlow you can split your data into training and testing and feed it directly into network, because pandas runs on top of numpy and TensorFlow know hot to deal with numpy.
I converted my image that had 2 label into one-hot, ie (0,1,2), but when I removed the first one, because I thought it was unnecessary (redundant, it presented the background), the model did not converge... my model only converges if I send all 3, including the background... does that make sense?
Thank you for the informative tutorial. I am learning deep learning using tensorflow as backend. May I ask whether it is really necessary to use one-hot encoding in training multiclass model? Since I understood in tensorflow there are sparsecategoricalcrossentropy (for non one-hot label) and categoricalcrossentropy (for one-hot label) for loss calculation. Hence I have an impression that both type of data structures are acceptable? Or is there a large difference in training performance? Thanks again
You cannot use a vector with just numbers if you want to use categorical cross entropy as loss function for multiclass classification in deep learning. Many loss functions for multiclass problems require you to convert it to categorical (one-hot). If you are working with traditional machine learning (RF or SVM) you do not need to convert.
This is a common question that people ask me and it is really challenging as there is no easy to use tool. Please sign up for a free APEER.com account and we plan on releasing a tool soon (mid-July 2020). By the way, APEER is an online image analysis platform that is free for academia, non-profits and individuals.
