The previously defined ‘scaler’ (for MaxAbsScaler()) was used for standard scaler transformation. The scaler for StandardScaler() was defined as ‘scalar’ Note the typo
Thank you for the comment. Trying to understand the suggestion, below is description on Wikipedia: "CSC is similar to CSR except that values are read first by column, a row index is stored for each value, and column pointers are stored." Link: en.wikipedia.org/wiki/Sparse_matrix
Hi Sir, can we say that StandardScalar() method is more suitable for dense matrix. As the average will be representative of the datapoints. Whereas in sparce data due to null values .. using mean to scale will be misleading... Is my thought process right ?
While sklearn.preprocessing.StandardScaler() does offer options to use it on sparse matrices, choosing the right scaler would depend on your data and the requirements of your specific machine learning problem "scikit-learn: z = (x - u) / s This scaler can also be applied to sparse CSR or CSC matrices by passing with_mean=False to avoid breaking the sparsity structure of the data." Having said that, your thought process is on the right track! StandardScaler (not StandardScalar) can be more suitable for dense matrices as it assumes that the data is normally distributed and uses the mean and standard deviation for scaling. This works well when you have a significant amount of data and the null or missing values are minimal. However, in sparse data with many null values, using the mean for scaling could be misleading because the null values are treated as zeros, affecting the mean and potentially skewing the scaling process. Hope it helps! Thanks for watching.
