Dimensionality Reduction - The Math of Intelligence #5 

This is easier to understand if you're already familiar with PCA. This video is a summary of what i've learnt in dimensionality reduction,Thanks Siraj.

Right on time Siraj ! Just learning PCA and dimensional reduction and here you are explaining it clearly . Thanks

Oh my, why did i discover your channel only now?! What an amazing content and approach to teaching! Love it! Keep up the good work.

Siraj,Maaaaahn I am working on my FYP on recommender systems and I was using SVD for reducing dimensions.Thanks for exposing me to PCA and the fact that how this field is breakthroufh worthy...keep up the work..#Respect

i am a prodigy machine learner with python and scikit learn and 16 year old boy i learned more in this video about dimension reduction than i could possibly learn in hours i am now on way to reduce dimensions of my data and improve the accuracy thanks siraj sir i really appreciate that u r posting educational content free of cost allah bless you

enjoy your videos. Nice to see a wee regional link to the UK with your data. Hope you keep posting.

Thank you Siraj. I am an engineer from an other field. Your work helping me a lot to widen my perspective.

2 videos in 1 week. Thank you!!!

One more amazing video from Siraj. I love this content !!!

Awesome video, keep up the great work Siraj!

Thanks for your content Siraj! It is both informative and entertaining. The videos have improved dramatically as well. Could you make a video comparing and contrasting some of the different neural networks? Especially convolutional neural nets versus deep nets versus artificial nets.

this was highly informative (and entertaining) thank you Siraj

I am pausing this just to laugh at all the pictures and side stuff. This is so informative and fun. What a refreshing surprise as I am trying to research this.

keep the flavor and flourishes. it's probably why i'm subscribed. there's reading material and other sources for the dry stuff.

Thanks Siraj........PCA just got demystified for me !!!!!! grt video

This by far the simplest and THE BEST explanation of PCA on the internet!!! Thanks Siraj!! :)

since when tSNE is superevised ?

Boy this is freaking awesome!!!

Hey Siraj, from minute 4 you are adding a video from some udacity course. Which one is that? P.S. Great video(s)!!

That looks like a 3blue1brown screengrab, did you site your sources Siraj???

As someone who has worked with high-dimensional data classification, I can say that PCA is a truly powerful technique, and not at all that complicated. Another interesting technique for dimensionality reduction is "random projection", which has the cool property of closely retaining distance relationships between high-dimensional data points in the lower dimensional space (en.wikipedia.org/wiki/Random_projection)

Great videos! keep them coming...

Can you do a video of unsupervised ML to recognize specific sounds? No one on RU-vid has done one yet! Great content btw, thanks!!

The use of memes is strong with this one.

you are awesome, very nice explanation. Thanks

Stay the same Siraj, very inspirating creation and creator

Thanks Siraj for this helpful video! Can you please help us understand the "Dictionary learning" algorithms like K-SVD ?

Hie siraj... I want to ask you.. Is it possible to detect the babesia(smaller pixel) from an image. First I convert it to grayscale image then to after applying erosion morphological operation... The babesia images are enhanced but how to extract such smaller object from image.. I really don't know

thank you for Dimensionality Reduction video

Siraj a question please, how did you get the top right part of the image @ 1:19?

Would singular value decomposition instead of eigen decomposition also work?

Can PCA for predicting data? example using same eigenvector but separate data (the score pc and eigenvalue) into traning and testing, thank you... hopely you read this comment. Best regards -Brata

6:01 is from Eugene. :D credit her too. Great explaination though. thx

i am kinda struggling with one doubt…..what to do when our independent variables are of different scale…. For example i have likert scale responses and also discrete( 2 or 3 scale) responses….. is there any way to perform PCA/EFA on such mixed data?????/ Plz do reply :(

good work man

Hey Siraj, Love watching your videos,just a doubt here, while explaining feature standardization the formula you used i feel is a normalization formula(x-xmin/xmax-xmin), standardization is x-mu/sigma for mu=0 and std dev=1 ,plz clarify here

What are his main sources for the very good visualizations (especially 1:26, 2:51) ? Thanks for any tips.

WOW! No one mentioned the most important stunt in this episode: DISCARD THE OBJECT AS IF IT HAD EXCESSIVE HEAT. Ok now I dropped it :)

i think i will watch some MIT lectures from ocw on linear algebra , i think i dont really understand all the stuff on a deep level and why it works

Please do one on L1 and L2 regularization

Which is better pca or tsne?

Shots fired at php

since Siraj knows a little German now....here is an deep learning Insider: what is a Zeitüberbrueckersystem ?? challenge of the week

man .. you made it look so simple

Finally some real Meat!. I didnt get the part following eigenvector (except the intution of eigenvector), i think i gotta watch all the resource links in the description and then re-watch this video.

2:03 That is a very tall 7-year-old

3:58 In order to standardize the data (getting a distribution of mean = 0 stdv=1) couldn't we do Xi' = (Xi - avg(x))/(stdev(x)) instead of Xi' = (Xi - min(X))/(max(X) - min(X))? Are both solutions equivalent? Why choose one over the other? P.S. Thank you a lot!!! Your channel is great!!!

can u plz make a video on ICA?

can u plz make a video on svd?

شكرا جزيلا اذا امكن ارسال بحث مفصل حول تقليل الابعاد باستخدام الانحدار العكسي المنتظم (spars dimension reduction by regularzed .......)

Hi Siraj! thanks for the videos. These are really helpful. Can you please make video about, How can I train my own word embedding, like google's google word2vec. I want to make word embedding for Bangla language.

My 4D data got converted to 2D data .. but I still don't understand what algorithm it used..

liked video the moment siraj slammed php

I cant access Ong's code

as usuall you rock

Unless you use PHP!! jajajaja Siraj, you are the best

Hey Siraj, what is the best laptop for programming that will not lag and will maximize the range of applications you can create? P.S. I really love your videos.

I've been noticing that you get snippets from other channels as well. You should ask 3Blue1Brown before using his footage. Also you should name him in the video cause that guy definitely deserves attention.

I've finished the English subtitles for this video. Please review and approve! :)

Finally, had to wait out 4 videos for this main one! edit: first

at 3:33 shouldn't he be saying projecting on a plane that MINIMIZES variance?

There is a typo in the Introduction, section, under heading "PCA and Dimensionality Reduction", in the last sentence. CURRENT TEXT: "larger magnitude than others THAT the reduction..." SUGGESTED CORRECTION: "larger magnitude than others THEN the reduction..."

6:15 I think Siraj loves this woman secretly ; )

thank you

hi. nice videos.

did you play the friend of carl in TWD?

Couldn't you say, all science and mind is the effective Dimensionality Reduction of a given sensory input?

Good video but from my knowledge I don't think 6:20 t-SNE is supervised.

memes are on point but you better not steal from 3blue1brown.

The python code you have used is from a RU-vidr bhavesh bhatt's code.

wtf i was watching in the office and your rap made it look like i'm wasting time, to other colleagues.

Does that mean that a dataset with n-columns is an n-dimensional dataset?

3Blue1Brown

4:05 x' = (x-xmin)/(xmax-xmin) x1' = (115-115)/(175-115)=0 x2' = (140-115)/(175-115)=0.417 x3' = (170-115)/(175-115)=1 "That means that data should have a MEAN of 0 and a variance of 1." But here the mean is greater than 0. Mean will be 0 if the formula is: x' = (x-xMEAN)/(xmax-xmin)

4:00 he's taking about min-max scaler, and standard scaler as if it's the same thing

4:50 "Eigen" translates to "of one's own"

Perfect! Thanks

Great work ! But i understand nothing, too hard for me X)

Eigen actually roughly translates to "your own", I guess it's a shortening of "Eigenschaft" which literally translates to "characteristic". Even to a German person the term Eigenvalue or Eigenvector is absolutely non-self-explanatory. You have to have your nose put into the fact that these things characterize a matrix, unless you have a brain that just gets that from looking at the relationship of matrices to their Eigenvalue/Eigenvector. Its actually kind of funny that the first 5 minutes of your video do this better than most LA courses in university, practical application FTW, I guess.

Mate you are using heaps of screeengrabs in this video, you should put a footnote in the vid when you do

he just had to throw the php comment in there

4:57 I'm now completely convinced that you have a +400 IQ

Are we going to pretend we didn't see the 'musical' part?

Coding Challenge: github.com/ritobanrc/principal-component-analysis

Did you just rip 3Brown1Blue ???

why do you write every thing from scratch,In kaggle most grandmasters are unaware of implementation but can predict the best results

that yo mama joke cleared the fog for me, real talk. ..the boy aint right though

If you cannot plot some points using PHP you should consider giving back your degree :P

Damn bro that song kinda slapped, ngl

Hi Siraj, This is my submission for this weeks challenge: github.com/Sri-Vishnu-Kumar-K/MathOfIntelligence/tree/master/dimensionality_reduction_pca I have implemented image compression using PCA and have also depicted it on a gray-scale image of Shakira. I hope you like it. I have taken down my previous solution, because of a valid flaw in the dataset pointed out by someone. Thanks a lot for that, saved me the blues. Thanks a lot for these videos, they serve as great learning.

No Offense but instead of all the tantrums if you could have explained the maths behind it and explained how the math works that would be way too cooler.

2:38 " for every COUNTRY in the UK "

siraj dont give me hope

Hi Siraj, here's my submission for this week's challenge! github.com/jrios6/Math-of-Intelligence/tree/master/5-Principal_Component_Analysis

Siraj, Hi, I am your student. can you check up your e-mail?! I send the my thought. I want to receive your comments. thank you

do you really reply to all these comments yourself or is it an AI...!! lol

4:32 the meme lmaf

Aahahahahaha unless you use PHP :')

I think you meant to say every country in the EU 😉

starts at 1:06