Word Embeddings 

This is so relevant in 2023, timeless explanation!

You spent considerable no of minutes to explain the nature of the problem before presenting the solution to the problem - I like that approach.

You're a genius. I was struggling to understand the word embedding concept through Stanford University videos. Your explanation and pace is much better. Thanks. Keep making such videos.

Dude, seriously, thank you. I am not a CS major but I got absurdly interested in AI during my last years of college. It's inspiring to see your self-learnt enthusiasm and it gives me strength to follow my interest. Keep up the good work and thank you! Cheers from Buenos Aires, Argentina

Glad to have come across this while looking for materials to learn about word embeddings to understand how text prompts work in stable diffusion text to image models in 2023. You're a great teacher. A lot of videos on this topic across yt is full of jargons without clear explanation. in 2023 there's still tons of relevance of this video with where the current state of technology is

You are very good at explaining pretty complex concepts. I appreciate the time you took to do this, it was very informative.

I recently started learning NLP, and the part of word embeddings was just not clear at all. It makes sense now. Thank you a lot ! You won a subscriber

good lecture, includes important information like what he finds cool, that it’s the best example what can be done etc. that allows the listener to organize information hierarchically. not like some university lectures, where just a monolithic dump of text is flowing out and we don’t know what is important to remember and what not.

More generally, word2vec is nothing more than graph2vec. Sentences can be seen as random walks in the english-language graph, in which each word is a node and every world is connected to other words. The strength of this connections depend on how frequent they appear on the same context. Seen this as graphs allow you to run network analysis and see what other kinds of information you can extract from it. By doing it right, you might even be able to estimate the connections for words that didn't appear in the training set and try to update the model to make it better. Or use the word embedding and try to embed sentences and see how that goes.

True genius knows how to explain a complex concept in a really simple and intuitive way. Solut.

For someone who knows nothing and wants to know everything in ML, this is the simply awesome explanation.

I honestly think this is a really good explanation to word embedding. Breaks the nitty gritty involved in word2vec and co-occurrence. I’ve read a couple article and watched a few videos but by far yours is the easiest to comprehend. Thank you so much. Cheers from Malaysia!

Fun fact: This guy went on to work at OpenAI and is one of the creators of DALL-E 2.

I've gone through several embedding explanations until I arrived at this one. Well done, finally all pieces make sense. Thanks!

An Exceptional Lecture! What I loved about it is its focus on implementation and results which really helps build good intution around it, to garner interest and dig deeper into math details later which most us are boggled before knowing how powerful/useful the results are. Thank you again.

Humble, concise, brilliant... Subscribed!

8:08 Thank you! This explained the missing piece to me. Multiple other videos on that topic where missing this nice and easy to understand diagram.

I'm glad you made this video. Now I don't have to download the vector for every word on twitter.

This is really well explained. Best word2vec explanation ive seen on youtube so far.

I haven't seen a better explanation on this. Thank you. This was really good.

still in 2022 much clearer explanation than my professor on my master.

Probably the most clear explanation of word2vec. Thanks for the video.

one word, awesome. Thanks a lot for the video. Humble, concise, brilliant... Subscribed! Why have you stopped making videos man. Miss your vids

I just started learning ML concepts, and this video helped tremendously with word embeddings. You got another subscriber. I do have a question though. In your example, how did you decide on 64 as the other dimension of your factored matrices? Would a larger number there give you a better word embedding?

i really appreciated the emphasis you gave on word embeddings rather than on word2vec.

Glad to see a relatively new video from you guys. I'm an old-time fan. And this is exactly what I'm attempting to learn right now.

Im new in AI, but impressed by what happens now and this hype around NLP and deep learning. Im self learning about all this. I really like what you have done, downloading tweets and word embedding. Is it possuble to access to your source code Tks

amazing man... i was going through 10's and 10's of online respurces to understand what is word embeddings! They way you explained it made me directly subscribe to your channel... very well. thank you very much

Temperature: 5 years ago Feels like 15 years ago

Awesome explanation , after hours of looking for good stuff to understand the word embeddings. Thanks !

This is much better explanation than that available on some of the paid courses.

The app is still up, impressive.

Great video. I was having difficulty understand 'context'. You explained clearly. Thank you.

I don't get, why all the dislikes, the explanation is great.

this is really intuitive. most places I got answers like representing words with vectors which wasn't helpful. now i think i have a proper idea.

HI Macheads, I have ran into so many hopelessly disappointing video presenters or live 'trainer" who just talk to themselves. Evenworse they make me confused about areas i already know and make you even confused about things you know. haha (even from the top IT corporations). Hope you can talk in our upcoming training session - just to demonstrate what is a quality presentation of tech ideas. You have inborn ability to explain and motivate

Excellent work. I was looking for such an explanation on word embeddings & I am happy that I found this. Thanks.

Just starting out to venture into this world. Thanks for the explanation. I have a medical background, but no background on computer science. So this gives me a little bit hope in learning totally something new.

Thank you so much for your video! It's the best learning material I can find on youtube.

I liked the way you explained...crisp and clear!

I just learning embedding layers and luckily I ran into this video.

Great Explanation!

I wish I could understand word embeddings just as well as you, im still a begginer as for now but this is what I want to become. Also I share your love for WE specially because I want to develop a NLP or a language model that generates answers but its too ambitious at the moment

Your explanation was very clear and simple, thanks for making this video.

very well explained imo

This is gold! thank you (from the future :-))

Awesome explanation, thanks.

Thanks for sharing! It helped a lot!

Man you are my teacher from now onwards.

Consider changing the tags and title of this video to something more like "Word2Vec uses" or something along those lines because This is the specific video I searched for but it was quite far down in the search

omg you look kinda like hiccup from how to train your dragon xD also thank you so much you literally just saved my life

I really appreciate this video sir !

Oh Man u are excellent in explaining word2vec, hoping to see some more in text NLP.

That was an amazing explanation. Thank you!

This was brilliant. Keep on creating.

Awesome video man. Extremely well presented. I'm impressed with your presentation skills. thanks for the video

Informative, interesting. Raised my interest in the topic of word embedding

wtf happened to this guy. his explanations are gold

The website is really cool🔥

Great explanation!

Beautiful, great work

Amazing man. Thank you for your time!

I think the famous calculation example is King - man + woman = queen Update: ah I see, you where referring to the example image on the glove website. Anyway interesting stuff will take a look at glove as well. Currently trying to see if I can make a resumé / job listing similarity model. I guess I can do something with cos distance of terms in the resumé to terms in the job listing. Ideas are welcome

Brilliant explanation! Thank you!

Awesome recap !

I'm wondering if something similar to Disentangled Variational Autoencoding could be used to improve a word2vec embedding. I'm not quite sure on the details, but it seems DVA has an effect similar to stuff like factor analysis, and principal component analysis, producing a latent space whose dimensions are more akin to real "separate" dimensions. Aka, producing separate dimensions for the italic-ness and boldness of a written digit, as seen in the paper Disentangled Variational Auto-Encoder for Semi-supervised Learning by Yang Li et al. (I would link it but RU-vid has a history of assuming comments with links in them are spam.) If that technique translates well into word vectors, it could for example result in a model where "maleness" is its own dimensions. i.e. "man" - "woman" ~= "king" - "queen" ~= (0, 0, 0, 1, 0, 0, ...) (A large vector that is parallel to one of the dimensions.) Another interesting venture would be to pre-process the data using an NLP library, so that different forms of the same lemma are already grouped together by default, and so that homographs can be separated. It could also expose information that a skip-gram or bag-of-words model would miss, such as dependency / sentence structure. I really ought to get my hands dirty some time instead of just thinking about this stuff in my head...

Why have you stopped making videos man. Miss your vids

great video!

Great explanation!!!

Great explanation!! Thanks a lot

Very nice explanation, thanks

Is this really how word embeddings work? Like, I'm blown away that this complex thought of what word embeddings were are just a word co-occurrence matrix decomposed into a smaller matrix. Is there more to it than that or is that the gist? More specifically, take text (documents), transform into tfidf 1-word token. Do dot product on tfifd matrix to get a square matrix (co-occurrence matrix). Take that and decompose it to say 64 components. Each of those rows are your word-vectors? The entire matrix is the word embedding at that point?

amazing video bro

Thanks man ! good explanation

Just checked out the web app and was wondering how you put that together? Django? It is brilliant!

Sir, I have one query that what exactly the output layer predicts? The embedding of the input word or the context of input word?

Can you just decompose a square matrix like this? The square matrix has 10^10 entries but the two decomposed matrices only have in total 1.28*10^7 unknowns, so this means you have 10^10 equations to solve for 1.28*10^7 unknowns, and you won't get a solution to fulfill all entries in the original matrix. How do you deal with this?

Brilliant video! Thanks :)

Thanks for the video. I have seen GloVe models that contain the stop words and basically it means that at least they do not remove stop words. I assume one can remove tham and create two different model or vectors. If so, I assume there is not that much space to talk about precision of the two approach. Right? The other thing is that in case of embedding we do not apply stemming or other similar techniques, since the process is largely on context level. Right?

Wow Great Video.

Very helpful!!!

If i understood well. You have to multiply the first weight matrix (between input layer and hidden layer) with all word vector (composed of 0 and 1) to obtain the embedding matrix ? (weight matrix obtained with BackWard Propagation etc... ?) Thanks :)

Hey how long did it take to download all that twitter data? Also, did you run it in the cloud or directly on your mac and if you ran it in the cloud, what service provider did u use? Thanks for your videos

nice video, thanks

Am a regular follower.. Keep up!!

Hello, do you invested time on learning all Calculus things like doing some partial derivative by hand or you can abstract this? I'm kind of slipping when I see mathematical content... but I'm able to build a CNN on Tensorflow without problems. Are this painfull way?

Amazing explanantion brother thank you

You are wrong. Today is the most beautiful color.

That was pretty awesome. It would be nice to see some code.

Steve Jobs, was that you?

very well explained word2vec Vs glove, thank you.

Great Job!

Thank you very much. How can we decide number of features in embedding matrix? When you deal with colors you have 3 features (R,G,B) but how can i define features for a 10000 words corpus? Thanks in advance.

Hey, do you get many job offers? Your github is phenomenal and you have a fair sized youtube channel with tons of videos, so I'd imagine tons of companies would love to hire you (just a question out of curiosity)

subscribed after 1 minute!

good job

def of word embeddings in one sentence: 4:21. Thanks for the cool tool and explanation. TU.

Superb.

Greetings from a GPT-4 future! 👋

So far I was able to only use normal dtm tfidf to compare speeches given at un - using text2vec in R. Not sure if I used it correctly though. It has an option for glove too, which I want to try out because I am a beginner and started in R. Python is soo confusing for me and do not let me start on gensin packages and docs.