seeing this video made me feel I am not even close to say I know excel. Your understanding of concept is really really deep as implementing something like GPT-2 in excel requires one to have thorough understanding of all the concepts. Hats off to you.
Not sure what you mean by equivalent. Technically they separate. That is the the neural networks in the model (also called the multi-layer perceptron stage) have their own set of weights that are separate from the embedding weights. But both sets of weights (embeddings and neural network) are learned during the training stage (i.e. backpropagation) so in that sense they are similar. Does that answer you question?
@@Spreadsheetsareallyouneed yes you’ve answered my question. I was asking whether the embedding weights were the perceptron layer weights; they are not.
This is so amazing - best way to see, play and tweak GPT. Thanks for so much work building the spreadsheet and making sure everything works. Must not be easy task. Waiting for you to cover "Text and Position Embeddings" video now.
Fantastic! I was looking forward to this video. I really appreciate your excellent explanations, such as when you explain embeddings via an analogy of house buying. Great stuff, thank you so much for putting it together.
Very nice product. But I have always wondered why there is nothing to learn about AI in a spreadsheet. I'd also like something more minimalistic that works minimally for rapid learning. And I would like to see a diffusion-generated model product made as well.
Why does having a large Embedding Table matter? Can't it just be treated as a lookup into the Table (which should be pretty manageable regardless of size)? Do we really have to perform the actual matrix multiply ?
Incredibly brilliant. Words fail me. Thank you for sharing this, it helps me enormously in understanding AI. On a side note: How ingenious can you handle Excel? Unbelievable!
This is an incredible video. The way you describe these advanced AI concepts is awesome. I'd love to see more, maybe on the GAN technology that Sora uses
interesting shit ^^ can you help how to teach an LED 2D matrix of 100*100LED with an AI to show images that you pronounce? #ubeartform instagram my shit about it -.-
Bravo! This is great way to introduce the nuts and bolts of large language models to a lay audience. Far, far more accessible than the baggage that comes with teaching it via pytorch or tensorflow. Just simple math and a little bit of excel wizadry. Lovely idea and thank you for making these videos.
Modern Day i.e. by 2024 , spreadsheet software is way too advanced, so this ain't surprising really. Sadly i can't list the weird things people have achieved using Excel (weird but not really in a bad way & was mostly inspirational & definitely creative). Really Glad someone did this.
It is an amazing lecture you compiled in this video. Do you also have one on the training of GPT2 or how the tokens get their semantic context/meaning?
Large language models and AI in general seems to do a good job of compressing and then turning back into an approximation of the input. Is this a byproduct of nural networks in general or just specific subsets? Could you make a large language model or a lot of purpose built AI that are good for various different compression situations and more often than not perform better than current compression algorithms?
I've been waiting for someone to comment on that...It was a funny throwaway observation right after I put the slide together but who knows. Ray Kurzweil has been predicting AGI by 2029 for over two decades now.
thank you for great explanation and the spreadsheet file, yet talking about v0.6.0, would you mind to explain where is the Split_Into_Characters formula is located? because when i run the manual calculation, sheet Prompt_to_Tokens, cell (column 2, row 4) only return #NAME? error
I haven't downloaded the sample spreadsheet, but he notes in a different video on tokenisation, that it splits words into one or more tokens, not one token per word, and certainly not one per character, as these would result in a drastically larger memory footprint.
I am sorry when It sounds like a cliche but honestly this video deserve way more views! so people can understand how GPTs work! excellent beautiful work thank you.
Great demo, cant wait to see the attention part video. Btw you can reduce excel lockups by disabling autosave. I just downloaded the excel file and looks like autosave is turned on.
