Transformers, explained: Understand the model behind GPT, BERT, and T5 

Ability to break down complex topic is such an underrated super power. Amazing job.

Transformers! More than meets the eye.

How did you condense so many pieces of information in such a short time? This video is on a next level, I loved it!

This is such an informative video about transformers in machine learning! It's amazing how a type of neural network architecture can do so much, from translating text to generating computer code. I appreciate the clear explanations of the challenges with using recurrent neural networks for language analysis, and how transformers have overcome these limitations through innovations like positional encodings and self-attention. It's also fascinating to hear about BERT, a popular transformer-based model that has become a versatile tool for natural language processing in many different applications. The tips on where to find pertrained transformer models and the popular transformers Python library are super helpful for anyone looking to start using transformers in their own app. Thanks for sharing this video!

This is awesome. This has been one of the best overall breakdowns I've found. Thank you!!

Where is optimus prime?

You have the gift of making things simple to understand. Keep up the good work 🙏

Thanks you did a great job. I spent some time already looking at different videos to capture the high level idea of what transformers are about and yours is the clearest explanation. I actually do have an educational background in neutral networks but don't go around remembering every details or the state of the art today so somebody removing all the unessesary technical details like you did here is very useful.

Great explanation of the key concept of position encoding and self attention. Amazing you get the gist covered in less than 10 minutes.

This is a really awesome video! Thank you so much for simplyifying the concepts.

This is a GREAT explanation! please lower the background music next time it could really help. thanks again! awesome video

Dale you are so good at explaining this tech, thank you!

Amazing video! Nice explanation and examples 😄👍 I would like to see more videos like this and practices ones

I loved it and very simple ,clear explanation.

This has to be the best explanation so far, and by a very large margin.

I love how to simplify something so complex, thank you so much Dale, the explanation was perfect

Positional Encoding, Attention and Self Attention. That's it! Really well summarized.

Charm, intelligence and clarity! Thanks!

You have no idea how much time I potentially have saved just by reading your blog and watching this video to get me up to speed quickly on this. "Liked" this video. Thanks

Love the content and thanks for the great video! (one thing that might help is lower the background music a bit, I found myself stopping the video because I thought another app was playing music)

So easy and clear to understand. Thanks

Wow, this is so well explained.

Nice amount of info parted in this video. Very clear info on what Transformers are and what made them so great.

i really enjoyed the concepts you explained. simple to understand

Amazing video! 🎉 You explained that difficult concepts of Transformers so clearly and made it easy to understand. Thanks for all your hard work!🙌👍

This is a very well produced video. Credits to the presenter and those involved in production with the graphics

Very well explained.. This really is a high level view of what Transformers are, but it's probably enough to just get your toes wet in the field!

Very interesting, informative, this added perspective to a hyped-up landscape. I'll admit, I'm new to this, but when I hear "pretrained transformer" I didn't even think about BERT. I appreciate getting the view from 10,000 feet.

Love how you simplified it. Thank you

This is one of the best vids I've watched on this topic!

That's a really good high-level explanation!

Fantastic!. Thanks for simplifying the concept

This is an excellent video introduction for transformers.

Hi Google! First of all, thank you for this wonderful video. I'm working on a multiclass (single label) supervised learning that uses Bert for transfer learning. I've got about 10 classes and a couple hundred thousand examples. Any tips on best practices (which Bert variants to use, what order of magnitude of dropout to use if any)? I know I could do hyperparameter search but that'd probably cost more time and money than I'm comfortable with (for a prototype), so I'm looking to make the most out of my local Nvidia 3080.

This was a really, really awesome breakdown 👏🏾

Thank you so much. I really needed this video, other videos were just confusing

OMG the BEST transformers video EVER!

The visuals are very helpful. Thanks.

thank you! I'm just starting to learn about gpt and this was quite helpful, though I will have to watch it again :)

Such a simple yet revolutionary 💡idea

Excellent presentation and explanation of concepts

Thanks for your hard work.This video is very helpful!!!

super well done. Thanks for this!

Easiest to understand explaination ive heard so far

so super helpful for my thesis, thank u

Simply loved it!

Informative! Thank you

I have more respect for Google after watching this Video. Not only did they provided their engineers with the funding to research, but they also let other companies like OpenAI to use said research. And they are opening up the knowledge for the general public with these video series.

Amazing explanation!

wow, what a great summary! thanks!!!

Very good lecture, thanks!

do transformers learn the internal representation one language at a time or all of them at the same time? I remember that Chomsky said that there's no underlying structure to language and that for every rule you try to make you'll always find an edge case that contradicts the rule.

Thanks Ma'am. You broke it down well.

crazy how things have changed so much

Great video for people who are curious but don’t really want to (or can’t) understand how transformers actually work.

Great video. Thank you!

Very well explained. Thank you.

I knew little on transformers before this video. I know little on transformers after this video. But I guess in order to know some, we'll need a 2-3 hours video.

Amazing video, thank you so much!

Soo cool! Great work

Very well explained. This video is must watch for anyone who wants to demystify the latest LLM technology. Wondering if this could be made into a more generic video with a quick high-level intro on neural networks for those who aren't in the field. I bet there are millions out there who want to get a basic understanding of how ChatGPT/Bard/Claude work without an in-depth technical deep dive.

Excellent explanation i ever seen, recommending everyone's this link

Takeaways: A transformer is a type of neural network architecture that is used in natural language processing. Unlike recurrent neural networks (RNNs), which analyze language by processing words one at a time in sequential order, transformers use a combination of positional encodings, attention, and self-attention to efficiently process and analyze large sequences of text. Neural networks, Convolutional neural networks (for image analysis), Recurrent neural networks (RNNs), Positional encodings, Attention, Self-attention Neural networks: A type of model used for analyzing complicated data, such as images, videos, audio, and text. Convolutional neural networks: A type of neural network designed for image analysis. Recurrent neural networks (RNNs): A type of neural network used for text analysis that processes words one at a time in sequential order. Positional encodings: A method of storing information about word order in the data itself, rather than in the structure of the network. Attention: A mechanism used in neural networks to selectively focus on parts of the input. Self-attention: A type of attention mechanism that allows the network to focus on different parts of the input simultaneously. Neural networks are like a computerized version of a human brain, that uses algorithms to analyze complex data. Convolutional neural networks are used for tasks like identifying objects in photos, similar to how a human brain processes vision. Recurrent neural networks are used for text analysis, and are like a machine trying to understand the meaning of a sentence in the same order as a human would. Positional encodings are like adding a number to each word in a sentence to remember its order, like indexing a book. Attention is like a spotlight that focuses on specific parts of the input, like a person paying attention to certain details in a conversation. Self-attention is like being able to pay attention to multiple parts of the input at the same time, like listening to multiple conversations at once.

Simplest Explanation ever

Very informative video. Thank you!

Nicely done. Very helpful. Thanks!

Thanks! This is a great intro video!

very well explained.👍

It was funny and instructive. Thanks 🙂

Super Explanation!!

Thanks! Great video.

great video, thanks!

Great video.

How did you sync your talking cadence to the background music?

Very impressive video. Thanks for the way you shared information via this video. Reference your video timeline 05:05, how you created such a video, please.

You have actually given the BEST explanation on Neural Machine Translation that I read so far but you are missing a few elements

Positional encoding = time, attention = context, self attention = thumbprint (knowledge)... looks like a good start for AGI 😀

phenomenal video

Thank you

Pretty nice, is there any automatic way of cleaning up data with errors such as a mislabel, or a grammar error?

NICE SUPERB PRESENTATION

Thanks, that was very interesting

this is brilliant

From 5:28, shouldn't it be the following: "when the model outputs the word “économique,” it’s attending heavily to both the input words “European” and “Economic.” "? For européenne, I see that it is attending only to European. Please let me know if I am missing something here. Thanks for the great video.

Great video. Thx.

Thank you for sharing

10/10. Very helpful

a very nice video. thanks

is there a deeper explaination?

When I saw this title, I was hoping to better understand the mathematical workings of transformers such as matrices and the like. Maybe you could do a follow-up video explaining mathematically how transformers work. thank you for your time

Beyond the great content and delivery, I loved your French accent ... ;)

An excellent video. I wonder if you can comment on "living the life" of a transformers user. For example, in another video by another RU-vidr I heard the sentiment that being an AI person in this era means constant - really constant - study. That may not be the lifestyle that everybody wants to adopt. I'm a retired neurologist and vice president of the faculty club at my state university. What interests me these days is how students "should" be educated in this era. And, at the end of the day, one of the critical aspects of that is matching individual human brains - with their individual proclivities - with the endless career opportunities of this era. So, I'm trying to gather perspectives (aka "data") on that topic. Maybe you could make some kind of video about it. Please do!

Wowww….thanks for clarifying my confusion.

I'll jump on where others are doing the same - would love advice for someone who understands half the concepts that are alluded to as complex naturally and the innovation feels obvious I'm unsure how to break into the space without some guidance or connection between having exactly that great natural grasp but wildly anxious that language and logic are strengths and math is a mental turn off. For someone needing that type of translation/guide where my approach is language usage & finer cues what is the key terms to get to that understanding? Hate being fascinated and all the tools to play in this space and being unable to start because how I approach topics so welcome any advice.

Thank you so much for your help. With the assistance of GPT-4, I have been able to transition from a seasonal programmer to a full-time programmer. I am truly grateful for your support!

Fantastic video

Amazing video, thank you ... can you use transformers to detect patterns in random data that which is supposedly unpredictable, like weather or stocks?

When I was a kid, I knew the trouble of translation were due to literally translation words, without contextual/ sequential awareness. I knew it's important to distinguish between synonyms. I've imagined there's a button that generate the translation output then you can highlights the you words that doesn't make sense or want improvement on it . then regenerate text translation. this type of nlp probably exist before I program my first hello world (+15y ago)!

Great content 👍

Well done