The moment we stopped understanding AI [AlexNet]

Подписаться 456 тыс.

Просмотров 133 тыс.

50% 1

Thanks to KiwiCo for sponsoring today's video! Go to www.kiwico.com/welchlabs and use code WELCHLABS for 50% off your first month of monthly lines and/or for 20% off your first Panda Crate.
Activation Atlas Posters!
www.welchlabs.com/resources/5...
www.welchlabs.com/resources/a...
www.welchlabs.com/resources/l...
www.welchlabs.com/resources/a...
Special thanks to the Patrons:
Juan Benet, Ross Hanson, Yan Babitski, AJ Englehardt, Alvin Khaled, Eduardo Barraza, Hitoshi Yamauchi, Jaewon Jung, Mrgoodlight, Shinichi Hayashi, Sid Sarasvati, Dominic Beaumont, Shannon Prater, Ubiquity Ventures, Matias Forti
Welch Labs
Ad free videos and exclusive perks: / welchlabs
Watch on TikTok: / welchlabs
Learn More or Contact: www.welchlabs.com/
Instagram: / welchlabs
X: / welchlabs
References
AlexNet Paper
proceedings.neurips.cc/paper_...
Original Activation Atlas Article- explore here - Great interactive Atlas! distill.pub/2019/activation-a...
Carter, et al., "Activation Atlas", Distill, 2019.
Feature Visualization Article: distill.pub/2017/feature-visu...
`Olah, et al., "Feature Visualization", Distill, 2017.`
Great LLM Explainability work: transformer-circuits.pub/2024...
Templeton, et al., "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet", Transformer Circuits Thread, 2024.
“Deep Visualization Toolbox" by Jason Yosinski video inspired many visuals:
• Deep Visualization Too...
Great LLM/GPT Intro paper
arxiv.org/pdf/2304.10557
3B1Bs GPT Videos are excellent, as always:
• Attention in transform...
• But what is a GPT? Vi...
Andrej Kerpathy's walkthrough is amazing:
• Let's build GPT: from ...
Goodfellow’s Deep Learning Book
www.deeplearningbook.org/
OpenAI’s 10,000 V100 GPU cluster (1+ exaflop) news.microsoft.com/source/fea...
GPT-3 size, etc: Language Models are Few-Shot Learners, Brown et al, 2020.
Unique token count for ChatGPT: cookbook.openai.com/examples/...
GPT-4 training size etc, speculative:
patmcguinness.substack.com/p/...
www.semianalysis.com/p/gpt-4-...
Historical Neural Network Videos
• Convolutional Network ...
• Perceptron Research fr...

Наука

Опубликовано:

30 июн 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 350

@WelchLabsVideo 2 дня назад

Thanks to KiwiCo for sponsoring today's video! Go to www.kiwico.com/welchlabs and use code WELCHLABS for 50% off your first month of monthly lines and/or for 20% off your first Panda Crate.

@samuelspace101 2 дня назад

AI today is growing exponentially, just curios, do you think we will ever hit a peak where innovation on AI will start to flat out, or hit a wall, and if so where and when do you think AI will hit its peak. You kind of skimmed over this in the end, i just wanted a bit of a longer explanation.

@KWifler День назад

Oops, I farted 4o

@michaelmangion6187 День назад

Was really keen to sign up for a crate for my daughter, but after 10 minutes of glitches on their system I just gave up. Not your fault of course, but you might want to let them know that their system is pants.

@jeffg4686 7 часов назад

could have done it with ASIC a long time ago. Just living out THEIR best life possible first...

@EdgarVerona День назад

30 years ago, I used to work with an older guy who retired from IBM. I was barely out of high school, and he used to tell me that neural networks were going to change the world once people figured out how to train them properly. He didn't live to see his dream become reality unfortunately, but he was totally right.

@spartaleonidas540 День назад

Same except the guy was at Toronto and his name was Hinton

@EdgarVerona День назад

@@spartaleonidas540 guy I knew was named David Modlin. I wonder how many folks who had their prime years in the 60's and 70's saw this coming? I wish they had been able to see it. I suppose some of them might have lived to see it. Crazy to think about.

@squamish4244 День назад

@@EdgarVerona Hinton's most important contributions came from the 80s onwards, but he has lived to see it, for one. He was working on neural nets in the 70s as a postdoc. It's all happened well within a human lifetime, is the crazy part.

@EdgarVerona День назад

@@squamish4244 Nice, that is very cool. Glad to hear he's still kicking! The guy I knew helped create handwriting recognition software in the 70's. It is crazy to think that someone could see basically the dawn of modern computing and also its progression to this crazy time we're in.

@squamish4244 День назад

@@EdgarVerona Ray Kurzweil did too, but he's obsessed with mind-uploading, cryonics and resurrecting digital copies of his father etc. which is distracting, and he has trouble with being challenged on the practical implications of his predictions. He was right about the computing revolution but he's also a very strange dude. Hinton was running circles around him recently in a debate when both of them were onstage.

@samuelspace101 2 дня назад

Most people think AI is a brand new technology, while in reality there have been studies on Computer Neural Networks all the way back in the 1940s, that's insane.

@louis-dieudonne5941 2 дня назад

But the real issue is that only now has computing power become strong enough to support everything, allowing research ideas to be realized into reality, and truly transforming these ideas into technologies with such astonishing effects.

@samuelspace101 2 дня назад

@@louis-dieudonne5941 makes you think, what are we studying now that will only be possible years in the future because of the lack of resources.

@empathogen75 День назад

It’s new in the sense that neural networks are relatively inexpensive and for the first time broadly applicable to a wide range of tasks.

@davidaugustofc2574 День назад

@@empathogen75 Its just a popularity phase, RU-vid paid for itself when it was rapidly gaining users, we'll have Adobe level subscriptions in the future.

@gljames24 День назад

@@louis-dieudonne5941Not just hardware, but data as well.

@frostebyte 2 дня назад

I really appreciate how well you communicate non-verbally despite using very little A-roll. You're expressions are clear yet natural even while reading, enunciating and employing tone, and there's no fluff; you have a neutral point for your hands to signal that there's no gesture to pay attention to. I couldn't find anything to critique in your vids if I tried but this seemed like the easiest to overlook. Thanks for every absolute banger!

@MathGPT 2 дня назад

@@frostebyte he is truly a master teacher we can all learn from

@sntslilhlpr6601 День назад

The vocal fry is annoying. A shame, because his vids are such fantastic quality otherwise. But I've literally just noped out of his vids before because it grates me so heavily. Use your lungs, my good man!

@StormTheSquid День назад

Half of these comments read like they were written by chatgpt lmao

@codycast День назад

@@sntslilhlpr6601I don’t know what “vocal fry” is but his voice doesn’t sound annoying to me.

@JorgetePanete День назад

Your*

@drhxa День назад

I've been in the field for 10 years and never had anyone describe this so clearly and visually. Brilliant, thank you!

@TheStickofWar День назад

same here (9 years)

@kellymoses8566 2 дня назад

Computers not being fast enough to make a correct algorithm practically usable reminds me of Reed-Solomon error correcting codes. They were developed in 1960 but computers were too slow for them to be practical. They went unused until 1982 when they were used in Compact Discs after computers had become fast enough.

@jimktrains0 День назад

RS codes were used on the Voyager probes in 1977. CDs were the first large scale usage. Your basic point is still true: it took a while for computers to be complex enough to use them.

@T3sl4 День назад

Bayesian models have followed a similar path; the basic idea is so fundamental as to be trivial, but actually using it in practice requires a high level (uh, I don't know what the big-O complexity is -- quadratic? worse?) of detail and thus computation to truly harness. The parameters might be trivial (individually, or conceptually), but there are so many of them for a problem of modest scale that it's only recently we've made much use of it.

@kellymoses8566 День назад

@@jimktrains0 I should have specified first wide-spread use.

@JustSayin24 День назад

That real-time kernel activation map was life-changing. If, whilst editing these videos, you've ever questioned if the vast amounts of effort are worth what amounts to a brief, 10s clip, just know that it's these moments which have stuck with me. Easy sub

@emrahe468 День назад

Amazing intro with scissor and carboards 👏

@WelchLabsVideo День назад

Thank you 😁

@siddharth-gandhi 2 дня назад

Stellar video, you’re gifted at communication. Keep at it!

@WelchLabsVideo День назад

Thank you! Will do!

@jackwhitestripe7342 День назад

sir, thank you

@optiphonic_ День назад

Your visualisations helped a few concepts click for me around the layers and activations Ive struggled to understand for years. Thanks!

@michaelala4924 2 дня назад

Awesome video! Funny how the moment we stopped understanding AI also appears to be the moment it started working lol

@andybaldman День назад

An astute observation.

@MrAvaricia День назад

It works like the brain. And like the brain, the moment the results are interesting is when they have enough oomph

@ObjectsInMotion 20 часов назад

"If the brain were so simple we could understand it, we would be so simple that we couldn't" The same is true for AI.

@SeanStClair-cr9jl День назад

It's rare to find an AI video this informative and interesting. Great pacing great focus, this is wonderful. I'm a particular fan of the sort of stop-motion / sped-up physical manipulation of papers on your desk with that overhead lighting. Very clean and engaging effect. Seeing the face-detecting kernel emerge after so few blocks was also mind-blowing!

@WelchLabsVideo День назад

Amazing - thank you!

@iccuwarn1781 День назад

Fantastic presentation on the inner workings of machine learning!

@WelchLabsVideo День назад

Glad you enjoyed it!

@michaelm358 2 дня назад

Very clever and nice vizualisations! Excellent as usual. Thank you!

@machinate 2 дня назад

hehe, "hotdog / not hotdog".

@CollinReinking 2 дня назад

The amount of work you must put into videos is mind boggling. Thank you for making them.

@tommartens1722 День назад

Fantastic video. I appreciate the time spent to create it

@ben9089 2 дня назад

This was an incredible introduction in just 18 minutes. I continue to be blown away by this channel.

@WelchLabsVideo День назад

Woohoo!

@alexandredalban5743 День назад

Excellent visualisations, super easy to understand, great vid!

@josemacalintal1915 8 часов назад

This was such a good video! Love the visualizations!

@alexvisan7622 День назад

Wow, so much effort has been put into the animations. Subscribed.

@wii3willRule День назад

This is amazing! Excellent presentation, this also illuminated for me what exactly a "latent space" is. Thank you!

@BunniesAI День назад

Really well thought out and shown. A lot of work went into this video 🙏🏻

@FranciscoJose-cu1jj День назад

Man what a cool video. I really love the topic and you make it seem so tangible. Congrats man

@alexeystolpovskiy7862 6 часов назад

Wow, what a nice and insightful video! Effects are awesome! TU so much

@JacobBumgarner День назад

Incredible video. Thanks for making this!

@Aofekiko День назад

The visual aid in this video is unlike I've seen anywhere else, it really helps grasp the ideas presented easily, wonderful video!

@blarvinius День назад

Really cool explanation and video! Subscribed! ❤

@VinMan-ql1yu День назад

very nice video with a lot of efforts put into it. Thanks very much.

@CheapoPremio 22 часа назад

What an amazing video, especially the 7-10min part. Just amazing.

@manic-pixie 2 дня назад

I was literally talking to my roommate about this last night!! Thanks for the excellent video

@vassilisworld 2 дня назад

Very beautiful. I loved the music background also, specially at the end!!

@WelchLabsVideo День назад

Thank you!

@FredPauling День назад

Great video. Super clear and concise. Well done

@JackFou День назад

This has to be one of the best and easy to follow explanations of neural networks operate. Just found out about this channel and I'm absolutely loving your videos, keep up the great work!

@user-iv9jg7lm3n 2 дня назад

Love this video. First one where I finally understand how gpt-4 works. Thank you.

@TheManinBlack9054 17 часов назад

Great video! Very cleverly and clearly explained!

@robert75019 День назад

You were there 7 years ago when i started my AI journey in images….back at it when i start in language. Truly great work….love your content 👏👏👏

@sergiograslopez986 Час назад

Impressive with the high quality of video and explanation, congrats!

@anakimluke 12 часов назад

whoa that example of you rotating the figure was really top notch!! I'd never seen it in real time before!

@alessi4249 2 дня назад

The amount of work that went into that visualisation i would love a behind the scenes video!

@WelchLabsVideo День назад

I have some on TikTok!

@qorje 19 часов назад

great video, great visualizations!

@j.503 День назад

Excellent video! Thanks.

@alegian7934 День назад

i really like the direction this channel has taken the last few months! keep it up :)

@WelchLabsVideo День назад

Thanks, will do!

@AvastarBin День назад

This was very well explained. And the fact that you took all the images of the video and put it in a nn just to show a visualisation for a few seconds shows how much you care about the quality of your videos

@nickoftricks День назад

Wow, this video was amazing! It helped me understand nuances of ML I hadn't yet grasped. In particular, the explanation of the filters through their use of the dot product as similarity maps. It's one of those things that seem obvious with hindsight, but require keen insight to find and explain!

@WelchLabsVideo День назад

Woohoo!

@jackinsights День назад

I really respect the effort you put into this RU-vid video! It is one of the most simply explained videos on genAI I've seen.

@WelchLabsVideo День назад

Glad it was helpful!

@MikeyMobes День назад

your library is SICK! Literally goals

@anuragpranav День назад

amazing video. thank you for your hard work

@DavidAspden День назад

You nailed it with this one. I'd love to see how much of your video was 'effect' and how much was real computation and composition. Seeing the third layer change with the video on that angle was very impressive.

@WelchLabsVideo День назад

I try to "show the real thing" whenever I can, only thing that was really "effect" i think was showing the 96 kernels learning. I thought about actually doing a training run, but didn't have the time!

@DavidAspden День назад

@@WelchLabsVideo thanks for that honest insight. I'd put this above Grant's effort, which is a rock solid series. I really enjoyed the whole pace and b roll inserts of historical research.

@benjaminburt4285 День назад

All your videos are amazing!

@pujaromulus5013 День назад

That nearest neighbor inspection of layer(n-1) output, is also utilized in Word2Vec a year later

@Darakkis День назад

Outstanding video!

@donelson52 День назад

Fabulous. Thank you

@DGVFX День назад

One of the best visual explanations out there!

@Larock-wu1uu День назад

This video is amazing! Incredibly well explained!

@WelchLabsVideo День назад

Glad you liked it!

@gbking911 День назад

Made my day. Exceptional video

@12rjig13gb День назад

superb delivery

@CarletonTorpin День назад

Your presentation of the visuals was engaging from start to finish. Thank you for using analog instruments to explain this digital phenomenon.

@shaunregenbaum 7 часов назад

This was a seriously fantastic video :)

@rexguo День назад

Thank you for making such an excellent video to explain a non trivial idea. Saves many of us hours otherwise.

@WelchLabsVideo День назад

Glad it was helpful!

@kewpietonkatsu День назад

really good explanation!

@agustinbs День назад

PLEASE we need a second part, now i understand well the embedings space!

@hapineapple День назад

Great Work!!

@peacekeeper9687 День назад

I admire the effort you put in you videos 👌🏻

@jacopomorganti404 2 дня назад

Excellent content, as usual!

@GoldenBeholden День назад

5:10 is such an incredible visualisation. I haven't seen anyone show the kernels like this before in a video.

@obiwanpez 12 часов назад

2:05 - I am reminded of the old Scout campfire game / skit “The Three-Headed Monster,” where three people are put on stage and have to say ONLY the next one word in the sentence they are constructing, each taking turns to fill in the next blank. The person running the skit will ask a question like, “where are you from?” and the three people forming the monster have to develop a clear (hopefully) answer on the fly, like “over… the… mountain… by… the… lake… next… question… please!”

@Jay-uh3no День назад

The sheer amount of effort it would take to make those animations is mind-blowing

@pvanukoff День назад

Excellent work sir.

@seriousbusiness2293 День назад

Perfect mix of engaging and informative. And im not saying that losely, compared to other creators you actually showed in depth workings of the networks that dont always get mentioned but you also managed to make it understandable for novices, i could really see that with your visuals you set up. subbed

@WelchLabsVideo День назад

woohoo!

@morteneriksen4589 6 часов назад

The production quality of your videos nowadays is unbelievably good. The graphics, animations, transitions, and how you tie everything to the script is just massively impressive, way better even than almost all mainstream professional educational and documentary productions. (And the content is mindblowing, but that goes without saying.)

@timh2859 День назад

very good video. learned a lot

@hussainsalih3520 День назад

amazing work , keep doing

@Kutsushita_yukino День назад

great vid. deserves more views

@AdvantestInc 7 часов назад

This video is a fantastic resource for anyone interested in AI. Your ability to explain the intricate workings of AlexNet and GPT is commendable. Keep up the great work!

@YATENDRA3192 День назад

Wow you earn a like and subscribe to explaining it in simple and beautiful way the effort you have put in this video is amazing

@leptir1 День назад

What a a fantastic video. Definitely going to be one of the top/first ones I share onwards. I would love to learn from you how you made the animations and video edits (first layer viz for example) for your video, efficiently. In any case you edited this and scripted it so well it surpasses so many other similarly geared videos. I hope you keep gaining success (and sponsors :))

@zaringers День назад

Thank you for this video..!

@lanzer22 День назад

This is a really helpful video that warrant a second and third watch as you understand more about AI.

@rotors_taker_0h 14 часов назад

Great video. The only nitpick is with title: we haven't stopped understanding AI at AlexNet (and video clearly shows that we only getting better at understanding since that moment), we finally had working "AI" starting from AlexNet. All those "expert handcrafted" AIs before were no simpler to understand (if not harder) despite being handcrafted. And they largely didn't work and it was very hard to understand why.

@jpphoton День назад

excellent synopsis and technical breakdown. how I don know bout dis before mon

@user-sf2fd1nk6i День назад

Looks promising! Can't wait to learn more about the AI...

@user-yg3vv3zp1l 5 часов назад

top notch vid. ... the chart comparing 1998 and 2023 compute is humbling. it shows why so few took PDPs seriously when i was in grad school. 😮

@martin.winkelhofer День назад

This video made an exceptional impact on my understanding of neural networks and AI. Thank you! And I think I have an explanation why - maybe that's because I'm a "visually thinking" guy and your videos are EXCEPTIONALLY GOOD at visualizing/showing ideas and concepts. Images and clips you're using are "real simulations", which are harder to produce but they convey so much richer and straightforward "description of reality". There's great value in it. Thanks for your effort you put into this - how you've used real simulations (you, holding the square, ...) taken from real version of AlexNet.

@kaustubhpandey1395 День назад

I didn't know all this. Very interesting ❤

@raxirex6443 Час назад

A math professor of mine actually worked on many of the papers coming out of AI lab at MIT and he also worked on AI to play Minecraft. At the time it was really interesting to me as a sophomore, many years after I can write my own GPT, how the times haves changed!

@nickshupe8375 День назад

I understand so much more now about neural nets then I did before! And I've been trying to learn about them for years now (albeit very casually)

@seeranos 22 часа назад

The last image of the Activation Atlases side by side really highlights how they look like land masses, one like Iceland, and the other like Africa

@TreesPlease42 4 часа назад

Seamless transitions through a latent space. It allows free exploration and curiosity!

@MichelGuo 14 часов назад

holy shit the production quality of this video is absolutely off the charts.

@stevechance150 День назад

This is an exciting time to be alive!

@adityamwagh День назад

I have been working with CNNs for the last 6 years. This is hands down the most beautiful video I have seen about CNNs. Great job! 🎉

@amarokorama 2 дня назад

Great video, insightful visualizations! Regarding your footnote at 6:15, though: the myth that mantis shrimp have great color vision has been debunked long ago. They're worse at it than we are. Just because they have many photoreceptor types doesn't mean they combine them in a way similar to humans or other animals. Shouldn't have been too surprising, given their lack of brainpower.

@kellymoses8566 2 дня назад

Yes. Human vision takes up a huge part of our brain.

@TheRevenant-pn2xi 2 дня назад

Very beautiful explanation to AI

@kellymoses8566 2 дня назад

The 3D visualizations of the neural network activation is incredible. What did you use to do it?

@WelchLabsVideo День назад

Really crappy VPython code I wrote.

@jvo1464 День назад

@@WelchLabsVideo it's not crappy if it works!

@eduardotijerina958 21 час назад

Wow really well explained

@maxuix2 День назад

great video. feature visualisation bit is the creepiest thing ive seen in a while. nightmare fuel

@stratfanstl 2 дня назад

Fantastic visualizations. It is very appropriate to try to think through this transformation process as you illustrate to first see how the algorithm first reorganizes info as we perceive it into info optimized for the algorithm to recursively refine. Once you see this first iteration, then "lose sight" of the next abstraction, it becomes apparent how impossible it will be for any human to identify and correct a "flaw" in an AI model. The only approach for "correcting" a flaw in "learned data" is to somehow feed the AI more data. That assumes an imperfect system WON'T become MORE imperfect by consuming more input. This defies logic.

@backwashjoe7864 2 дня назад

How does that defy logic?

@stratfanstl 2 дня назад

@@backwashjoe7864 Round #1 of the example showed that the algorithm is capble of creating flawed "links" or probabilities that lead to "incorrect" information being spit out for a given set of inputs. All of the inputs processsed in round #1 aren't "right" or "wrong," they just ARE. If the solution to (data)===> (partially incorrect output) is to feed more data in, there's no reason to expect round #2 to ELIMINATE the type of probabistic mistake encontered in round #1. It might REDUCE it but NEW errors can creep in, creating new errors in output, either for the original topic or some other prompt given the system.