The Largest Mamba LLM Experiment Just Dropped

Подписаться 150 тыс.

Просмотров 38 тыс.

50% 1

Check out HubSpot's ChatGPT at work bundle! clickhubspot.com/2os
A long awaited sequel in LLM research has appeared, AI21Labs has dropped the biggest Mamba experiment, which is on par with other open source LLM models! Just with a few twists...
Original Mamba Paper
[Paper] arxiv.org/abs/2312.00752
[Code] github.com/state-spaces/mamba
MambaFormer
[Paper] arxiv.org/pdf/2402.04248.pdf
AI21Labs
[Blog] www.ai21.com/blog/announcing-...
[Huggingface] huggingface.co/ai21labs/Jamba...
[NVIDIA NIM] nvda.ws/3Jn5pxb
VideoMamba
[Paper] arxiv.org/abs/2403.06977
[Code] github.com/OpenGVLab/VideoMamba
Special thanks to LDJ for helping out with the content in this video!
This video is supported by the kind Patrons & RU-vid Members:
🙏Andrew Lescelius, alex j, Chris LeDoux, Alex Maurice, Miguilim, Deagan, FiFaŁ, Daddy Wen, Tony Jimenez, Panther Modern, Jake Disco, Demilson Quintao, Shuhong Chen, Hongbo Men, happi nyuu nyaa, Carol Lo, Mose Sakashita, Miguel, Bandera, Gennaro Schiano, gunwoo, Ravid Freedman, Mert Seftali, Mrityunjay, Richárd Nagyfi, Timo Steiner, Henrik G Sundt, projectAnthony, Brigham Hall, Kyle Hudson, Kalila, Jef Come, Jvari Williams, Tien Tien, BIll Mangrum, owned, Janne Kytölä, SO, Richárd Nagyfi, Hector, Drexon, Claxvii 177th, Inferencer, Michael Brenner
[Discord] / discord
[Twitter] / bycloudai
[Patreon] / bycloud
[Music] Massobeats - Lush
[Profile & Banner Art] / pygm7
[Video Editor] Silas
0:00 Intro
1:16 Hubspot
2:24 Jamba
8:08 VideoMamba

Наука

Опубликовано:

3 авг 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 70

@bycloudAI 3 месяца назад

Check out HubSpot's ChatGPT at work bundle here: clickhubspot.com/2os unfortunately topping the last mamba edit is way too hard, but I guess now at least we know *_mamba is real_*

@rounaksen1683 3 месяца назад

Hove you seen google's griffin and hawk?

@vinc6966 3 месяца назад

If mamba does not scale well, we still have diffusion models for text

@thipoktham5164 3 месяца назад

Why not both?

@sascha_becker 3 месяца назад

Jamba Mamba ¡Ay, caramba!

@user-zu6wg9wt8m Месяц назад

bien dicho

@dolcruz6838 3 месяца назад

Would be interesting to see the infinite context from the "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" Paper explained.

@farrael004 3 месяца назад

Ikr. I wonder why that paper didn't get more traction

@beerbytes9895 3 месяца назад

@fireship game up your memes this boy is strapped to the teeth.

@bernard-ng 3 месяца назад

wait.... this is not a @fireship video damm

@sunshine19088 28 дней назад

close enough

@svendpai 3 месяца назад

love your memes so much

@vongolashodaime1975 3 месяца назад

Hey, would you be interested in making a video about ponydiffusion ?

@kolkoki 3 месяца назад

Isn't pony diffusion just a latent diffusion foundation model, like stable diffusion?

@vongolashodaime1975 Месяц назад

@@kolkoki I got no clue about any of that sorry, I just know that, at least back then, pony revolutionized accuracy to character LoRAs and made the generations of already existing characters so much more accurate than other checkpoints.

@RealTwiner 3 месяца назад

I dont watch this channel much, but I did see that epic mamba short in one of your videos and it has been ingrained in my mind ever since.

@drexon88 3 месяца назад

Everyone is combining models rn. Some people combine NeRF and GS and that worked as well. I guess that ML will become just a mixer for architectures at least for some commercial devs

@dsgda153 3 месяца назад

Oh god. How much of a memelord can you be?! The "can you get much higher" right after the lobotomy? I love you man.

@zzzzzzz8473 3 месяца назад

appreciate these videos . the main thing ive heard regarding mamba v transformers is that the discoveries of optimizations within transformers are still abundant , quantization alone is massive in enabling the networks to run on average hardware , and the ridiculousness of 1.56bit quantization working is incredible where as with mamba no quantization is available .

@akaanoone6939 3 месяца назад

If you enjoy RU-vid and it pays bills then sure but play it safe so you don't make life much harder than necessary. Plus you might be able to do research at the same time and present it to people in a more consumable form

@OxygenGenesis 3 месяца назад

Love your video essays, good and easy to understand and nice to catch up to SOTA methods.

@mrrespected5948 3 месяца назад

Very nice

@JackCrossSama 3 месяца назад

we need one called Mongoose

@Metruzanca 3 месяца назад

The part on Jamba honestly sounds like someone making shit up with fake words, but thats actually all real. The "Microservices" video by KRAZAM is now reality.

@tnguyen8633 3 месяца назад

dank af

@Ivan.Wright 3 месяца назад

Every time I hear Mamba I can only think of the Python CLI

@rasuru_dev 3 месяца назад

Gemma 7B competing with llama70b, mixtral, and jamba damn scale that thing up

@lobiqpidol818 3 месяца назад

Nah bro infini attention is where it's at

@edhofiko7624 3 месяца назад

so whats next? kalman filter with learned dynamic?

@OfficialNierto 3 месяца назад

could we use it through ollama?

@cvs2fan 3 месяца назад

wait a sec bycloud still makes videos? :V

@logangarcia 3 месяца назад

so is this cheaper than Mistral 7B? ❤

@diadetediotedio6918 3 месяца назад

3:36 It would still be good for people wanting small models to run on very cheap devices without needing all the quality, no?

@JorgetePanete 3 месяца назад

7:17 LLM Models live inside ATM Machines

@erickmarin6147 3 месяца назад

Im trying to write bitnet layers for Veri log

@TerrinX 3 месяца назад

The Mambaaaaaaa the Mamba is reaaaaaaaaaaaaallllllll

@jessedbrown1980 Месяц назад

Obviously. I published in December of 2023: Anchoring_Global_Security_Autonomous_Shipping_with_Mind_Reading_AI_GPT-core_and_MAMBA-_core_Agents_RAG-Fusion_AI_Communities_Hive-_AI_and_the_Human_Psyche #mindreading #AI #agent cores #Mamba2 and GPT4, 5 and sequential models #IDE

@hakimehamdouchi7468 2 месяца назад

so.... still waiting on the guff file ey?

@JosephCatrambone 3 месяца назад

Isn't mashing together RNNs and Transformers just RWKV?

@smellthel 3 месяца назад

we live in the future bros

@Kazekoge101 3 месяца назад

what happened with Hyena?

@user-fr2jc8xb9g 3 месяца назад

Man i'm tired of waiting for GPT-5 , what are they waiting for?

@VisionaryPathway 3 месяца назад

They're currently red-teaming the model

@user-fr2jc8xb9g 3 месяца назад

@@VisionaryPathway thanks for answering! How long do you think it will take until release?

@VisionaryPathway 3 месяца назад

@@user-fr2jc8xb9g personally, I think it’s releasing anytime within next 4-12 weeks (my own opinion/prediction)

@googleyoutubechannel8554 2 месяца назад

In the next improvement paper... they're going to suggest a 'hybrid architecture' where you skip the mamba layer entirely....

@annaczgli2983 3 месяца назад

Why copy Fireship's thumbnails? Sad, man.

@joshford256 3 месяца назад

There's no way you think someone can own the format of, "character on the right highlighting big text on the left"??? Thumbnails are like, the least important part of a video when you watch it as a viewer, but it's the most important part when it comes to grabbing viewers' attention. Why shouldn't you use other creators' ideas on what works, when that's not where your creative input is, and it's super important to know you have a successful thumbnail style?

@pizzadog9876 3 месяца назад

Who cares, we're here for him, not his thumbnail

@iceshadow487 3 месяца назад

He's been making these style thumbnails for 2+ years now. It's not copying, and it never will be. It's fine to take inspiration from other people when you like their work. And have you considered that he could have also just had this idea himself? It's extremely common for multiple people to have essentially the exact same idea.

@Injazz1 3 месяца назад

Thumbnails look similar because there are literally common guidelines that are proven to improve the reach of any YT video either by being more likeable to eyes or because algorithm picks them to trending tab