LLaMA 405b is here! Open-source is now FRONTIER! 

I already tested 405b with my LLM Rubric, how do you think it did? 😉

Economics have been sounding off on just how bad they think the next downturn might be. I need ideas and advice on what investments to make to set myself up for retirement

Meta is the last company I would have imagined doing this.

looks like zuckerbergs ethernet cables are leaking light... and giving him a tan

Zucc is looking more and more like a surfer bro

Facebook, who stole our data, are now giving it back, so I'd say we're even.

Zucks Ai implant is making him seem more human lately

Zuc definitely hopped on the psychedelics or ketamine with Elon or something. He's evolving

💥 A giant leap for the Open Source community. Many good products will come from it. 🎉❤❤❤

Meta is democratizing the use of AI. Amazing. Greetings from Argentina

Never thought I'd be rooting for the Zucc. This is awesome. Can't wait to try it out.

The synthetic data can be used to train a small model to be a specialist at a specific set of related tasks. Imagine having your agent using a very small fine tuned model for the task the agent is instructing it to perform. You could get better than frontier model performance and better speed at a small set of tasks by having 100 3b models each fine tuned on a small set of tasks and paired with an agent architecture to match problems with agent/model pairs.

just please adjust your questions, now LLMs are trained to answer questions like "code snake game in python". You need to give harder questions, like "code chess game in python" or "code go game in python"

I'm favoured, $27K every week! I can now give back to the locals in my community and also support God's work and the church. God bless America.

Zucc became a legend🙌 totally changed my mind about him 😊

I'm curious how much computational power is needed to support this model. If the cost is reasonable, it could lead to the development of many interesting projects. Meta has truly become the ambassadors of open-source AI, unlike OpenAI.

When I heard "16,000 H100 GPUs" I started freaking out like Doc Brown going, "1.21 GIGAWATTS!?"

Same, i would never thought i would jave thought Zuck would have play such a fair game, but so far he did and im happy to change my mind. Also, Merci Yann LeCun!

Zucc's move here is intelligent. Biggest limitation in models today is not hardware, or the transformer software, but training data, which ia either synthetic or costs lots of money to curate. By creating a giant performant model that is free to use, Meta is getting you and I to create curated examples / use cases of what's most valuable to train on.

Cool but, I'm gonna need an extra 50K to run this bad boy. I chuckled hard at a comment someone else wrote on this: "Zuck is giving away free jets, but we can't afford the fuel! 😄"

Well, I remember asking ChatGPT 3.5 a question when it first came out. Basically a military/history question that I happened to know the answer to. Not only did it fail miserably, it just invented facts for an answer. When I asked it to cite its sources, it invented them too. Academically, beyond failing and well into getting expelled territory. Just tried it now with this model. It nailed it. Oh yes, things are improving rather quickly. Oh yes.

zucc's redemption arc

knight x b6! your move openai

This new version of Zuckerberg is very human

Everyone* in the future should have a well trained ai. *Anyone with a nuke bunker.

Am I starting to like Meta!? Thank you Zuck, and you Mat! ❤

The best news of the day is the 128k context window. The new 8B, if it's even close to Gemma2 9B,it would be a great model. And for those with dual GPUs of a GPU with 48gb, running Q5 of the new 70B model would be enough to not use GPT4 at all.

Im not noticing a big difference between 3.1 70B and 405B - fractions better. Is this to be expected? Are we at the upper limits of throwing parameters at the problem?

This actually pretty awesome! 1. Facebook = steals data 2. Uses data for FREE quality AI 3. Gives 'data' back to everyone

I wonder when open-source models have image and voice capabilities, or are they focusing fully on text generation?

Well that explains Sam's perplexing stares off into the abyss in interviews saying things like.. I'm very worried right now

Rest of the closed source AI companies are like 'What the F>>>>??'

Didn't believe i ever would say this: Thanks Zucc

Compared the 3.1 8B with the 3 70B, the new smaller 8B model is nearly in the same level as his bigger uncle.

What a great day. Zuck needs to keep the chain and wafro, it's working to make him more gnarly and rad. Seriously thank God for Yan Lecun

Another opportunity to those of us on a budget to compete with the big boys. Never thought I would say this but …Thanks Zuc

I wonder when the student becomes the teacher. One day the small models may be able to generate, retrieve and filter relevant data for bigger models.

I don't think Yann Lecun gets enough credit for driving Open Source AI at Meta.

If something is free, you are the product.

Crazy that it got Snake right on first try. In a year we're gonna need to benchmark with Doom or something.

Crazy that Meta is more open than OpenAI 😂

I tried the 3.1 8b for coding small plug-ins and it blew my mind. Imagine how good is the 405b model !!!

If you rephrase the number question it will get it right. It was answering based on 9.11 being higher than 9.9 like a version number. If you instead ask: "Which is a larger number? 9.11 or 9.9" then it gets it correct and explains why. If you ask it any other way then it gets it incredibly wrong.

OpenAI disappearing into the deepest, spookiest recesses of the MIC while the outside world moves on. Fare thee well, Sam!

wow NICE!!!!!! ai is awesome! and nice to see meta actually doing something positive for a change . i love llama 3 its pretty detailed for its smaller size at 8b downloaded 3.1 8b now!

wow! native 128k context? that's fkn awesome! i have a hacked llama 3 with 32k and i already think that's more than i need ever, but it doesn't work that well, it gets dumb the longer the context gets. if it's native, it's not going to suffer from this. can't wait for the quantized uncensored models to drop

Did I miss something? Everyone keeps comparing it to GPT 4o, when Claude 3.5 Sonnet is by far the leading model in the world right now. I know they're both right up there but 4o is undeniably second place.

Based Adam Corolla. I tried the 70b model on my 10th gen i9, it works but very slowly. Half a second for a token. I can't even imagine what is needed for the 405B model.

A giant win for open source community!

I installed both the 8B and 70B models on my MacBook Pro M1 Max with 64GB RAM. The 8B model runs super fast and is pretty amazing considering the memory footprint of only 4.7GB. (Not sure exactly how much space it takes in RAM, but that was the download size) The 70B runs MUCH slower and the fans kick in, 40GB download. Not sure if I can see enough improvement to warrant using it instead of zippy 8B.

So happy to see Meta doing this!

I said it in another channel about how crucial it has been to me as a developer to have a local free llm to use while I build web applications that use OpenAi or Claude api when deployed (until I can get better at hosting Ollama 😊)

Zuck actually lived long enough to see himself become a hero again.

Thank you zucc 🙏🙏

Yes, that is a watershed moment ...

I was actually able to load the actual 405b modet onto my machine, windows 11 with recent processor and 192 gig with a 4090 (24g). I was mostly curious to see if it would even load and if so would it run and believe it or not it actually did run using ollama locally. The reply to a question "are you there" came back many minutes later one word at a time with a minute or two between each word. Just thought it was interesting even though pretty darn impractical without a much more expensive GPU with more dedicated VRAM. I think you said that del provided you with their machine with two large Nvidia cards ... sweet. it would be so great to have such a machine

Great job explaining LLaMA 405b! Your clear breakdown made the tech accessible to all. Thanks for sharing your knowledge and enthusiasm!

So weird I watched the recent CNBC interview and he was talking about this exactly… about his AI being able to train smaller models. This is Great to see. Thanks Matthew B.!

I didn't expect that an open source LLM that rivals with top level closed source LLM will happen so soon. Amazing job and decision by Meta and Mark Zuckerberg

I don't understand synthetic data. Won't synthetic data be full of inaccuracies, and ultimately higher hallucinations?

I'm so confused, how do you have 96gb of vram? Can you explain your pc or build I would like to know how you are doing that. Thanks.

I'm even more concerned about censorship with it.

mark is actually becoming cooler and cooler as this goes on

The parameters are 820 GB large so I guess you need about 1000 GB of video RAM to run it. That would be like 12 H100 in a cluster.

Every time you test, you ask for it to write the snake game in python... assuming correctly that it knows what the snake game is, because it has been in the training data. Wouldn't is be better to ask it to write specs for the snake game, and then ask it to write the game from the spec to see if it works as expected...

Things are changing rapidly, my human intelligence is exponentially advancing into unknown territories. Great coverage and comment. 🚀

I can't believe people aren't even shocked

Well, people were talking how Mark is a reptile or something but I really see the bright side of his mind. Well, he may have a business agenda but that's the way, bro! Props for boosting the open source community. We all have to remember our roots. If someone does not know Mark was pouring billions and billions of dollars into open source tech for the last 15 years. It is not only the latest LLAMA. This guy really deserves respect!

You know the world is fcked, when the zuck is the good guy.

Zucky has achieved a new arc era

Heck yeah Matt, this is huge. Your coverage did it justice, as always. Thanks

It was a fun chat discussing using bubbles of space-time to move smaller black holes en masse to manipulate a larger one

larger modle comes out, the smaller ones go through heavy testing and refining through wide use

1) What kind of hardware do I need to own to run the full-size model at full speed? 2) What kind of hardware do I need to refine it further? (speed doesn't matter much there)

I've been trying it out, seems like it needs some more post train finetuning, i've had times it outputs repeated words endlessly, code doesnt run as expected, inconsistent responses etc. Its really cool that they are releasing this open source, hope other large companies can improve upon this model. Or maybe it needs a better system prompt on meta ai

Zucc is a wild child ngl.

So exciting. Can't wait to see this turned loose on Groq

I think 405b is going to be 99.9% peak llm. Can't wait to see how it goes for you running it. I love ollama ....the llama3:8b crushes all my needs.

Love your videos dude! So killer thank you for all you do!

Already on Ollama, downloading now. Exciting!!! Right off the bat it has a sense of humor. "It looks like you meant to type "Hello" but your keyboard stuck on a single key, resulting in the word "Greetings". That's a funny bug! If you intended to say hello, I'd be happy to respond in kind. Otherwise, is there something on your mind that you'd like to discuss?"

Put AI and voice commands into the Quest 3 Gosh, you have to hunt and peck at a virtual keyboard at the moment.

Nobody expected the Zucc redemption arc, but here we are. Let's see how far he takes it.

Bro looking like he blew up a science experiment.

But can it rebuild the Python's set of standard and developer libraries so there is no dependency hell or abandoned or lazy maintenance? And improve their efficiency and remove unneeded redundancy? Because that is what will possibly mark the creation of AI brains, or machines that can think (catalogue everything, mobilise and orchestrate) dynamically toward ongoing objectives and aims as they arise. Large scale enterprises and their maintenance, large projects.

Thats cool, but how much vRAM it need to run? Edit: Llama 3.1 405B has 405 billion parameters, requiring roughly 800 GB memory to be served in its original BF16 precision, exceeding the total GPU memory capacity of a single AWS P4 or P5 instance with 8 x 80GB A100/H100 (640GB memory capacity).

hey guys, how to run these big models on regular gaming pc, I tried the lama (think 40B), but my pc almost burned down 😅 (ryzen7, rtx4070,32gb ram)

Really looking forward its dolphin version and maid version.

Very long contexts locally will be very useful. If the model doesn't slow down horribly.

This is welcomed but not altruistic. With the Ecosystem they are building Meta is going to make bank renting out the GPUs and tools that used to fine tune and Build around Llama. You know, the old "In a gold rush you sell shovels." And Honestly as long as they keep it Free and open as they are now. This is a win win, so hey good on Meta for quite the crafty business model.

I did snake too, run smoothly. thank you

Great effort! Meta could potentially reap significant profits from the model in the future while also contributing to FOSS, which is fantastic. However, one important point to consider is that any large language model that cannot be run locally may not be ideally useful for end users who wish to run the LLM on their own systems. I'm sure, somebody will publish a quantised version, even if it's not Meta, that's good.

I am looking forward to implementing this into my new local LLM based roleplaying game system I am working on 🥳 I was using phi3-128K, but it got worse as the game progressed and the chat history got long…

groq is already live

Appreciate you advocating for the OpenAI API standard. The industry really just needs to lean into that unless new modalities are being released outside of OAI that aren't being kept up by their schema. Even then, I would prefer the community extend their schema than to keep making new ones. Its all the same payloads...

yeah nice, but... where can we use it? at huggingface it always says "overloaded" :/

Great video. Thanks for sharing.

Just made a query with groq with lama3.1 8b instant model: 750.000 T/s! Super fast!❤ And it still understands and answers in the not (3.1) supported Dutch language, awesome

Meta is underrated in this they do some good stuff

At some point they will stop open sourcing it. Specially after hitting the petaflop limit set by us government

Interestingly, the 3.1-70B beats the 3.1-405B on a couple benchmarks. Would be curious to know how that is even possible given the massive difference in network size, unless the 405B model has not been trained to its maximum potential yet.