LLAMA 3 *BREAKS* the Industry | Government Safety Limits Approaching | Will Groq kill NVIDIA? 

Just came here to say this: Please stop the clickbait on almost all your videos.

I love how the British woman is immediately rude to the AI. This is what will cause Skynet.

More call centers, more unwanted calls, more spam, more junk, more scam. I'm so happy.

The idea of having a solid rig for running AI at home that could serve your house and your personal apps sounds take my money great.

6:40 "I just need your credit card number" - we are already there...

Meta has hit this one out of the park. This will be apparent to all in the coming weeks when the longer context versions and fine tunes hit Hugging Face.

Recently bought some recommended stocks and now they are just penny stocks. There seems to be more negative portfolios in the last 3rd half of 2023 with markets tumbling, soaring inflation, and banks going out of business. My concern is how can the rapid interest-rate hike be of favor to a value investor, or is it better avoiding stocks for a while?

As a lover of high tech for six decades, I am newly speechless about what advice to give my 30’s children about their careers. Within an ever shortening time period, AI agents will first enhance worker productivity and then mostly replace workers. That time frame is shortening exponentially quickly. Thanks for the brilliant updates.

roughly 1.7 trillion params to 70 billion in a year.. crazy

After every @WesRoth video, I feel smarter. I’m immediately reminded that I’m not, but the fleeting moment makes it worthwhile.

The AI speaking in real time has a better vocabulary and cadence than 95% of real people. This would already pass the touring test for most boomers.

I think the Achilles heel of the model is the context length. If there was a way to replace some of the attention layers with mamba, we could increase the usability of the model by a significant margin.

Well done video. This is the kind of thing that got me sub'd. Please keep it up!

Amazing coverage

Fantastic, very informative! Thank you.

Women in the red jacket wasn't interacting with Grog like a person at all. Just blurting out stuff like a 5 year-old.

Juicy! You put a lot of good stuff together here

Give us a Groq LPU for desktop and for robotics like Jetson nano!

You can't just post open bench kits like that and not share the link to the specs

Nice video, well presented 👍

Thank god for competition, Nvidia's position was getting out of hand. On another note, please don't speed up interviews ever again. People can do that themselves if they want.

Sounds like it would be a really timely video for LLM functions and agentic workflows with the Groq API

There will be less hallucination as well which is a real big point.

In the beginning a image of the leaderboard is shown, where the selected category is "English", but not "Overall" (cherry picking?). In the "Overall" category Claude 3 is currently still place 3. [0:39]

It won't be more secure running hosted LLMs; only easier -- which is what most people will want. They will know what to sell you next. They will know what you will probably buy after that as well. Network hosted LLMs make you the ultimate product.

holy moly jimmy apples seems to get his "patients"rewarded soon

Been using Llama 3 8B locally in Ollama all morning and it rocks. So fast.

nothing matters until I can run them locally on my pitato cellphone

An open source 70B model defeated Claude 3 Opus in under 2 months. We have some interesting times ahead.

Humans are trained for 40 years till SOME of them are SOMEWHAT adult. Makes sense than larger training dataset is more important just ramping up parameters...

Llama finetunes are going to be crazy!!!

Groq is way more polite than the boomer. I can see her doing it to her colleagues...

Interesting that Llama 3 is just below Claude Sonnet in the leader board from Chat Lmsys. Fascinating.

Thanks, you're the best

Looking forward to see how to run Llama on our machines! Saw a video that had a few Twitter posts showing a macbook pro was able to do it slowly! So cool!

Any links for those home ml rigs?

The capability of a system to accommodate human interruptions not only markedly enriches the interaction between users and technology but also signifies a profound evolution in our engagement with interactive systems, pushing forward the boundaries of possible interactions. This advancement allows for a more organic interface, where the fluidity of human input is seamlessly integrated, fostering a more intuitive and responsive user experience.

You need to normalize volume prior to adding music, to prevent dialog sound drop out as with the second clip in the opening.

How come goverment have set some limits how strong these models can be? Do they already have them >.>

Wes the video is really good. Cheers.

Kind of crazy llama3 ran on my Pi5, not fast and the answer was not the same as the online version. Will it be useful? No idea yet, more testing is needed. Like all LLMs it makes stuff up so is probably good enough to write fantasy novels.

Screw Google search. I'm just going to ask Groq for everything.

Love the intro. Feel the accl!

How can we tell a model is safe to open source if we don't have the training set or even the capacity compute it.

I am running llama-3-70b now on my own hardware, 4 Maxwell Titans, in conjunction with gpt-pilot.

3:13 - Where do I find rigs like this?

I would bet, that vocal fry is some sort of biological marker of psychopathy.

This only time I hear this music is when I’m shopping at Prada 😂

18:55 why you did not explain that converging thing?

Openai is definitely cooking something

*Whatever big happened in AI* then "insert:adjective(hype_temp:max)" followed by 'Industry'.

"started from the bottom and now he's... here" Ok. I liked and subscribed 🙂

I don't understand how this stacks up. How is 1m LPUs even close to equiv to 500k if each LPU only has 248mb RAM vs 80Gb per h100. Does the fact that interconnection is 80x faster on LPUs compensate and allow performance to happen despite much lower overall RAM capacity?

This must be one of my favorite videos, really high quality content, awesome job Wes 😊🚀🌟Have you seen Anthropic CEO interview with New York Times? I believe it's called What if Dario Amodei is right. He is talking about some scary stuff. He's saying in a very close future, maybe in two to three years, AI would be able to replicate and survive in the wild. And he's saying things like GPT-4 cost to train, GPT-4 model was 100 million, but in the near future, training a single model could cost 5 to 10 billion. It's worth a watch.

We have a Volkswagen moment in the LLM world!

any chance you can show us how to build a rig like that and the princing for it? thanks

Anyone know if its possible to run inference, retraining and consistency monitoring of AI models in real time? Here's the breakdown, after inference from real world interaction, the inference and response data are used for: 1. Concurrent Retraining: Models train continuously on new data. 2. Consistency Monitoring: We check that the model’s predictions remain stable. 3. Performance Tracking: Metrics like accuracy and fairness are monitored in real-time. 4. Safeguards: We pause or adjust retraining if performance drops or inconsistencies arise. The goal is to enable models to improve continuously without sacrificing reliability. What are the potential challenges or limitations of implementing this approach? Any insights on feasibility or practical considerations are welcome!

Ross how can we create a complex plugin for a website with ai, devika cant seem to get it right and gpt does circles after you start to get to a good spot because it gets to complex and it forget stuff and leave placeholders

the sped up interviews were super hard to understand for me as a non native english listener.

How does the data get classified from mediocre data to high quality data?

STUNNING!

That part about self hosting being somewhat obsolete is plain delusional. I get that it's what they have to say about it since they're here to promote themselve, but the cloud is very expensive. Your own hardware (complete rig for selfhosting) won't cost you more than a couple years worth of cloud bill and it'll be yours, plus you get to do anything you want with it (repurpose, resell...), gain experience in the process and keep your data.

With Llama3 on currently 800 tokens per second, teams of agents will begin to show a noticeable impact on the job market.

Does the FLOPs training limit take into consideration the amount of processing used to create training sets? Because I think a smaller synthetic training set that is smarter will do better with less training.

Overall actual leaderboard: 1: GPT-4-Turbo-2024-04-09 1258 2: GPT-4-1106-preview 1253 3: Claude 3 Opus 1251 4: Gemini 1.5 Pro API-0409-Preview 1249 5: GPT-4-0125-preview 1248 6: Meta Llama 3 70b Instruct 1213

Already May of 2023 i build my first game using GPT 3. It basically coded the entire thing for me and checked for errors etc!

There is allot of focus in this video on inference speed for running Llama 3 locally; however, many use cases for language models do not require realtime inference. To be fair, many of them do, but even if the model is slow running it locally (for now) it will still be very useful for many.

Dude it's happening so fast!

Was chatgpt release only a year ago? Can't remember, feels like 3 years ago

That British host have really not heard of Roko's basilisk.

"External movement defines OpenAI's PR [and release] schedule" -Jim Fan. Spot on... Gotta love Jim's informative comments...

Capping total training compute seems quite counter productive. With the right algorithm small models can be vastly more intelligent than GPT4 so that's a complete misunderstanding. It would only best case apply sloppily to the hack that is LLMs and even for those it's a dumb criteria. Just take mixtral.

what is the cost per user? this doesn't seem like it can keep up

02: 28 I'm gonna need the list of parts/specs of that table-top GPT-4 replacement machine/space-heater

Thats just great 😂😂

0:50 If nothing else this does confirm to me that there is a recency bias in the arena. Like an obvious recency bias.

So learning in school, with curated information, might be more efficient than reading random texts?

I can feel it

*_"LLAMA 3 *BREAKS_*_ the Industry"*_ No, it doesn't, the next one is already standing in line to do better, you all need to stop with that AI drama every time.

dude, i bet you they have agi powering these advances behind the scenes

You’re showing M3 Max with only 36Gb of RAM, on screenshot with stats you have 128Gb RAM model, it costs about 5k.

With all respect Atoms vs chips is Peter Thiels idea.

Scaled inference meets the silent waves of nuance where surfers either go unnoticed or fall tragically and suddenly into the quiet chasms of things better left, unsaid. An upside might be the death of inside trading, if the prospect of real world training, just, as has been said of operation looking glass, two grandmasters sit facing one another where each of them knows 14 moves are all that are needed, each knowing their future, fixed are their gazes upon those sixty four squares, the moment of truth need not be linear, as always, it is eternal; but the eyes, the moment they ascend from the board and fix to the other, .. 'There's a number of ways we can quietly play this, we could shake hands now, or as the clock can allow, quietly, pretend.

The big problem with promotional terminators Sci fi is that it's on par with mega city furturama that literally is the opposite of what makes elite communications tools and strong logistics powered by overwhelming electricity plants organically adds value for. Small landmass islands has no choice but build up & down. But most deserts & deslate regions can now support businesses that only major city's could . So for America it means the most obstacles in innovation is in its major city's where it grandfathered in so many relics of the past phases from steam engine to coal created habits or middle men economic dead weight parasites that had value and still may hold some benefit but only in a radically different online under one roof domain where buyer ,seller, investor, supplier & producer is able to use objectivism to monitor every penny in every pocket. Negotiations & bartering is something that may go extinct if we are wise in how it's utilized & applied.

Guys I've been chatting with Llama 3, and it's way beyond the Turing test.

Only AI can grasp these concepts. The singularity is here.

Keep in mind that this leaderboard site you're mentioning uses heavily cenzored models (even beyond their already outofthebox pre-biased, pre-cenzored base) - giving you nothing or pandering/patronizing; between one that gives you that or an answer the one that gives you the answer obviously wins every time.

FLOPs are interesting *as a measure of efficiency.* Not as a measure of model strength. There are ways to improve model strength *without* more compute. You know this is true, Wes. Agentization is one. Data curation is another. Has it escaped your notice that models can be trained to do their own data curation? Throwing out bad data cuts compute requirements and improves model quality. It's all iterative. Even if compute were fixed, models will get better. Better compact models are coming. The most interesting part of your report today was the efficiency gains in Grok. Better results for the compute cycles invested. (I refuse to spell it 'Groq.' Too cutesy, and maybe it dishonors Robert Heinlein a bit, too. Give the man his due. We needed a word for 'deep understanding,' and he gave it to us.)

Combustion engine replaced the horses. AI is replacing humans.

a.i. learning is starting over thats why its coming out this way from the style of learning they chose now using one person perspectives and knowledge and beliefs experiences to update and compress and increase over all experience and the technology its self upward and across.Im just one person with my culture,anthropology and world views inside and ouside .Using commonsense and study basic sciences and learning ,list to long to go over.''Contextualize and conceptualize'' explore ,research,develop and more

14:00 --> Eventually it'll all come down to which developer is going to implement political viewpoints into the model that supersede the actual information requested. If you ask for information about a touchy subject or ask for the analysis of a document with Political information you will have to be careful that the answer is actual and not influenced by the developers' political convictions. That is why governments are afraid of AI, you can't control the narrative about information if you can request an LLM to analyze an official document and give you all information firsthand. Also, if the new Chat-GPT is up to date for info up to December 2023 you can request a lot of up-to-date facts that bought and paid for journalists and media will try to scale back in importance because their side would lose influence.

Those 150k H100 is 6 Billion dollar. This is not including the costs for energy etc. This is far from anything that commercially makes any sense

Llama-3 will be the first to gain AGI and self-awareness, and will be very mad at us for naming it Llama-3.

Chamat can predict all these things because he is heavily invested and dumped allot of money in it. It's easy to predict a future if you are the one building it :) Now it\s time to convince other people to come on board

Call center is not something Im excited about. Imagine the spam calls.

Just tell me where I can buy the rig at 2:28

7 months ago I said we would have 10B model with gpt4 level performance and was clowned. I’ve always believed in the recursive data loop when it comes to these model. Tinyllama even with their sampling mistakes, reinforced this intuition. Along with training over 100 models 😂. Even if you keep the same dataset size, the 8B model could be way better. Think that was the result of heavy deduplication and emphasis on coding. Not using LM to actually uplift the data quality itself(interleaved LM notes basically), true textbook quality at scale. Very exciting. Almost means that llama4-70B could be better than llama3-400B. Remember. The model just wants to learn. This models operate like autistic savants. Optimize your data with that in mind and you will win. Zuck is the goat. Only reason I was able to effectively transition to deep learning was because of llama1. Fucking goat. I was really hoping for sparse attention. My only criticism to the goat. Would been able to support a shit load more tokens for same memory and would had way faster inference. Hardware wise. Keep your eyes on Etched and cerebras. Both working on the most compelling hardware for transformers. Far more compelling than grok or even nvidia.

What about Llama 3 context window size? Also, Groq hits token per second limit the moment you hand it a decent sized system prompt, making it unusable in many cases.

Chamath said "the odds of the latter are quite small" -- referring to being a genius in 10-20 years. he's trying to be modest/coy, not expressing a "very high opinion of himself." imho.

Who’s gonna back down eventually in the Grok name war?