Meta's LLAMA 3 SHOCKS the Industry | OpenAI Killer? Better than GPT-4, Claude 3 and Gemini Pro 

It's seriously surprises me how much more lovable and relatable Zuck is that he used to be. I think they must have updated his firmware.

Zuck wearing a bling chain is the most SHOCKING THING!

It's almost like Zuck is excited for the first time in his life. Think about it. The dude always worked on php codebases, social networks and the metaverse. Now he's got to play with AI and forget all his past sins.

Open source is the way to go. Big Tech should not be trusted has the keepers of ai.

Mark's recent rebranding campaign is paying dividends.

Who would have thought that Meta & Zuck would be the heroes of OSS

I think the Romans would overrun a Sherman quite fast because of the tank's limited ammo, fuel and blind spots.

We keep interchanging the terms "Open Source" and "Open Weights". It's an important distinction, I think.

For every open source project, there is a close source one twice as powerful.

Who told him to wear that chain outside his shirt???

How was all that sherman tank firing hundreds of weapons simultaneously?😂

Interesting how The Matrix decades ago predicted the energy crisis for the AI and then humans became batteries

He said "interfere with elections".... I guess he would know all about that.

Here is an analogy... 1 country with nuclear weapons is dangerous -- they will attack with impunity. When multiple countries have nuclear weapons they would be MAD to use them!

This is actually shocking and really good for the industry

i am here for the content, but boy is that voice soothing. im following you along now since a couple of months, and its so relaxing to listen to you.

It used to be that if you were using an excessive amount of power and generating a lot of heat at your home you were likely running an indoor cannabis grow tent, now it’s gonna be that you’re training a model to take over the world lol

Mad respect for Mark Z.

I would be interested if some companies will provide a GPT-4 compatible API, for Llama 3 400b+ for cheaper. It would be good to experiment with agents where GPT-4 API is prohibitively expensive currently, but anything below GPT-4 level, the agents aren't always working.

Love the titles. Informative and not click-baity at all.

One crucial limitation is that the Llama materials have restrictions that hinder their ability to improve other well-known language models. This means developers might struggle to refine models for better performance because they're required to use Llama 3 specifically due to licensing constraints. This limitation could have a major impact on certain projects or applications that rely on this model, if developers need to train the model with their own dataset or create derivative works, they may find it inadequate because it's NOT entirely Open source.

That Sherman tank appears to have been souped up a bit.

Bad actors are the big danger, yeah. Karpathy warned that taking a trained model and modifying it with add-on training data will overwrite the original weights - without considering the original training data. You can add capability but you'll be subtracting, too. To really get a better model, you need *all* of the training data. And that's a problem for open models. They share weights and code, but they don't share original training data. They can't publish it. They're drawing on data from Facebook, X, the internet. People own that data. Using it to train is a gray area in copywriter law. It *might* be legal. But republishing it without permission isn't.

Awesome, thank you.

Grok 1.5 ? (opensource)

When an AI can create better AI algorithms, beyond transformers, then a runaway situation could take off.

I’m not even the industry and I’m shocked. Also a bit stunned.

I wonder if Zuckerberg’s Hawaiian doomsday bunker has a moat.

Hi Wes, I’ve been captivated by your series on the swift advancements in AI and the discussions around the possibility of a ‘runaway moment.’ I wonder if concepts like entropy, well-known in physics as the measure of disorder in a system, might also metaphorically apply to AI development. For instance, as AI models evolve from one version to the next, there appears to be a buildup of errors or ‘noise,’ reminiscent of entropy increasing in closed systems. These distortions seem to amplify with each iteration, potentially acting as a natural check on the progression towards AGI. Could these entropy-like phenomena, which manifest as accumulating imperfections or deviations in AI models, serve as a natural form of regulation that might prevent AI from advancing too rapidly or uncontrollably? Moreover, could understanding and harnessing these effects help us design AI systems that are inherently safer and more aligned with human oversight? I’d love to hear your thoughts on whether this perspective might add a valuable dimension to the safety and control strategies in AI development.

Your left aligned picture in pictures headshot covers key information in the benchmark table. Thanks for the interpretation of this though

You’re a true gem, Wes. Great work. Lex Fridman should give you a call.

When I listen to him talk about open source as a way to level the playing field, I feel as though I'm listening to an interview of an in-progress race participant where they are suggesting everyone should cross the finish line together. Despite the sentiment, it is likely that other participants will race to win. It ain't over 'til it's over.

"And we know how to work with your data, because we are Facebook." - see you at the next hearing Zuck.

The thought of being able to run open source agents with the LLama 3 models is actually kind of insane. I feel like there's quite a few possibilities once you have the weights in hand that you just can't do without them, and being able to do multi modal function calling will probably be ridiculous. With that in mind, is there any possibility you could show off function calling with Llama 3 models?

LLAMA 3 isn't available in UK yet.

how is changing background with zoom?

3:02 can someone please help to elcucidate what are the meaing of numbers shown in this table? how these numbers were calculated?

Llama 3 is really great when you interact with it. Confirms this video.

It gets confused with the context of what we were talking about significantly more often than Claude 3 Sonnet. It's very nice and friendly, though.

I’m actually really glad that Zuck is thinking along the lines about concentration of power. This is exactly what I also think is the biggest existential danger - one actor or actor class having exponentially more power than everyone else. I started down this train of thought just considering that internal OpenAI staff already have access to a lot more raw AI power than the gated, tuned version for the public provides. Just extrapolate from there.

7:40 only problem is: AI isn't like regular software, AI isn't predictable. At all.

Normalize audio pls

I thought I was gonna hear your take. I already watched this interview

ooooo baby im ready!!! so ready to test lama to the limits! already AI has done so much for me gyat dahym

Have you heard ? The industry just died. It was SHOCKED so many times, this was bound to happen. RIP industry.

Great pr for zuc, the former lizard!

Well learning ab instrument or lewrning to code will make you smarter on everything else.

So now there's Wes Roth reaction videos?

Very interesting to hear that synthetic data is very effective to train models, although it is not surprising, I wonder how important the quality of the real data that was used to create the synthetic data needs to be? For FSD for instance, how would you create a plastic bag blown by wind not look like a solid object on the street. Or, how would you synthesize a vandalized stop sign (dirty, with stickers or sprayed graffitis,) if the synthetic generator has no natural training sets?

Sounds like what happens when you re-save an image over and over. You get a buildup of error inherent in the algorithms.

On lamba 3’s 8k token context, i suspect this is just for the smaller models. Google gemma 7b also only has 8k tokens. Also since playing with these I was surprised how big these are.

Word in the street is that the Saudis started construction on a new gigawatt GPU cluster - I hope the people that need this info get it.

Marc Zuckerberg sounds like a completely different person. He actually makes sense instead of that corporate talk before

def best life like and likable zuck model to date maybe there is hope

I have my audio cranked and I can't hear half of what is being said.

8k context window genius and allows more computers in the world to run it. The higher that context window goes up the more GPU required and the world don't have massive Nvidia cards. If Zuckerberg thought through that and made the context window for the majority to be able to run in their laptops that would be brilliant. 😮 Plus this opens up the world too Mobile devices too if you think about it. I'm thinking the next release The Big Mama is going to have a much bigger context window for those that have horsepower.

We need that 8x70b

In the 60's student riots half a dozen students using materials found in every ancient Roman household took out a tank.....humans are smarter than AI thinks :)

He is wearing Dinesh’s chain

Please sir. Consider covering VASA-1 the talking avatar thingy that is miles ahead of everything right now. I'd love to hear your perspective. thanks

It has now become a tradition to look for my new "shocking" AI video every day. Never stop ! :)

This is the truest, most massive, most important "race" for anything that has ever happened in existence. We are witnessing the beginning of the most extraordinary movie life has ever written. A little dramatic? Sure, but make no mistake- these titans are NEVER gonna stop one-upping each other. And that is the single most exciting thing we could've asked for.

All these SHOCKs on this channel. I am so shocked. ;-)

The reason they make these models open source is not to give everyone access to AI. They are so big that you need the big players ti run them. This will be cheap for the honey moon phase. But as soon as monompolysation starts, prizes will go up. And by that time you are forced to subscribe and pay. Not paying would mean you have a big disadvantage in life. And i promise you have to pay more if you want usefull AI.

He's open sourcing them because he made the money to train them by selling our private messages. Strange part is that I'm happy that he did all of this. Good job overall.

Let's assume you can reach with synthetic data AGI at level 4, but you go on and reach level 7. With the corruption thing also going on in this process, this might pose an important "paper clip dystopia"-like security issue, where the AI is still getting smarter but ethical value are corrupted in between.

Zuck rocking that chain 🙌

I think the SHOCKING take away from this video is how much Zuck can look human now

Samaritan vs the Machine in Person of Interest.

The audio in this video is so quiet at even max volume. Needs compression.

I see that Sapienspace's comment is shadow banned....

Hell, yeah! This is gonna save me like 40 bucks a month!😂

When children first learn about metaphors: 9:23

Your channel is kind of a lot more quiet today. I can barely hear the sound

Now is the time for silence. For those who do not know what is really going on will not be among those who will inherit the earth. And those who know are wisest to keep their mouths shut. Because if those who don't know find out who does, they will try to force that knowledge out of them.

Better not let Debo see that video Chain!

I need much more disk space to keep all those AIs parameters xDD

Unless a model is a good programmer, it won;t speed up research that much. Btw the spike in use of some certain words in science papers could be people using prompts like "please rewrite this paraghraph to sound better". How many papers are written by non native english speakers? It the best use for me atm when writing anything science related.

I dont think enough people are talking about the math score on gpt-4 being over 10 points better than the next highest

I find it difficult to believe Mark's primary motivation for releasing his software as open source is solely to protect people, despite it being a commendable argument. It seems more likely that he's driven more by the desire to regain relevance, possibly influenced by personal ego or pressure from his company to innovate at any cost. This approach, however, carries way too many risks and there will be way more complicated cyber attacks and it'll be next to impossible to keep up with new updates. Imagine for every single malicious online hacker that exists today after aquiring this open source software he will now have access to expanding his team to 1000x accomplices all in the form of complicated ai agents. The damage will be exponential.

I love the insight and vision for open software Zuck has - he couldn't be more right. By sharing we evolve together to higher organization intelligences of us acting as groups with congruent aligned causes. This makes the groups using open tech more powerful due to knowledge sharing and co-learning where intelligence of one is distributed to many others making the entire group exponentially stronger. The arms race is mostly going to be in the mixture of expert context systems, agential systems, memory, and rewards with everyone's code having ability to interchange to newest, latest and best as a service.

Information extraction or automated data curation, that is filtering data to remove any that contains no information (according to information theory), i.e. what effectively amounts to noise, should massively improve the efficiency of training by compressing the datasets. We as humans are also desperately in need of this as we try to filter the signal from the noise (isn't there some project called Signal AI?) so i really want Elon to use that as the principle rule of the X/Twitter algorithm. Not only would it show the most interesting posts more but it would remove the emotional noise from discussions while still allowing free speech by incentivising originality, critical thinking and first principles thinking and shadow banning virtue signalling emotive posts that basically say "I'm on this political team", which nobody else cares about.

Zuck looks better with his hair grown out a bit more like in this video. That doesn't mean he should grow it hippie long. The short hairstyle from the past made him look robotic. Now he looks a bit like a pro Tennis player a bit past his prime.

Generational barrier for improvement of models. Seems to be simlar with humans having child with close family members or refining dog breeds (they get crazy)?

Funny how google is still just farting around.. while Meta's AI seems to be kicking ChatGPT on it's butt

How many SHOCKS can the industry take??, is it gonna fall soon?

70b parameter model ------------------------ You Which version of llama are you Meta AI I’m a large language model based on Meta Llama 3. You Construct a word ladder from the word fool to the word sage Meta AI Here is a word ladder from "fool" to "sage": fool cool pool pall pall → pall (remove the second "l") paul sale sage Each step is a valid English word, and each word differs from the previous one by only one letter. Enjoy the ladder from "fool" to "sage"! You I think you made a mistake between 6 and 7 Meta AI I apologize for the error. You are correct, I made a mistake between steps 6 and 7. Here is the corrected word ladder from "fool" to "sage": fool cool pool pall pall → pall (remove the second "l") paul pall pall → pall (replace "p" with "s") salle sale sage Thank you for pointing out the mistake! ------------------------ It still messed up with "salle" That's a typical GPT3 mistake

I keep wanting to increase the speed of this video, but these guys naturally talk at 1.5x

Mark's personal branding people need to be congratulated, I feel myself warming to him even though I know there's nothing but evil under the hood.

Zuck does open source LLMs not because of altruism, it's because he's behind OpenAI. If he was the first you would be damn sure he would have tried to monetize the crap out of it

at these speeds, GPT5 better release this week

BASED ZUCK!

Im getting less and less impressed by a new model beating another model in some random test tbh, maybe I'm waiting for actually implication before I'm shocked and stunned

ZUK for PRESIDENT! ;)

Wes Roth by now; it's clear his MO is to hype anything.

This is the first time I have ever seen a grown man when Mark was talking. Slightly less existential horror, since other billionaires like Musk, Bezos, and Thiel have the ideologies/moral philosophies of children. Now Mark both looks like and seems to be conducting himself like an actual adult man. I'm shocked. I might someday say this guy isn't all that bad.

6:00 Smartest thing Mark Zuckerberg has ever said. Maybe the smartest thing anyone has ever said about near-AGI LLMs.

I like him with more hair. Less lizardy

2 Thumbs up to counteract any complaints about the title which helps ensure your success with the algorithm

Intuitively, training new models on the output of prior generations should probably create more accurate models, but seems unlikely to generate models that are more creative or have superior knowledge. I think you end up distilling or refining the future generation. To actually make it smarter or give it greater creativity, I think you need to give it new information.