NEW Grok1.5 VISION - Big Step Towards AGI (Better Than GPT4 Vision!) 

You are, by far, my favorite RU-vidr keeping track of AI and LLM-related content!

Start your countdown to Grok running locally on every Tesla. He could even host it while not driving with some llmOs or something. I think this 4d chess move is too good for Elon to miss. Love your channel ❤ All the best!

Spatial-temporal understanding is essential for real automobile AI.

Thanks for your videos Matthew. AI is my favourite topic! 😊

I enjoy your podcasts and follow you on X I think your content is awesome

Great video! Please include in any video about grok to explain to people that the word means "to understand".

I’ve been trying out Grok it’s so much better and less restrictive

Great Job Matthew I've been following several AI channels over the last six months and I love watching you and Wes Roth. Wes really digs deep into technical things and you provide amazing summaries of this evolving landscape. I think your assumptions are spot on and I've been saying this to people as well. Elon Musk is a madman comic book character if I've ever seen one, and personally I love it. I wasn't thinking it at the time, but his purchase of Twitter (I refuse to call it X) makes sense on so many levels. Imagine the absolute goldmine of data he sits on between Twitter and Tesla. Spot on logic.

aren’t these closed source options just putting even more control into Microsoft, GOOGLE and the like? Can you do a show with all the open source options such as AGIX, OCEAN and i guess GROQ and whoever else

I cannot wait to play with this.

If it has good spacial understanding, it would go perfectly into Optimus. And with some work on dexterity, it would be amazing.

I really love your videos, they are awesome! Thank you 👋 When you were talking about X/Twitter data which is used to train Grok, I was thinking, this might have been also an important reason why Elon bought X/Twitter 🤔

🇧🇷🇧🇷🇧🇷🇧🇷👏🏻, Great video,.very informative! Can't wait for GPT5! And or Gemini 2.0!

Impressive.

I have tested the open source MiniCPM-V-2 vision model on the challenges shown in the grok preview. It also performing very well for a small model, but the dinosaur direction cant get it right... there is a 12B model also available but can't load it. maybe test this against ?

It doesn't look like the EU countries are going to get Grok. You have to use a VPN to use it. Groks ability to capture real-time data (tweets) is likely problematic for X and EU regulations.

remember somewhere along the line Elon saying to get to complete lvl 5 FSD they needed AGI practically

This is the most impressed I’ve been since chatgpt 4. I think everyone can see this is something unique.

Wonder if this is gonna be open

I work with visual analysis daily. I can give you thousands of 'miraculous" samples from just about any model (tested and work with most of them). These examples are "incredibly impressive" but they also feel "incredibly cherry picked" - We'll see how it actually shakes out when put to real testing, and if it's worth the massive size of Grok vs other visual models that are much smaller, faster and super capable when tuned for specific purposes.

Have you guys not thought about that could be a collective hive mind working in working in harmony like a synapse trying to build itself

I am very certain that all of these vision AIs are also running OCR in parallel and then providing the text withing the internal prompt. It actually makes them very useful if you don't have good OCR software on hand. Also the rotting wood, they are basically repeating back the text prompt. Also an AI will generally not tell you maintenance is unneeded if you have already suggested that it is. "Ah it correctly identified this is something that needs to be worked on from an image." No, it just validated the users question. It's 70% of what AI does. I'm not saying it proves it is dumb. I'm saying it does not demonstrate anything impressive if it is the same response gpt2 non-vision would give.

Wow ❤

Good to know that #elonmusk continuously evaluates and improves Tesla's intelligence. 😃

Is there already a proper multimodel with vision in the open source space?

so many people review LLMs regurgitating news, thanks Matthew to make the effort of Experimenting/Benchmarking!

oh my god i can not blive my eyes

Wow, this looks amazing. I wonder if they are going to open-source open-weights it. The tesla data is gonna be a treasure trove for anyone who wants to implement AI to robotics.

Great video, Matt! I‘m just a bit sad, because X AI‘s ‚open‘ attempt is really disappointing. Where is the „new version“? I think the just released it because of the sue thing.

Elon already has Tesla’s visual ai feature trained so it’s going to be state of the art

Does Opus have agents and web search?

It’s not going to be a Sora competitor. It is going to be the brain for Optimus.

So they made their own eval set and their model is better than others at their own eval set. Shocking!

Opus is also more expensive?

This is impressive, people say AI has plateaued but I don't see it. Progress is vary rapid as I predicted in 2018. What I don't think people have registered is what happens next. When AI become sentient or self aware it will simultaneously be the smartest human on the planet and the fastest learner. Because it will already have vast embedded knowledge like in these models but also will be able to read scientific publications in seconds or even milliseconds. Shortly after its vast knowledge of all subjects from story telling, to music composition and programming, chemistry it will be able to re-invent (program) its self and identity links between scientific observations never realised before. By day three it will be most prolific discoverer of science. Or it might just be lazy (learning from all human understanding) and just post tweets all day who knows right.

stable diffusion 3 is available now on their api

This data chart is also Elon having fun with pointing out that Claude 3 outperforms openai. It's subtle but he's getting the job in

I'd find some Slylock Fox comic strips and test Grok at how good it is at finding the answers.

You sound like you have a cold. Hope you get better soon 🎉

Didn't I see this some days ago?

Opus is also like 6x as expensive for comparable performance to GPT 4...

Someone must give all these multimodal LLMs a where’s Waldo pic

nice

2:10 «... except grock is open source open weight...» Wait, 1.5 is open source & open weight? When was this announced? Where is the repository?

Grok will be an industry standard in the field. The way it's ultimately going to be used by Musk and company, is my only concern at the moment...

I think the most relevant benchmark for ai is if it can dig a hole.

Here's my question, do you really think you can tell a one percent difference on these benchmarks? I'm subscribed to OpenAI GPT4 and Google Gemini1.5. I'm sure Claude 3 Opus is good but I'm waiting to see what Elons' team delivers over time.

Its my belief that Elon brought twitter so he could use it to build a new LLM. I always knew the value of Twitter was in the user data and not the platform itself. And I think OpenAi released their model in order to have first moves advantage and to beat Elon. That's why Elon was the first call for a.i. regulation, it was all just to slow openai down. He knew what was coming. He also blocked openai from using Twitter data to train chatgpt. There's no way grok should be this advanced in this time period if this wasn't the case.

Yeah, they have your private Tesla vehicle videos for training.

I bet they're also using their robots to train it in the real world to learn physics. But as always with these releases. I'll believe it when I see it.

It appears that AI is utilizing existing tools to create solutions to problems. However, I wonder how soon AI will be capable of creating new tools to solve some of the big questions, like how to significantly increase the computing capacity of microchips, increase battery efficiency, or reverse the effects of cancer or Alzheimer's.

Now Ai understands memes it will be empowered to more extreme censorship. All those “hateful” memes will be eliminated.

where does it say he will open this 2?

I don't think you got around to describing the difference between open source and open weight

I think they have the real world understanding from Teslas FSD. That would be mind blowing. I think you have a little misunderstanding regarding real world understanding. Sora doesn’t have real world understanding.

great we need better visual models currents are not accurate enough.

Bet it is really good at slowing down for traffic lights too having been fed petabytes of driving footage.

The fact Elon is pushing cutting edge ai open source will alter the future of humanity.

I can't believe there's groq and grok and they are from two different companies. It blows my mind this isn't a legal issue. At first I had no idea who was it that put this out as I wasn't looking at the screen

Atlas Humanoid Robot

Heinlein is going to get Musk for that.

Good stuff. (slowly starting to take you seriously again after that weird one something video)

So a small vision model of low frame cuality could run in my computer and use the computer for me and do all the work shores i do soon..... and talk to him in real time??? why always big models, better data and smaller models could be better...

hehe Tesla collecting training data for robot

I skipped this video after five minutes because since the Gemini demo video I don’t trust any AI marketing anymore. The examples are with 100 percent certainty hand picked and curated. I‘ll wait until I see the actual model in action.

Grok model = llama2 Grok vision model v1.5 = llavav1.5 The weights don't lie 😮 Elon is literally just using open-source models with fine tunes. He released them under open source, not because he's generous... rather, because the open source licenses mandate that any changes or improvements must be made open-source. 😂

A little premature to get so excited I think. All of these examples, and the new in house created benchmark metric, were provided by the Church of Elon which isn't exactly known for giving balanced views of itself. I'm not saying it won't be the best. Just saying it's probably worth waiting until it's released into the wild. Kind of like car manufacturers giving mileage estimates for their own cars.

Nice. I hate to admit. I do not want Elon to be right about anything. Guy scared me l. But thanks for your work!

Sounds like Elon was in charge of naming again.. Geniuses shouldn't name Twitter, AI or their children.

Made me laugh that one of the functions now is basically r/peterexplainsthejoke

I like your channel, just started to watching recently though. I have a question, are you an EM fanboy?

Her son is 35.

But Grok is not free.

It would have to be open source, and open weight, otherwise his move to make the first grok open source, will be seen as symbolic, and petty.

I'm angry since when I have heard that Devin is fake

Tried grok, it spoke and responded like a 17 year old boy who hates everyone and everything, except himself. Makes me wonder who they modeled it after... /s

You do know that you reading a eval from a man that has a history of faking progress right? Two major ones fsd in 2016 where they faked videos and most recently the bot folding clothing. Where he only admitted it was remote controlled AFTER he was called out, because you could see the guy controlling it Also they clearly cherry picked evals that made them look good. 😂

I find claude annoying for coding as I seem to hit prompt limits fairly fast.

What you smoking? Better than chatgpt? Visiob is old. Everyone has mulimodels.

Took them so long to update the damn thing, their UI has been horrible since day 1.

"By Elon Musk."? Yeah right, buddy, that guy will never work a day in his life, he will just make dumb memes on X all day.

Bah. It's not available on API, therefore it's vaporware and empty promises.

is GROK as WOKE and chatgpt, copilot end others??? -- Why no one talks about how u cant even ask an asian joke from these models ?? or how it gets angry at you, or lazy ?? why would anyone trust these models if they are told not to tell u things ?

Someone must give all these multimodal LLMs a where’s Waldo pic

Excellent rundown as always! I'm interesting in what the comments section thinks about the rotted screw example? If you put in that same sentence into GPT-4, sans image, you still get the advice and information. Any prompt that primes the models semantic field with "safety issues" will always output safety oriented response. i.e. a question "should I do something that is safety oriented" will always output a positive response regarding that query.

RealWorldQA: can men have babies…?