Stanford "Octopus v2" SUPER AGENT beats GPT-4 | Runs on Google Tech | Tiny Agent Function Calls 

I just realized the correlation of the word 'agent' to the Matrix movie. In the movie the agents are the ones who keep track of everything and keep the AI system working IE agent Smith.

Openai needs to release gpt5 😂

plural of octopus -> octopussies

Why no one is getting STUNNED and SHOCKED???

This is very nice, although I am surprised at the latency numbers discrepancy because I would imagine standford researches would at least try to compare 1 to 1. Llama 2 is not inherently slow (not to that degree) it all depends on the hardware running it. But of course, that’s not the point of the paper. Specialized tiny models do really well on their niche task, we’ve seen that. The reason we need both tiny and bigger and more robust and generally more intelligent model is for orchestration. With smaller and more specialized models for execution. That reduces costs in a major way. I run some agentic workflows for software development and sometimes you can spend $150 (GPT4) in a day and still not achieve what was needed. That may not sound like a lot but I’m only running tiny experiments. For large operations, reducing the costs by 1,000x using smaller models for execution is a necessity.

Awesome breakdown!

Companies will always think of the "more, more, more" approach, that's kinda the main idea of companies especially within big tech. It's nice to know that smaller things like this are on the horizon, producing great results with less effort. I can't wait to see what smaller models can do as well as what brand new news we get with AI from your channel.

Coming right after OAI are investing in a new AI device, interesting

just imagine what a massive data center running millions of these tiny models could achieve

This is what I've always been saying, it's all about the algorithm, once they have that right then compute size, speed and power can be reduced.

Helpful video ❤

Thank you.

> allround performance of a model goes up as you scale up > specialized performance can go up as you scale down. hmm. sounds like the human brain with all the small localized and specialized regions combined with big-ass frontal cortex. maybe, AGI can be achieved by having many different models work together very closesly.

Did this paper reduce dramaticaly the semantic field to the function call small semantic area only? If yes, why outperform larger models is new?

soooo epic!

Shocking that a 2B parameter model is fast. Lol.

Bigger and more general or smaller and more specialized. This has always been the trade-off.

Small model is essentially what Tesla is doing with FSD. The software has to run local in the car and using 13 volts architecture. Maybe 48 volts on the Cybertruck and Robotaxi vehicle to be announced in August.

This hints at a route to AGI that can be achieved by more than a single company with $Bns - a LLM which is good at reasoning and coordinating agents, still "big" and maybe a few Tn parameters, with an *extensible* plug-in architecture of agents with function-calling to access APIs, where each agent is highly attuned to a *single* task - very small but highly proficient. There's no reason the agents need to be run in the same datacentre, they could even be federated - like Folding@Home. Imagine a world where every website/server has an Agent Interface, as opposed to an API.

Finally a video with a durantion I can watch without reschudeling my day :D

Imagine when we'll have actual neuromorphic chips/analogic ai chips as a normal thing in the market like 10 years in the future. With some sort of multi-octopus v96 in every electronic device doing gpt5 level calculations, consuming 2 watts, with multimodal models constantly checking and improving each other generation hundreds of times before giving a perfect answer in a second. Even on things where safety is fundamental

If Apple releases Siri with agent abilities and with scores above GPT-4, OpenAI will be in trouble. You got to think about how many people have iPhones. Same with Google with Android. OpenAI doesn't have influence like those two because either everyone has an iPhone or Android.

Bro now stanford is beating ai sheesh, the race is heating up

Ah the vampiric squid.

Recently saw an article by a comp science professor saying how AI is a waste of time. The energy consumption was one of the points. I thought to myself...can he not see the trend? Sigh.

What also is bound to happen: all these models are pretty rough when it comes to how efficiently they run, once optimised and perhaps moved away from Python there is double digit performance to be gained on pretty much every device (perhaps even more on non-cuda hardware). These all could very quickly become small & way way faster.

One thing I am excited about with AI is how it’s going to change gaming. Imagine in GTA talking to an NPC, and the character walking buy says “F*** you” and you respond, and he responds based on how you respond, literally could say anything that’s not coded. Would be insane.

If the key to making smaller models reliable at performing function calling is training them on a thousand examples of each function API they need to support, would I be right to think that we likely need separate fine tunes of Mixtral (the best OS LLM at coding for its size) for SWE Agent, Open Devin & Devika; where each fine tune contains a thousand examples per API that those agents support. If so, given just a few examples per function API, a model like Claude Opus could automatically generate the thousand unique examples for us, making this an easier task to perform. Thoughts?

Multion does not work anymore...bt it was so good

Training a lot of maths's data does not make a good mathematician, the problem is how the brain that calculates the maths is still being undiscovered.

Is anyone here STUNNED or SHOCKED? Or BOMBSHELLED? Wes must be slipping with those AI-generated video titles.

Few issues: this paper is hyperbole, poorly written, with clear differences in parts written by gpt and by the authors, where the main innovation is something that practitioners do every day (adding tokens). Not sure endorsing this sort of work does well to the community.

OpenAI think they can decide how much progress humanity should have. They need to be humbled.

So that's where Marvel's Hydra came from ^^.

Wes, do you ever take a break? It's Sunday :)

Make movie about Grow API and his custom chips

Neat, I'm early.

We need to backronym AGENT stat!

ChatGPT-5 is gonna be shockingly underwhelming. I can feel it.

what is the context token size?

I read super agent in a paper title and my instinct is to block the authors and forget.

Its all about function calling, not about perfomance

Does anyone know what they used to control the phone? (After the llm made a decision...) Is it a VM ? Is it an accessibility app? Anyone know how to do this?

So they wrote a whole paper about having finetuned a tiny model that is a liiiittttle bit more accuracy in terms of function calling than GTP 4 which was already at over 98% and that this tiny model has fast response time (which is to be expected)?! I don't get where the big adavancment here is and I find it so absurd and craving for attention to compare these small models with GTP4 and claiming they beat it at something. I mean despite tiny models being very light weight and hence fast, you can only do so much with a tiny bit more accuracy in terms of funtion calling and if the model itself is dumb it still can't solve more complex tasks, like for example this often used "write me a snake game" task. You can have a swarm of 1000 dumb agents and they most likely wont be able to get it down except by pulling the solution from the internet. I have yet to see a 2b model that has about the same performance in terms of reasoning than just decent 7b model like Open Hermes 2.5. Also to get anything more complex done other than creating a new calendar entry and creating another search interface for google, you want a big context size, so decreasing it is not an advantage but a disadvantage! Maybe I'm just missing something here.

LOL my grandma beats GPT4 now days

dude you're burning me out with all of these tiny, insignificant updates. we get it, there's companies one upping each other daily in a race to be 1st in line with this tech. maybe you should wait a week or two until there's actually some notable news....

WTF with their Intro citations. They shouldve used IEEE style.

“Octopus” comes from Greek, so the correct plural would be “octopodes.”

a 'super agent' as a master of 1-10 compared to gpt - arguably a master of 100s-1000s? such a comparison should lead some to see this is a hype train.

Wes, was this video generated with AI?

We should stop giving AI systems Villain names, cause, whats next, Destroyer of GPTs?

Finally; isn't?

'Octopodes'

Not a fan of the two part videos, but im not going to stop watching you. Thanks foe what you do. I get that your audience is pickey AF.

oh, this is not STUNNING or SHOCKING? then I shall skip this one out.

First

Your mouse cursor is annoying ! Cant focus on the text with something buzzing on the screen already - get a fidget spinner if you can keep your hand still please !

These small models are cute and effective and all, but they will never achieve ASI. That’s why the chip race is so important right now. ASI beats everything

the trade off is that you're back to a narrow AI that has to be propped up by traditional programming. a true intelligence wouldn't need "def" anything, it would just open youtube for you. period.

Dr. Robert Sapolsky; John A. and Cynthia Fry Gunn Professor, Professor of Biology, of Neurology and of Neurosurgery -Stanford Dr. Garry Nolan; Rachford and Carlota A. Harris Professor in the Department of Pathology ,Teal Innovator Award (2012) from the Department of Defense, cloning/characterization of NF-κB p65/ RelA and the development of rapid retroviral production systems -Stanford Dr. Andrew D. Huberman; Associate Professor of Neurobiology and, by courtesy, of Psychiatry and Behavioral Sciences -Stanford Arxix (dot) org. "Octopus v2: On-device language model for super agent" -Stanford I see a pattern.

This has long been possible with small models. I'm not sure why this research paper is news. You can Qstar and Lora a small 7b model to outperform GPT4/Claude at very specific tasks. It's full comprehension of the task that ends up falling apart without a larger model. 7b models are large enough to have a reasonable amount of broad knowledge. For things like art this becomes kinda weird because Claude will continuously out perform on creative stuff versus a small model. It seems the only way to get more creativity is larger models with some allowance of hallucination.