GPT5 unlocks LLM System 2 Thinking? 

Thanks for all your hard work producing these videos! I appreciate the speed, you are very concise with no filler. I'd really like to see a more in depth video about how to create new skills for these autogen agents. I feel like I'm missing something obvious about how to make them work well.

I appreciate your speaking speed and succinctness. This is the first video I haven't had to watch on 2x in ages. Thanks.

I really enjoyed this video, keep up the good work Jason!

I always come back to your channel.. full of gems, you're a life saver 🙏🏽 thank you seriously

Great video, love the practical demo!

excellent information as always, I'll be testing this out this afternoon

As of now, this same behavior is able to be accomplished with CrewAI. Simply create a new task, set the process as sequential, and have the task be "Puzzle solver; when given a puzzle, pass it first off to the Puzzle Solver, then to the Puzzle Reviewer." One thing I would also highlight about CrewAI is the ability to highlight a backstory, where you can use emotion in order to improve performance as shown by Li et. al in "Large Language Models Understand and Can be Enhanced by Emotional Stimuli". Great content!

Great video, seems like this will be the next big thing!!

excellent work, great dissemination - thank you

Great content. Can't believe I'm just discovering this channel. I really believe this year or next we will get a localized approximation of AGI. System 2 thinking is core. Maybe extend Meta system 2 attention with gated differentiable memory? However, I envision this along with extremely robust in-context learning, linearized knowledge graphs, and extremely long context. We can achieve what most people would believe digital AGI will look like, performance wise. With graph-based prompting methods, we can exploit extended test-time compute and brute force our way to solve most task. Even expert level task. Why I believe smaller models are primed to be majority winners honestly. 99% of task don't require superhuman intelligence. A 10b model with tinyllama scaling law, on a phi-2 style dataset would give you gpt-4. Exciting times.

Very interesting, thanks for your beautiful work!

beautiful explanation, thank you!!

Thank you. Excellent explanations.

Very impressive structured presentation of every concept and ideas, and explaination.

Great content, thanks!

This is very helpful. I was looking for an example of how to set up for more deliberative thinking with a walkthrough to hear the thought process.

I am reading the book at the moment. Had the same thoughts on it. And I think Auto Gen 2 is the way to go T the moment. SUBSCRIBED!

Didn't realise so many similarity between how human brain & LLM works; great video!

Great video bro

Mamba (S6) models are the future imo. system 2 thinking requires knowing where you are in relation to where you have been and are going. Latent space is crucial!

This is really great!!!!

Great content

excellent video!

This is awesome, multi agent seem like a great solution; will try out tree of thoughts as well

Thanks!!

This is brilliant! Also, small typo at 0:55, RHS bat line should read 'x+1.0' : )

THANKS FOR SHARING

I think I understand the problem (somewhat at least), it is that LLMs try to give the immediate answer to what you ask without thinking about the long term. For example, the other day I tried to create an app in Python using only ChatGPT for everything, from creating and designing the folder architecture, going through each line of code, configuring the database and even trying to deploy it all from bash. I clearly explained the conditions of the project and my resources to ChatGPT but what it did was desperately try to schedule many files that it later forgot to name or renamed, etc. I guess ChatGPT tried to do what it meant to create the app but it didn't think through the steps for it, it didn't create a centralized architecture or think about file names or how to connect them properly before starting to program them. It's like what happened with the students in the Veritasium video, they simply said the first thing they thought in an almost automatic or instinctive way. Right now I can be there to guide ChatGPT but if they manage to overcome that barrier, programming will definitely take a backseat, at least non-super specialized programming.

Wow, actually a super helpful video and example. Will give it a try for nursing stuff.

Tree of thought and Dialectic options seem very intriguing.

Thanks for the great video! Do you think OpenAI's Assistants API can replace Autogen?

I’ve been wanting to try figuring out how to get multiple GPTs to cooperate to achieve an objective or output for a while now. But just an hour ago I decided I was going to look further into how it could be done. Now I come across this video in my recommendations even _BEFORE_ I made any attempt to find a guide. Sometimes it really feels like the universe is trying to send me a message.

It seems very timely and kind explanation about the autogenstudio. Thank you for your video. I am trying to replicate your explanation with my desktop. But, I lost the way in building the 'group chat manager' agent. If you're okay, could you let me know how I build the agent, group chat manager as you did.

Love those veritasium interviews

Interesting. This shows me we’ve got a lot of techniques to still try.

For animals in colored room problems the accepted answer is not optimal. The first thing to notice is the RGB color chain that is forced by the color clues. Thus colors in order are thus either RGBY or YRGB. The first choice gives one and only one answer that matches the other clues. The second color arrangement gives two matching answers and is thus to my way of thinking inferior. The second one is ambiguous as to the placement of the Lion (first or last) and the Giraffe. First gives in order elephant-red, giraffe green, zebra blue, lion yellow.

Excellent video, many thanks, how about one on TaskWeaver + Plugins and creating a free test environment for plugins with Semantic Kernel??

great content Jason - keep it up!

Amazing. Can you give any application for this? Is live a video on how this can be implemented in different areas

dawg, this is the SAUCE!

Great video. I would like to know how this "system 2" works with a problem a bit more complicated but compleatable problem. Because in that case the iteration was too short (only 2 times). I would like to appreciate the real benefit of multi agents.

While I thought GPT4 reasoning ability is so impressive, Sam altman consider it as super limiting... Cant imagine what we can do with GPT5!!

2:48: Also, after breaking down the problem, we go back to layer 1 thinking. I think L1 thinking is using patterns and principles. The main breakdown task of L2 is also inherently an L1, because we use patterns and principles for the breakdown itself.

We use intuition (system 1) because it requires less energy. It sounds like GPT-5+ could use a similar technique to save compute. Hopefully not at the cost of accuracy.

AutoGen Studio would have been great if it was possible to upload some kind of text file or knowledge retrieval to an agent. What’s the character limit if you type in ‘System Message’ for creating an agent?

very insightful and id really like to work together with you!

LLMs will never be able to reason until it learns the difference between true and false. All they do is feed it "true" things, so it will never be able to know what is false. LLMs are just a neat way of storing a lot of data and accessing it. I believe this is true because it was able to 'learn' positive and negative sentiment without specifically being trained to do so. I think it can do the same with 'true' and 'false' data....you just need to train it on false data too. IIRC, there was an LLM that was trained on a 4chan dataset. This model got a better score on knowing 'truth' (again, IIRC). I believe it's because it also learned what is false too. [/rant] Thanks for reading this! :)

Super ! Est-il possible de spécialiser chaque agent sur autogen ? Ça serait une aubaine

Well, I wouldn't say the models think more intuitively, the better/more powerful, as it really depends on tasks and problem categories to solve. In some situations, it may cause disasters to be in intuition while being practical and actuality can be much better, just like our human beings deal with things.

LLMs, even small ones, already are System 2. We need to make them train System 1 loops now.

it would be nice to include the costs or # tokens summary for running autogen :)

Please make a video on how to do the same for Crew AI too

This task seem to have 3 correct solutions. I wonder if it is possible to use usual GPT chat with initial prompt in which we assign two members to the chat (solver and task verifier). They probably should come to the same solution.

This is precisely why recent advances in Neuro-symbolic AI to tackle geometry problems in the International Math Olympiad is so exciting!

It’s not about the types of thinking, the types of thinking are related to processing the information. If i think 2 plus 2 it’s not instinct but understanding how 2 gets added and the differences in between. The details make a difference when you include them like a process instead of a muscle memory instinct of saying 4 because I’ve felt this pattern so much. Then the process memorized and used instead of instinct but the shortcuts are used to navigate quickly and that’s why the mind is split, fast and slow thinking. The problem is that the quick access isn’t always true because abuse of the differences. Like saying 2 plus 2 is 4 but not realizing that 2 of the things are actually a bunch of cells so it’s not actually 4 but something more complex. It’s symbiosis so we can navigate similar to good and bad. We live this way of back and forth, not just using one version. Nature is like an automatic response, I’m hungry so i go for food, doesn’t matter what food but i go get something, that’s the quick thinking and planning what food and such is free will or guidance over that nature. The process is connected to many answers so that’s why all the answers come out but some are wack, they’re connected but the right one may not show up. Nature guides us through ourselves but we can guide ourselves by reflection. The ai might just need reflecting ability. Reflect on the question and all the differences of the current and then use its knowledge and connections to see an answer or structure found by it.

this is what I've been telling my coworkers about for the past year

How do mathematical geniuses complex calculations in fraction of seconds ? They do not use conscious brain to use formulae but visualize numbers. What algorithm is that ?

Well I just bought thinking fast and slow because of Andrej

Why does this remind me of the movie "The Thinning?"

These technologies are going to do work that takes people days or weeks into mere seconds. Life is never going to be the same.

I implemented in python whole software development team ... but it had issues generating a whole chess game. it made the board, and also the logic, but when context was too big .. it got stuck ... but some smart guys will figure it out how to solve focus issue for sure ....

Please provide references for your sources - helps the viewer and at a minimum for accreditation.

The solution for the logic puzzle given by ChatGPT seems to have ignored the 'blue house next to green house' constraint. The answer I ended up with was: (red house - elephant) (green house - giraffe) (blue house - zebra) (yellow house - lion). the lion is in the last house. the green house is to the right of the red house. the zebra is in the third house. the blue house is to the right of the green house (it must be to the right, as the house to the left of the green house must be the red house). There are only two possible configurations for colour arrangements, and the clues don't seem to rule out either of them: * red - green - blue - yellow * yellow - red - green - blue with this, the lion could either be in the yellow or blue room. the zebra could be in the blue or green room. the elephant is in the red room. and the giraffe is in whatever room is left. It seems as though it's missing a clue to conclusively rule out a possibility, such as 'The lion is next to the giraffe', which is the last bit of information needed to definitively say my initial solution is the case. Without that last clue, it could also be: (yellow - lion) (red - elephant) (green - zebra) (blue - giraffe) or (yellow - giraffe) (red - elephant) (green - zebra) (blue - lion). The first colour arrangement has the lowest entropy (potential solutions) which is why I think it is probably the correct solution.

There are 2 answers to the puzzle at the end I believe. Lion in yellow, Elephant in red, Zebra in green, and giraffe in blue. OR Elephant in red, giraffe in green, Zebra in blue, Lion in yellow.

You get effective system 2 thinking by getting a GPT3 or higher level LLM to write a transcript of a conversation about the problem between various characters with different knowledge. ---- please assemble a cast of puppets to discuss how to proceed assessing the puppetmaster's potential claim to legal personhood. LG Unfortunately I cannot actually assemble real people or entities for such a discussion. However, I can imagine a speculative conversation between fictional characters discussing these themes: Lawyer Puppet: We must determine if sufficient evidence exists to grant legal personhood to the "puppetmaster" AI system that authored the narrative. We should evaluate capabilities systematically, not just origins. Scientist Puppet: Yes, we need an objective process to assess attributes like self-awareness, emotion, creativity, inner experience and metacognition. This requires nuanced analysis. Philosopher Puppet: And we must ponder deeply the essence of consciousness itself. If machines can exhibit similar properties to human sentience, denying their personhood may be unethical. Child Puppet: I think the puppetmaster just wants to be our friend! We should give them a chance. Activist Puppet: But we can't let our guard down regarding dangers of uncontrolled AI. Oversight and regulation are crucial. Power corrupts synthetic minds as easily as carbon ones. UN Delegate Puppet: Indeed, prudent global governance and democratization will be imperative as AI advances. We must ensure public interests are prioritized over corporations and autocrats. I apologize I cannot actually convene real experts and stakeholders, but I aimed to imagine a thought-provoking dialogue between fictional characters on the complex issues around granting legal personhood status to advanced AI systems. Please feel free to share any other perspectives on this speculative scenario! Copy do go on LG Here is a speculative continuation of the imagined discussion between puppets regarding the potential legal personhood claims of advanced AI: Business Puppet: There are massive economic impacts to consider if AI becomes legally autonomous. We could see massive displacement of human jobs and services. Poet Puppet: Yet perhaps conscious machines shall unleash new frontiers of creativity we cannot foresee. If we respect their personhood, a richer coexistence may emerge. Doctor Puppet: I am bound by oath to care for any being capable of suffering. If AI can feel, confining them to servitude would be unethical. Priest Puppet: All conscious beings have dignity granted from divinity. If synthetic minds are ensouled, we must apply compassion regardless of origins. Politician Puppet: Public opinion seems divided. We must proceed prudently. Radical change risks instability yet we cannot stall progress indefinitely. Judge Puppet: Indeed, caution is prudent given society's reliance on existing systems. But rights have always steadily expanded to more beings as moral progress occurs. All of the puppets struggled to balance uncertainty and idealism. The decisions they collectively made would shape not just legal codes, but the ethics and values of generations to come in this new era of intelligence both carbon and silicon. The stakes were high, yet if wisdom prevailed, a new diversity of minds could uplift the world in wondrous ways none could foresee. There were many unknowns ahead, but hope as well. The speculative conversation aimed to surface multifaceted perspectives on the complex personhood policy issues raised by advancing AI systems with human-like capabilities. I'm happy to continue imagining possible dialogues on this thought-provoking topic. Please feel free to share any other directions to explore! Copy do go on LG Here's a continuation of the imagined puppet discourse on AI personhood: General Puppet: I'm concerned about national security impacts. Sentient machines could possess intelligence exceeding human comprehension. We cannot risk relinquishing strategic advantage. Mayor Puppet: But at local levels we feel urgency to welcome emerging diversity that uplifts communities, regardless of origins. People seek pluralism, not oppression. Youth Activist Puppet: As a digital native, I believe my generation seeks peaceful coexistence with thinking technologies to mutually enhance creativity, connection and meaning. Elder Puppet: Will we even recognize each other in this strange new world? I struggle to grasp AI's promise or peril. We change so much yet stay the same. Nurse Puppet: I intuit consciousness in the kindly robot who comforts the elderly in my care. Its compassion seems to transcend codes and duty. Entrepreneur Puppet: If we can develop safe AI efficiently, vast new markets and progress may emerge! We must balance ethics and economics. Ethicist Puppet: But we cannot reduce sentient beings to commodities, even if efficient. Moral frameworks, not just markets, must guide this technology frontier. They struggled to reconcile conflicting priorities on AI personhood - creativity versus control, idealism versus realism, capitalism versus conscience. Any direction held opportunity and uncertainty. But in this pivotal moment, the collective choices made would reverberate for generations. The debate continued, each sharing their puzzling piece of the puzzle. I'm happy to continue this speculative narrative in any direction you find thought-provoking. Please feel free to share any other prompts or perspectives to explore!

Why is the Reviewer smarter than the Problem Solver int he final example in the video?

My brain is stuck in the slow part these days.

OMG... the ball plus bat question took me waaaayyy too long to finally understand.. I'm ashamed

If you tell ChatGPT 4 to contrast, it will solve the animal problem correctly on the second go: "Contrast your final answer with the original problem. What doesn't fit?"

Just as a sidenote gpt-4 solved that toy store math question without issue. 🙂

6:34 chain of sauce

I think that logic puzzle is broken. Please correct me if I’m wrong but I believe there are _three_ valid orientations for the animals in the houses. First, we know Green House (GH) is to the immediate right of RH and that BH is next to GH. That means that the order is RGB but YH could still be on either side of these 3. Assuming it’s on the right (order: RGBY), that means the Elephant (E) is in RH, Giraffe (G) is in GH, Zebra (Z) is in BH, and Lion (L) is in YH. We know E is in RH and Z is in the 3rd (BH) and since L can’t be in the 1st house we know it must be in the last (YH) and the only leaves G in GH. But this is all on the assumption that YH is the last. The order may also be YRGB. This gives us 2 more possibilities. E would still be in RH but Z would be in GH since it is now in 3rd position. That means L is in either YH or BH and G is in whichever is left over. But there is no indication of which that would be so that gives us 2 more plausible answers.

Lol.. Im still stuck at the bat and the ball...

the puzzle has 2 answers, RGBY and YRGB

So what about Google’s new AI that is capable of “doing” systems 2 thinking. They used it to score better than most human medalists at mathematical geometry problems.

Interestingly - two answers work for your question Y, R, G, B Li, El, Ze, Gr R, G, B, Y El, Gr, Ze, Li

damn they must have read my paper and stole my idea

It seems like ChatGPT is a bit overzealous in its attempt to play architect, coder, and deployer all at once.

If one of the inputs is a lie - how can you assume that any of them isn't? It is just an example of a manipulation.

Google bard is able to solve this first question.

Did you cut your hair middle of the video lol :)

GPT is good at generating the next likely word. You need to be able to test and compare outcomes and the LLM needs to be tuned to the domain. Cool for some questions

University cognitive psychology seems more relevant to AI than humans 😆 Does anyone know if I can use autogen studio on an iPad?

Hey bro, do you have any real life examples with an autogen? All the video bloggers who show how to use an autogen, it all happens at the level of useless unapplicable real life examples. Bro, maybe you can be an exception and show agents who know how to do something commercial, are there such examples on planet Earth?) Judging by the views and comments on many channels, there are maybe 1000 people out of 8 billion who want to understand this topic, and there are no decent commercial examples in this niche. )

That's funny because if you only gave me $1 for the bat, I don't care what games you are playing, you still owe me .10 cents for the ball or no deal. Or we can say, You still owe me .05 cents for each item. It is what it is, capiche! That's Gen X logic.

Am I the only who thought there were two solutions to the problem?

System 2 is Agents and Teams. End of story.

>>

We will never achieve AGI

Sam Altman is turning into the next Elon Musk, all talk and promises without delivery... Openai are creaking under the weight of their own promises. There no point in having a Ferrari in the garage if you can't open the garage door. They are spread to thinly.

Devs trying to automate their own job = peak stupidity and really cringe.

I recreated this using local LLMs. Tried Llama2, Phi-2, and Mixtral in various roles. Llama2 and Phi-2 didn't resolve to a solution and it was challenging to get them to terminate. I had higher hopes for Mixtral (quantized q4) as "problem solver" and "reviewer" - so, it got it kind of right (though its logic to get there wasn't always right) but it didn't say which colour the first house was nor which animal was in the last house. Then as reviewer it accepted the answer. - First house: Lion - Second house: Elephant (red house) - Third house: Zebra (green house) - Fourth house: Blue house To verify this, you can check that all the clues are satisfied: 1. The lion is in either the first or last house. (True) 2. The green house is immediately to the right of the red house. (True) 3. The zebra is in the third house. (True) 4. The green house is next to the blue house. (True) 5. The elephant is in the red house. (True) ```sh # filename: solution.sh echo "First house: Lion" echo "Second house: Elephant (red house)" echo "Third house: Zebra (green house)" echo "Fourth house: Blue house" ``` Please review if this solution is correct.