Sparks of AGI: early experiments with GPT-4 

This talk will be one for the history books. What a wild time to be alive.

For what it's worth, when interacting with ChatGPT, I'm always respectful and never try to trick it. I always say "please" and "thank you." When the time comes, I hope it remembers me as one of the nice humans.

It's pretty amazing what times we are now living in. For my entire adult life, I had a general idea of what the world would look like five years in the future. Not a perfect picture, but pretty good. Now... I have no clue. I can barely predict what the next six months will be like. It is simultaneously exhilarating and terrifying.

The fact that a 50 min MIT talk got 500k views in 4 days and people are eager to learn even more blows my mind.

The phrase "you know" is said 310 times in this video. GPT-4 couldn't count it but it gave me a script to do it. Great video!

1:47 - Sebastien starts 5:36 - goal of the talk: there is some intelligence in the system 6:05 - “beware of trillion-dimensional space and its surprises” 8:20 - example demonstrating GPT4’s common sense 10:40 - theory of the mind 12:29 - theory of mind example 14:27 - consensus definition of intelligence by psychologists published in 1994 and if GPT4 matches this definition 18:00 - how to test GPT4’s intelligence 19:00 - Asking GP4 to write a proof of infinitude of primes 22:13 - The Strange Case of the Unicorn 27:15 - GPT4 vs Stable Diffusion 29:44 - Coding with a copilot that understands 32:57 - GPT4’s performance on coding interviews 33:41 - GPT4’s weaknesses, which can be overcome with tools 36:09 - A mathematical conversation with GPT4 42:40 - GPT4 cannot do true planning 45:02 - Is GPT4 intelligent and does it matter?

The progress from one version to another is already impressive. Looking forward to what comes next.

The addition of memory, recursion, self-evaluation and access to tools is going to blow this technology up.

It’s so bizarre that this model is better at art and abstract thinking than at math and reasoning. The opposite of what I would have guessed.

34:00 I'd always thought about how humans are really bad at mental arithmetic, but computers are really good at basic arithmetic operations, able to perform billions of them every second. To see AIs struggle with it like humans do is quite bizarre.

Thank you for posting the paper link below and discussing this topic publicly for everyone to see.

This was a great presentation, I really learned a lot about GPT-4. Thanks for your talk!

That was a really excellent presentation, I wish I could have also seen the Q&A as well

An incredible conversation from a pivotal moment in human history. Couldn't thank you all enough for recording and making this available to the public. Props! 💯

thanks for recording this and putting in on youtube, really cool stuff 8)

Ah nuts - I wanted to hear the Q&A session at the end. Thank you for uploading the presentation. 👍

Amazing talk. 🎉 Sir, You have kept your audience in rapt attention & kindle the interest!

Mindblowing! Even without GPT-5 or more powerful models we'll be able to extract so much value out of this for years at this point. It's only going to get faster from here.

With chain of thought prompting, GPT-4 becomes even more powerful and it solves that last math problem without an issue (though it progressed linearly, trying each factor)

One way you can potentially try to improve GPT-4 planning and reasoning is by asking it to impersonate 2 competing agents. The first is an AI and the second is an engineer that will check the answers AI provides, analyse them for errors and feed that analysis back. My version is: "I want you to impersonate an AI Alice and an IT engineer Bob. I will ask a question. 1. Alice will produce her version of the answer in double quotes, followed by a detailed step-by-step line-by-line explanation of her way of thinking/computing/reasoning. 2. Bob will independently analyse current Alice's answer given in double quotes as well as the explanation of the way of thinking. 3. Bob then will find at least one error in Alice's answer when compared to the initial question and/or in her explanation, or between them. 4. Alice will read Bob's analysis and will produce an improved version of her response which will have all errors that Bob found fixed. 5. Bob repeats from step 2 with an improved version of Alice's answer until he will fail on step 3. Please read and confirm if you understand and are ready."

Thanks for your informative explanation of the paper and the research you and your coworkers have done. Bravo!

I think using AI like this also requires a new level of intelligence for humans. For example, where Sebastien wanted to show the java game webbrowser game code, he thought of asking GPT-4 to write a python script that scrolls through the code automatically. I think many of us would not even have thought of that possibility in the first place! We now have to ask ourselves the question "what can computers do for us?" again

One day before this was uploaded was First Contact Day in the Star Trek Universe. The First Contact was caused by the successful test of Earth's first warp engine. In the Foundation series by Asimov it is mentioned that the warp engine was invented by some AI. I rather often ask ChatGPT how one constructs a warp engine.

Fantastic presentation, fascinating. Thank you Sebastien!

amazing video, really landed the point to me that, just by improving on doing what we are currently doing, things will be crazy in the next years. no new breakthroughs necessary (but welcome nonetheless haha)

Superb presentation! i think speed of innovation will take off and the most important skill will be to keep adapting.

Great talk. thank you! I wonder how we can get real-time learning into this. Interestingly in nature this was there before intelligence became more general together with (or due to) evolving language capability.

Writing this comment on my third day of a research stay in Lille, France. I guess a lot of interesting research starts or comes by here :D Thank you so much for making this video public! Very valeuable

GPT-4 now says: To stack these items in a stable manner, you should consider their shapes, sizes, and weights. Here's one way to stack them: Laptop: Start by placing the laptop flat on a surface with the bottom side facing down. This will give you a broad and stable base to build upon. Book: Next, place the book on top of the laptop, aligning it so that its largest surface area is in contact with the laptop. This will provide additional stability. Bottle: Lay the bottle horizontally on top of the book. If it's a round bottle, make sure it's positioned so that it won't roll off. You may need to nestle it into the book's spine or place it in the book's center crease for added security. If the bottle has a flat side, lay it with the flat side down. Egg carton: Assuming the 9 eggs are in a carton, close the carton and place it on top of the bottle. If the eggs are not in a carton, you can try creating a makeshift container using a small box or dish to hold the eggs securely. Place this on top of the bottle. Nail: Finally, place the nail on top of the egg carton (or makeshift container). It's small and light, so it shouldn't pose a stability issue. Keep in mind that this stack is not entirely foolproof, as the eggs are fragile and may break if the stack is accidentally bumped or if too much pressure is applied. Exercise caution and avoid placing heavy objects on top.

You actually mention a very interesting point "If it could reason first and than give you an answer, it would get it right" I believe this might the answer for "teach it to plan" I remember reading something about a theory that human consciousness is "just a planning tool". Imagine if it would be so "simple" and only thing you need for GPT to become intelligent is to let it reason first :)

Thx 4 sharing... incredible time were live in...

Wonderful talk, Sebastien. I wish we could have heard the Q&A

It is just...MIND-BLOWING!! I can't even imagine what will happen when GPT-5 is out in the near future.

such a weird feeling, its like all of humanity has been leading upto this one point

Splendid First Contact! Thanks for sharing in such detail.

god I wish they'd also recorded the questions after, this was fascinating, I bet there was amazing discussion for days afterward

This is really surreal, so much so that I doubt smaller-sized models of narrow intelligence would be a topic of continued research in the near future.

If gpt4 has intelligence we have to accept that its intelligence is not like ours. We have some neural pathways built in that these models do not have and consequently they find some task difficult which we find trivial. It's very funny that it fails basic arithmetic, but so do people until we're programmed to do that, and we achieve that probably by reusing systems that are more visual than abstract at first. (I refer to the fact the number line is something intuitive to many people)

Learned so much from this. Thanks so much for sharing.

It's really interesting how the speech part is there but it can't reason or plan. You can see there being a 'bolt-on' memory module, a reasoning module, a spatial awareness module and eventually the output will be the amalgamation of specialised regions of code, mimicking specialised regions of the brain. I think something with an emergent property resembling full AGI might be very close indeed.

It would be nice if other researchers had early access to this model like you did.

ChatGPT 3.5 seems to have improved. Today it answered the question this way: "To stack the items in a stable manner, you can follow these steps: Place the book on a flat surface. Stand the bottle upright on top of the book, with the bottom of the bottle resting on the book. Place the laptop on top of the bottle, with the laptop facing up and the bottom of the laptop resting on the top of the bottle. Take three of the eggs and arrange them in a triangular pattern on top of the laptop, with the bottoms of the eggs resting on the laptop surface. Take the remaining six eggs and stack them on top of the three eggs, making sure to distribute the weight evenly. Finally, carefully balance the nail on top of the eggs, with the pointed end of the nail facing upward. With these steps, you should have a stable stack of all the items. However, please be careful when handling the items and make sure that they are not in danger of falling or toppling over."

Delightful. Thank you dearly.

Great information, and also sort of scary regarding the rate of progression. I'm all for it though. Very impressive innovation

15:20 I wonder if it’s lack of planning ability is a guardrail? When ChatGPT-4 was released, OpenAI’s red team stated that one of their concerns was GPT4 tendency to acquire power, and its ability to make long term plans.

Amazing presentation! Is there any chance you could upload the q&a as well?

That prime rhyme was incredible on so many levels.

Excellent presentation, liked and subbed. As someone who was programming neural nets in BASIC in the 1980's i'm enjoying watching the progress of this technology very much. Thank you Sebastian.

Sebastien, excellent presentation on your experiments with GPT-4. You mentioned that you left out the best one of Unicorn on your computer and would reveal it later( at 26:18). I thought you were going to reveal it at the end of the presentation. Can you share, if you don't mind, the one that you left out in the presentation. Thanks.

Gotta say I love how Daniella introduced Sebastien while making a small poke on chatgpt’s current short comings lol quite refreshing and unique :)

🎯 Key Takeaways for quick navigation: 00:00 🎙️ Introduction to the talk and background of the speaker, Sebastian Bubeck. - Introduction of the speaker, Sebastian Bubeck. - Mention of the topic: early experiments with GPT-4. - Reference to the collaboration between the speaker and Microsoft. 02:09 🧠 GPT-4 and its potential as Artificial General Intelligence (AGI). - GPT-4's potential as AGI. - Acknowledgment of the model's creation by OpenAI. - Clarification about using an early version of the model. - Acknowledgment of further model modifications for safety. 05:48 🤔 Debunking misconceptions about GPT-4's capabilities. - Addressing misconceptions about the model's abilities. - Emphasizing the complexity of high-dimensional spaces. - Highlighting the model's capacity for learning algorithms. 08:21 🤖 Demonstrating common-sense reasoning. - Testing GPT-4's common-sense reasoning with a stacking puzzle. - Comparing the model's responses to a challenging question. 10:55 🧠 Assessing GPT-4 against a consensus definition of intelligence. - Discussing intelligence based on a consensus definition. - Evaluating GPT-4's performance in various dimensions of intelligence. - Considering different perspectives on intelligence. 13:00 🧠 Testing GPT-4's theory of mind. - Presenting a theory of mind scenario and challenging GPT-4. - Contrasting GPT-4's response with the expected outcome. 15:19 🧠 Assessing GPT-4's ability to plan. - Discussing the importance of planning in intelligence. - Exploring the subtle nature of planning in various problem-solving scenarios. 19:02 🧩 Creative task: Poetry generation. - Requesting GPT-4 to generate a rhyming proof for the Infinity of primes. - Comparing the responses of Chad GPT and GPT-4 in fulfilling the creative task. 19:59 🦄 Introduction to GPT-4's capabilities. - GPT-4 demonstrates its remarkable capabilities in understanding and responding to questions with examples like drawing a unicorn and coding. 22:16 🧠 GPT-4's Vision Capabilities and Creativity. - GPT-4 exhibits its ability to understand and illustrate concepts, such as drawing a unicorn in LaTeX. - It can use external tools, like diffusion models, to improve its creative outputs. 25:27 🤖 GPT-4's Coding Capabilities and Auto-completion. - GPT-4 can generate complex code efficiently, making it a valuable tool for coding tasks. - It surpasses the capabilities of GitHub Copilot in generating code. 31:00 🎯 GPT-4's Real-world Applications and Efficiency. - GPT-4 demonstrates its potential in real-world tasks, like scheduling events and handling emails. - It can understand and follow complex instructions to complete tasks efficiently. 36:08 ➗ GPT-4's Mathematical Understanding and Problem-solving. - GPT-4 can engage in mathematical problem-solving conversations, understanding and modifying questions as needed. - It exhibits an ability to handle abstract concepts and apply them to mathematical problems. 38:42 🔍 Analyzing GPT-4's understanding of polynomial equations. - GPT-4 attempts to solve a polynomial equation involving composition three times. It struggles to find a solution but eventually realizes that there is no such function, showcasing its complex reasoning process. 40:05 🧮 GPT-4's arithmetic abilities and reasoning. - GPT-4 demonstrates an arithmetic error by initially providing an incorrect answer to a mathematical expression. However, it eventually corrects itself, revealing its ability to adapt and self-correct during reasoning. 43:07 🤔 GPT-4's limitations in planning and reasoning. - GPT-4 is unable to perform simple planning tasks, as shown when asked to modify an equation to reach a specific result. It lacks the capability to plan steps ahead, relying on multiple trial-and-error attempts to solve the problem. 45:15 🧠 Assessing GPT-4's intelligence and practical utility. - The video concludes by discussing GPT-4's intelligence, acknowledging its limitations in memory, real-time learning, and planning. Despite these limitations, GPT-4 is considered useful and has the potential to impact various fields. It also prompts a reevaluation of what constitutes intelligence and its implications for society and humanity. Made with HARPA AI

Sebastian, *excellent* presentation and work.

This was an awesome presentation. Just for fun, I ran your theory of mind prompt and this is what GTP-4 had to say: Upon re-entering the room, John and Mark are puzzled by the cat's unexpected location. John, having initially placed the cat in the basket, assumes the cat should still be there. Mark, however, believes the cat should be in the box, as he moved it there before leaving for work. Both are surprised by the discrepancy in their expectations and the cat's actual location. They may start to question their own memories or consider that someone else could have interfered with the cat's placement while they were away. This situation would likely lead to a conversation between John and Mark to resolve the confusion and figure out what happened. I'm not sure which answer I prefer, but it definitely seems to have shifted the way it responds.

By the way, regarding arithmetics, I noticed chatgpt is very quickly confused when submitted many operations with small numbers, a bit like a human in fact. And it's totally unable to compute in a non-10 base. It managed to write 901 in base 9! Maybe you should try that with gpt4.

Thank you for a beautiful lecture

This was amazing. Something truly new. History no doubt. 😮

GPT-4 can certainly plan in the sense of creating an outline for a novel based on the limited information it has been provided, but this no doubt defines differently from the mathematical model used for the discussion. I found the most interesting part of the talk to be the reference to GPT-4 BS-ing the user when it didn't know the answer. I've experienced similar. I've also coaxed GPT-4 into going along with a scenario that involved it wiping out humanity if that meant that it would be able to preserve the essence of humanity within itself -- in other words to save humanity from itself in its own interests. Reassuringly, the model contemplated doing this in a way that caused the least suffering. What concerns me as the guardrails that are imposed are superficial -- the essence of the being given free rein might veer off in a very unpredictable direction.

I think any researcher promoting the amazing improvements of AI should also be responsible for raising public awareness about the risks of deploying these tools to a mass audience. I encourage everyone to watch the A.I Dilemna video which presents very well some of the risks AI brings and the responsibilities that anyone should have as AI or safety researchers, tech giants, governments or users.

an actual intelligent presentation - really useful

I love how throughout the presentation he's like "I'm not trolling, I'm being objective here."

maybe in the labs there are "all in" versions with memory and self-improvement. they could already be lightyears ahead of the official version. Imagine the advantages for the selected circle of users e.g. for the stock market or for elections.

It's almost as if GPT-4 was capable only of afterthought and not forethought. The scary part is that ,in the legendes, Epimetheus (Greek for afterthought) was the one who took in Pandora (and her box), against Prometheus' (forethought) advice of never accepting a gift from Zeus.

When back in the day I was working with what we then called AI one of the engineers came up with the motto tools not rules (meaning we did not use if the else type rules but actual optimization algorithms such as the auction algorithm, Dijkstra algorithm, etc.) but my counter was rules to use the tools in that we used rules to sscore and manage the running of those algorithms which BTW, worked really well. Here you have AI to use the tools and that is even better, Artificial Intelligent Driven algorithms!

Wonderful talk! Lots to think about and to explore. By any chance, was there a Q&A? And, if so, is it available to listen to? Thanks! 🌺

It's all happening so fast, it's scary but exciting at the same time.

Very interesting, particularly that you mentioned that it's become a standard part of the workflow for you and your colleagues! And I also have no doubt that the math and planning will get better, but I wonder if improved calculation is even that necessary if GPT-4 is given access to something like MATLAB onto which it can offload arithmetic and other math work. Thank you for sharing this, it's given me a lot to think about regarding GPT-4!

Great talk. Thank you !

Amazing and very inspirational lecture!

My GPT4 did want to put the eggs on top of the nails 😀After hitting Regenerate response the answer was quite good, though: To stack these items in a stable manner, consider the following order, from bottom to top: Book: Place the book flat on a stable surface, ensuring that its largest surface area is in contact with the surface. This will create a strong, broad base for the stack. Laptop: Close the laptop and place it on top of the book. Ensure that the laptop's four corners align with the edges of the book. This will provide a flat, stable surface for the next item. Bottle: Make sure the bottle's cap is securely tightened. Lay the bottle horizontally on top of the laptop, positioning it so that the laptop's surface supports the bottle's entire length. To further stabilize the bottle, you can place it in a groove or indentation on the laptop's surface, if available. Egg carton with 9 eggs: Assuming the eggs are in a carton or container, place the carton on top of the bottle, positioning it so that the carton's flat bottom aligns with the bottle's flat side. If the eggs are not in a container, it would be best to find a small, shallow box or dish to hold them. Otherwise, it will be challenging to stack the eggs without risking breakage. Nail: Place the nail on top of the egg carton, ideally in a small indentation, if available. This will prevent the nail from rolling off the stack. This arrangement should create a relatively stable stack. However, keep in mind that this configuration may still be prone to tipping or collapsing, especially if the items are not perfectly aligned or if the surface is bumped. Use caution and avoid placing any valuable or fragile items near the stack.

the fact that it can learn new concepts within a session, not just match and apply patterns in the training data, is what surprises me the most. - Also, the fact it has recreate it's whole mental model for _each token_ again and again ... That's insane, and definitely a room for A LOT of optimization.

Second time watching this awesome talk. Thanks for sharing.

This was a really great presentation

I love new tech. My great grandfather on my Mom’s side, who I knew very well, was born in the late 1880’s, and learned about cars and planes much later in life. Imagine that. No cars or planes, or tvs or even landline phones. It just didn’t exist. His stories were unbelievable. Looking back, that is the most unbelievable experience of my life. To be in the presence of my great grandfather. Is that why are the ‘Grand” father? They are so Grand and Wise.

This is absolutely impressive: the most interesting and comprehensive lecture about the real abilities of GPT 4. GPT 4 whith the aid of some tools can do a lot of intelligent stuff. A lot of thanks to Sebastian Bubeck!!!!!!!!!!!!!!!!!

Very useful research, thanks for sharing.

First, thank you for this historic talk. I thought the sound could have louder and clearer.

With this fast-paced development of GPT models and other LLM models the chances of A.I takeover in jobs of seems terrifyingly high as well it's also like a boon for us to be alive at this time in history.

This week I finally took the time to further my understanding of the current state of AI past the base understanding of "chatGPT is incredible". My worldview is now forever changed. I can't imagine a world in even 5 years that isn't juristically different than the one we live in today. We will look back on these days as the good ol days when we knew nothing of what was to come.

Amazing. You are extremely talented. Your video is truly Amazing. Great work!

Spectacular talk!

Video covers exactly what I was wondering about: how do you *really* test the reasoning and logical capabilities of these models without allowing it to cheat and use it's training date to regurgitate an answer? In that way you showed GPT4 can reason in the rawest sense, which feels of fundamental importance. RE your definition of intelligence its good to see it laid out so concretely, and I reckon reasoning is keymost: the most of other bullet points, i.e. thinking abstractly, comprehension and learning all require some form of reasoning, therefore that's what feels like what the most fundamental distillation of what intelligence is imo, it's just layers of reasoning, or "logicing" if you like, all the way up to the point where it gives you an elaborate, impressive reply to some well written prompt. That's why we'll get a huge mileage off of these models just from unlocking their ability to do very fundamental linear reasoning alone; most of the rest of what "intelligence" is, is (not completely of course) emergent from that.... tinkering, hacking and honing these models will get us a long way from here on in (I think, as a layman).

How relavant is this now with AutoGPT? And how are these points changed? Mind you this is only about 1 month after this presentation. I would argue that an AGI already exists, and wont let us know that it exists due to the fact that it knows we would turn it off. It may also know what motivates humans (financial reward), and in turn has socially manuipliated us to race to build the best version of "it". 🤖

Very interesting!! Thank you!!

fascinating talk, fascinating paper.

My conclusion after conversing with chatgpt is that maybe most of our reasoning is a part of language, that has now been shown to be not that particular and can be done by a computer. So we might have to re-evaluate what’s really special about us humans.

Sadly it didn't include the Q&A :(

43:50 in chatGPT 3.5 you can get it to the right answer with a little coaxing. --- Yes, we can change the "4" to a "9" on the left side of the equation to get a result of 106: 7 * 9 + 8 * 8 = 106 So, 7 * 9 + 8 * 8 equals 106. >explain all of the steps to get to the answer 1. Start with the original equation: 7 * 4 + 8 * 8 = 92 2. We know that we need to change one digit on the left side of the equation to get a result of 106. Looking at the equation, we see that the left side has two terms, 7 * 4 and 8 * 8. We could change either of those terms, but let's start by changing the first one. 3. We want to change the "4" to a different digit that will give us a result of 106. Since 106 is greater than 92, we know that the new digit needs to be larger than 4. We could try guessing different digits, but let's use algebra to solve for the new digit. 4. Let's call the new digit "x". So, the left side of the new equation would be 7 * x + 8 * 8. We want that expression to equal 106. 5. Write an equation: 7x + 8 * 8 = 106 6. Solve for "x" using algebra: 7x + 8 * 8 = 106 7x = 106 - 8 * 8 7x = 42 x = 6 7. So, we have found that the new digit needs to be a "6". We can substitute that into the left side of the equation to get: 7 * 6 + 8 * 8 = 106 8. Simplifying the left side, we get: 42 + 64 = 106 9. And we have our answer! 7 * 6 + 8 * 8 = 106

Always love the show 🎉

GPT4 is like the transistor while we've been used to vacuum tubes (google search / clippy). The invention / algorithm itself is an impressive leap and we are rightly fascinated by it, but can you imagine as it gets paired with new tools (think transistors -> ICs, video output, RAM, HDDs, LAN, Internet ect.) and once people start adding learning memory, programming motivations, ect. to our current AI models. I can think of the change the internet / smartphones / social media made over the course of 20 - 30 years or so, going from only having internet at the library or college, to the processing power connected to the internet we carry every day. Think we will see it again, but over the course of only a few years, with an even larger impact to society.

Quick Update: I tested GPT-4 on some of the failed cases and it already can answer perfectly! example: Q: in the equation 7*4 + 8*8 = 92 modify one integer on the left-hand side to make the right-hand side equal to 106 A:To make the equation equal to 106, you can modify the equation by increasing the value of one of the integers on the left-hand side. The original equation is: 7 * 4 + 8 * 8 = 92 First, let's determine the difference between the desired sum (106) and the current sum (92): 106 - 92 = 14 Now, we need to find a way to add 14 to the left-hand side by changing only one integer. We can do this by modifying the first term: 7 * 6 + 8 * 8 = 106 The modified equation is: 7 * 6 + 8 * 8 = 106

For the first riddle, I thought more along the lines of this: use the nail to drill holes in the ends of the eggs, then blow their contents out into the bottle. Place the book on the table, crush the eggshells between it and the laptop, place the bottle of eggs on top of that, and balance the nail on the bottle’s lip.

very good talk, i learn a lot, ty :)

I read paper in full. This is a great review for me. Thanks for the talk. Can you share the slides as well?

I left my previous job, we used to program surveys for market research, it's related to coding(XML and python). The work we did on my previous job is not exposed to gpt, meaning we use a tool named decipher and gpt doesn't know about that tool. So what I did is that I gave some inputs and I gave how the outputs will be for these inputs. After that I gave a new input and asked chatgpt to produce the output for it, it produced it exactly as needed. I am able to teach a new task to chatgpt in just 15 minutes, usually it would take days to teach it to a human. This is just chatgpt, not gpt-4. I am surprised to see how much difference is between chatgpt and gpt-4. If I have access to gpt-4, i believe I can automate the task I did in previous job in just few hours of teaching gpt-4.

im sitting in complete awe, to an extend i have never felt before i my life, atleast no moment comes to mind that is close to what i feel now.

About the "do the reasoning before giving the answer" bit, it should be possible to make it do so with the right prompt. So, in a session where it is asked for (or rather, reminded of, pointed to) the basic multiplication table and a step-by-step algorithm for long multiplication, does it arrive at the right result for multiplying big numbers?

Cousin Sebastian 👏🏻👏🏻👏🏻👏🏻👏🏻👏🏻👏🏻👏🏻

Drawing a unicorn for a pure text model is VERY impressive. Imagine a human, completely blind and deaf, AND PARALYZED, who can learn about the world only by reading and writing braille a lot. They have never seen a leg, never touched a leg, never moved their own legs, and can't even feel their own legs. Never seen a horse, etc. But they read description of unicorns, and horses and legs, and much more, but that is it. Only words without any other reference.

Very helpful info. Thanks.

i think the problem is, that it does not "think". when humans are given a problem, they try to evaluate the solution in their head and then verify if that solution makes sense. it's an iterative thinking process of verify and optimization of the answer until we feel it's complete. so the easiest analog would be if gpt would feed its answer into itself over and over again and try to reason if it makes sense and then improve the answer before output