One advice to host: You need to give your guest space. You are not a salesman. Or a missionary. Challenging them does not mean repeating the same argument over and over again. It was suffocating to listen to your challenges. If it was not for the call and patient demeanor of Chollet, it would be impossible to watch. We were not able to listen to Chollet expanding upon his ideas because host just reversed the clock to zero by repeating the same "but memorization is intelligence" argument. It should be about your host, not showing the supremacy of your ideology or beliefs. If your host is wrong, you can prove them wrong by showing their arguments and ask questions as they expand upon them and then show if they are inconsistent. Not repeating the same thing over and over and over again
One advice to you: You can make this point better without insulting Dwarhesh, who is a young man that is still learning. Perhaps you should try hosting a podcast and see if you do better. Want my guess? You would do much worse than you think.
@@Limitless1717 this is the most mundane criticism of a criticism. Dwarhesh himself is doing a podcast on topics he is not a specialist on and he is openly criticizing and challenging the views of a specialist on a topic here. So maybe he should work on AGI before challenging François here if he were to take your advice seriously (though he should try to educate himself on topics in any case) And I am not doing podcasts but I have taught many many classes with a lot of teaching awards. Not the same but similar when it comes to expanding on topics. When I teach a concept I don't just attack it on the first sentence. I explain it, allow it to ripe. Put different light on different aspects of the topic. I don't try to destroy the whole concept on the first sentence. So my advice doesn't come out of nowhere. And if he puts himself to public spotlight, my criticism is actually one of the most innocent stuff that he is thrown in his direction. If he takes into account he can improve upon. I am mostly criticizing how he is doing some stuff and even provide what he can do better. It is weird that you take offense to that. Anyways it is up to him to do what he wants but I won't watch him anytime sooner again. As it is now, this is really bad way of arguing with anyone even in private, let alone on a podcast. When someone interrupt me like he does and don't know how to argue, in general I just don't bother
@@ClaudioMartella I may have commented too quickly, it seems thar later on in the video I got the impression Dwarkesh was playing devils advocate. Not sure...
@@wiczus6102 Right but I couldn't tell if he meant just in that shorter exchange or if the whole interview was him taking the opposite site for the debate.
@@kjoseph7766 They were clearly putting their own opinions forward, which didn’t align. I would say that put pressure on each other to defend their beliefs. The one dude got super panicked as the French guy kept putting forward clearly articulated ideas and rebuttals.
After listening to this interview and reflecting on it, I was actually thinking that (not Dwarkesh because he's legitimately smart in my opinion) but "people" like Destiny talk extremely fast to compensate for shallow ill-formed thoughts.
@@ahpacific Dwarkesh could help himself out a lot if he slowed down. Simple things like rephrasing what 'french guy' was saying in a way that Francois would agree with would also help tremendously. There is a fundamental difference in epistemology between these two, Francois is emphasising true understanding, and Dwarkesh seems to imply that large gross memorisation leads to understanding- which I don't think Francois would agree with
@@ahpacific I am pretty sure Destiny is very transparent on when his thoughts are shallow or not. Notice when he formulates things as open questions and or the adjectives and intonations he uses to suggest the difference between perspective and fact. You can make a false statement and still create a good converstation like that. People like that are fun to talk to as opposed to people who will only say something if they are fully certain.
@@ModernCentristthat's the impression you get from content focused on hype merchants, alarmists, and those with a vested interest in overinflating the value of current methods. Academia is full of skeptics/realists.
So good to see someone let some air out of the LLM bubble. Dwarkesh might be a little challenged by this, but it’s great to get out of the echo chamber regularly.
This strikes me as moving the goal posts. Chollet doesn't seem to understand how dumb the average person is. Can LLMs replace programmers? Can the average human replace a programmer? Go watch Jerry Springer and then tell me how LLMs won't reach AGI. To the average human, these thing are already AGI. Everybody in this video is in the top 1%. They are so far up the intelligence scale that they can't even imagine what average intelligence looks like.
Wanted to agree with him, but Francois Chollet is way off. From the moment he mentioned not "copying" from stack overflow as some sort of example of humans handling novelty in the wild, it was clear he was idealizing some belief that he holds. He refuses to believe that creativity is mostly interpolation.
Francois Chollet is an amazing guy. The best thing is he, like all the LLM guys, also wants to work toward AGI! He just doesn't think the current LLM paradigm will get us there. I'm really excited to see where this goes because he's challenging the current option space in exactly the right way
This interview actually got me believing LLMs might indeed get us there. The guest seems to believe in a form of intelligence that he idolizes but we haven't really seen. Dwarkesh was spot on that no scientist zero-shots their ideas.
Dwarkesh dismissed a few completely valid answers to try and steer the answer in his preconceived idea of LLMs, I didn’t like that, dude is smart, let him finish and actually take onboard his answer before asking another question
He said he was playing devil’s advocate, calm down. He does that with most of his guests. It generally makes for a more informative and engaging interview than simply taking everything your interview subject says at face value.
@@therainman7777 Dwarkesh said himself that they were going in circles. I think this was mostly due to Dwarkesh not really thinking about Chollet's responses in the moment. LLM hype causes brain rot in smart people too.
It is pretty clear Dwarkesh has a lot of influence from the local SF AI folks. Watch his interview with Leopold. His command of the subject is admirable, and he smartly relies on the researchers in his circle to inform his understanding. Many notable people maintain quite strongly it's only a matter of scaling, and I thought he thoroughly went through these types of arguments. It was a valuable thing to do. What is an example of something Francois wasn't able to eventually effectively articulate because he was cut off?
I think Francois really got his point across in the end there. I was leaning somewhat to the scaling hypothisis side, he made me question that more. In any case you have to give him credit him for actually coming up with interesting stuff to support his arguments, unlike many other critics.
“Coming up with” you mean like on the fly? 😂 but seriously though this guys a verified genius in his field, he didn’t just “com up with it” one morning
There is so much great information here.... I'm at 24:32 and Francois was saying "Generality isn't specificity scaled up..." He seems very aware that the current approach to LLM is bigger, better, more data, and he is right in noting that is not how human intelligence works. We don't need to absorb the whole of human information to be considered intelligent.
Dwarkesh brings up a solid point that no scientist ever "zero shots" their ideas. Francois is partly correct, but he's holding onto some tight beliefs there about creativity, novelty and interpolation.
Train an LLM to solve math but don't include anything related to calculus in the training data. Then ask the LLM to solve a calculus problem and you'll see it fail. That's essentially what Francois Chollet was saying. Isaac Newton was able to introduce calculus based on his FOUNDATION OF MATH (Memory) and actual INTELLIGENCE (Ability to adapt to change)
Francois said the opposite, he actually said that a lot of human math skills rely on memorization too. But actual intelligence to discover/invent new math goes beyond this. This is why even winning a math olympiad would be as meaningless as winning chess, it's old math. Actual intelligence wouldn't win chess but invent the game - without being told so!
Making an AI Isaac Newton vs an AI Calculus Student is a nice and simple way to capture what they're trying to do. Making a great AI Calculus Student is an awesome accomplishment, but we really want a Newton.
Many math teachers I've known say that if you memorize and apply the tricks, at least you'll pass the exams. You won'be great at math, but good enough. Up to some level math is about memory, like chess.
Great Interview. I've personally done about 50 of the online version of the ARC challenge and the gist of solving them is simply to recognize the basic rules that are used to solve the examples and apply that same rule to get the answer. While some are challenging, most are using basic rules such as symmetry, contained or not contained in, change in color or rotation; or a combo of more than on rules. I'm sure that current large LLMs like GPT4 have internalized these basic rules in order to answer questions. so proficiently. What is perplexing to me is why can't LLM extract those rules and apply them to get more than 90% on any ARC challenge. I think that is the crux of the matter that Francois is getting at. If to solve any ARC challenge basically requires one to identify the simple rules in an example then apply those rules, why are LLMs not crushing it?
Because LLMs - once trained - don't extract rules from input data and do another step of applying those rules. That would be precisely the "synthesizing" step that Chollet talked about. LLMs just ingest the input and vomit out the most likely output. The human equivalent is a gut-feel reaction (what we call intuition) without attempt of reasoning.
There was 20mins of: "Can LLMs replace programmers?" "No" "But can they? "No" "But can they? "No" "But can they? "No" "But can they? "No" "But can they? "No" "But can they? XD ... it simply becomes clear that LLMs can't replace programmers when you start using them everyday on your programmer job and realize how Bad they perform when you start to do just slightly complex logic
This strikes me as moving the goal posts. Chollet doesn't seem to understand how dumb the average person is. Can LLMs replace programmers? Can the average human replace a programmer? Go watch Jerry Springer and then tell me how LLMs won't reach AGI. To the average human, these things already are AGI.
@jasonk125 they are not AGI because they can't perform every task an average human can. Also he was trying to explain they probably can't learn new novel tasks. The things at which LLMs excels are problems that have been solved numerous times and there is a lot of data around those. Even so. The world and its operations is not interconnected and digitized enough for LLMs to take over.
@@GabrielMatusevich Well if Chollet wants to redefine AGI, and then say LLMs aren't AGI (which is what he does) then I guess there is no point arguing with him. From his website: Consensus definition of AGI, "a system that can automate the majority of economically valuable work," Chollet's definition: "The intelligence of a system is a measure of its skill-acquisition efficiency over a scope of tasks, with respect to priors, experience, and generalization difficulty." So they should have first come to an agreed upon definition of AGI (which they did not), before arguing about whether LLMs could meet that definition. Your statement: "they are not AGI because they can't perform every task an average human can" is not arguing within Chollet's definitional framework. It is closer to the consensus framework.
@jasonk125 yea, is a good point. That reminds that I don't there is even actual consensus on a definition of just "intelligence" .. which makes it even harder 😆
literally tortures him with "okay but isn't this AGI why is that not AGI?" questions and after not having any positive feedback asks "okay but let's suppose you lost your job to AGI"
I think these are two separate conversations. Host goes technical, guest goes philosophical on how these LLMs currently lack processes that helps them keep up with conversation the natural way. I kinda know what chollet i's talking about. In case of Gemini, it's unable to answer from an existing thread. It goes into a cycle of vérification from its database automatically. It could simply mined from past conversations to respond. It'll be more efficient in my view, use less tokens, less resources and will be more efficient than the current architecture its in. Also gemini can't process "on the fly' response either. For example for breaking news, it won't be notified but hours later.
And in turn Francois keeps redefining what LLMs are doing and what intelligence is. He starts with LLMs just memorize, then they memorize templates but can't generate new templates, then they generalize but only locally, then they generalize enough to learn new languages that they haven't been exposed to but thats not intelligence either.... sure Francois. Call us when you've discovered AGI
@@netscrooge Thank you for that insight. I assumed, considering his roots, that he was simply part of the LLM, crowd. Now I will listen with fresh ears.
@@jameshuddle4712 I'm mostly going by what comes out of his own mouth. But if you can find the right interview, we can also hear what Hinton says about working with Ilya when he was his student, the startling way his mind could leap ahead.
TL;DR the host was combative in a way that made him come off as a salesman for AI rather than having a conversation about what the guest thinks for the first half of the conversation. also, the host refused to budge on his *belief* that current LLMs are capable of real understanding despite the guest's points to the contrary. first time seeing this podcast so i don't have a frame of reference for the normal vibes of the host but he seemed extremely defensive. the guest seemed to keep calmly stating that memorization and understanding are two completely different things while the host just kept referring to anecdotes and examples of things that he thinks displays understanding. the example that set my radar off to this was the obscure language dictionary example. After being shot down the first time by the guest by claiming that the ARC puzzles are a set of tests that it would be very hard to make training data for and if LLM's developed true understanding/adaptive capabilities then they should be able to pass the ARC puzzles easily. the host then tries to bring up the example of the chess models which the guest points out is almost exclusively pattern recognition and memorization and instead of wrestling with that point he moves back to the obscure language point. i think that evasion of the chess point is actually extremely telling. if he truly believed that was a good point, he might have pushed back on it or tried to justify why he brought it up but instead he says "sure we can leave that aside" immediately. maybe I'm being a little cynical. maybe he realized that was actually a bad point for the argument he was trying to make. regardless, he went back to the obscure language point which may have been impressive if it was not for the rest of this conversation to this point. earlier, the host tried to give an example of a simple word problem that had to do with counting. the guest countered that with all of its training data, it probably was just referencing a word problem that it had before which, from my understanding of how these things work, is probably accurate. the host clearly did not understand this earlier point because the thing about language models that the guest has to point out AGAIN is that the training data probably contains similar information. not necessarily data on that language but to my imagination, the data probably contains a lot of different dictionaries in a lot of different languages. dictionaries on top of having similar formats across most languages also typically have complete sentences, verb conjugations and word classes. i can see how the guest's point about memorization and pattern recognition would apply to LLM's in this aspect. as i continue watching i am realizing that this has turned into a debate on whether or not LLM's have the capability to understand and process information as well as synthesize new information which i was not expecting nor did i want. i think it is intuitively understood that current models are not capable of these things. this is why they require so much training data to be useful. there were genuinely good parts of this podcast, but the host insisting that LLMs understand things in the way that humans do were not it. this is a little nitpicky but there was a point when the host said something like 'lets say in one year a model can solve ARC, do we have AGI?'. to me this comes of as extremely desperate because the most obnoxious part of that question is also the most useless. the timeframe in which this may happen is completely irrelevant to the question. the guest at no point argued anything about timeframes of when he thinks AGI might happen. in fact when the guest answered in the affirmative the conversation took a turn for the better. finally if you haven't gone and taken the ARC test, i would encourage you to do so because neither the host nor the guest did a very good job explaining what it was. but on the second or third puzzle, i intuitively understood why it would be hard to get our current generation of models to preform well on those tests. they require too much deliberate thought about what you are looking at for the current models to pass. it almost reminded me of the video game "the witness" in its simplicity with the only clues as to how to solve the puzzles in both games is with context of earlier puzzles.
@@young9534 i probably wont. the host did not really make me want to see more of him. i am kind of tired of being evangelized to about this tech in its current state. i will likely still follow the space and continue to learn more and seek more information. i hope this host does the same honestly. seems like the space is very full with people who want me to either believe that AGI is impossible or AGI is coming next year. i personally dont appreciate either side's dogmatism and will continue to try and find people with a more measured view on this stuff.
You are overly judgmental. He pushed back because the answers felt too abstract. If they felt too abstract to him, they would ALSO feel too abstract to others. There is literally no personal attachment involved. Too many hosts don’t push back nearly enough due to stuff like this.
Thanks Francois for the reminder that we can't just scale our way to mastering intelligence you can't memorize everything. I took this approach in college and it ultimately fails.
@@JumpDiffusion Exactly. Why do they keep parroting the memorization bit. François knows better than to say that there's some copy of the code that the LLMs memorized.
Dude, with all due respect, ease up and let the guest speak. You keep cutting him off with the same question while he's trying to finish his thoughts. Do you even code? If you used different LLM coding assistants for a while, you'd understand what he means. It's essentially glorified Stack Overflow. And the story won't be any different with any other transformer neural networks.
Listened to another podcast where he and his guest spent an hour chasing their tails about nationalisation of ai research. It was a very frustrating listen, and I finally gave up, just like I'm giving up on this one.
I appreciate and very much enjoy these podcasts. I also fully understand the need to play devils advocate. However, to me this felt a lot more biased than most of the other episodes. It's clear which position Dwarkesh has chosen. That's fine, but it really shines through when someone who is not an LLM maximalist is on the podcast. Devils advocate? Yes, always do that. Extreme bias where it becomes waiting for your turn to speak over a discussion? Not ideal in my opinion. I hope if he sees this he doesn't take it personally. Obviously he's very excited about this tech. Most tech folks are either excited or at the very least quite impressed with the advances that have been made over the last few years. I just hope the quality of discussion remains consistent regardless of who is the guest.
It seems like the ARC thing maybe would be difficult for LLMs because they are reliant on visual symmetry that wouldn't be preserved through tokenization? I mean, I'm sure it's not that simple, because then natively visual models would probably be solving them easily. But still, there should be a version of this test that has complete parity between what the human is working with and what the LLM is working with, I.E. already tokenized text data.
Chollet addressed this objection in the video by pointing out that LLMs actually do quite well on these kinds of simple visual puzzles, when the puzzles are very similar to puzzles they've been trained on. So this can't be the answer. We'll find out soon enough how well the multi-modal ones do.
This is the most constructive debate I have watched on AGI to be honest. Bravo Patel for asking the right question to Francois. Definitely makes me think more deeper about all of it
People are actually much smarter on average than one tends to give them credit for. It's just that we are very very reluctant to use System II. We'll do literally everything else before deploying the full power. But if one's life depended on it or there was sufficient incentive, we can be extremely fast learners. We just naturally try not to get challenged this way in everyday life.
Even though I agree with what you're saying- one of the things researchers found that exists as a general difference between persons widely separated along the I.Q. spectrum was that the glucose uptake & thermal output in brains of lower I.Q. people were much greater than those on the higher. This indicates that a more generally intelligent mind is both thermally and resource efficient: expending less fuel and generating less waste per unit of output. What this points to is that some people can activate system 2 with considerably less cognitive burden. Since most of us are pleasure-maximising, instinctually and natively, and since it's distinctly unpleasurable to be in the uncomfortable state of mental strain/discomfort associated with glucose starvation or one's brain overheating, one might expect that behavioural inclination follows from ability. In the same way that a natural weakling doesn't enjoy lifting weights, and so avoids it, an intellectual weakling doesn't enjoy activating system II, and so avoids it. The fundamental reason is the same in both cases: we avoid that for which we lack rewarding feedback (relative to peers) and which relatively strains us (relative to peers). The fact that anyone _can_ activate system II means simply that everyone has and utilises a general form of intelligence. However, the fact that people _don't_ do this suggests that they have a relative deficit in system II (or rather in its utilisation) which explains this avoidant tendency, while simultaneously pointing to the degrees of difference in the general intelligence of people.
This dude actually threw out the line "I studied Computer Science" to Francois Chollet. We get it, you're an LLM fanboy, but you're speaking to someone that actually knows what they're talking about. Let the man speak!
@@jimbojimbo6873 Please define "come up with something someone hasn't already" and tell me, have you actually ever done this - and what did you come up with?
@@everettvitols5690 i’ve made a dish I’ve never seen in a recipe book or online before. If we entertain hyperbole then something as grand as self driving cars that meets regulatory requirements is something we don’t haven’t conceived, if it can do something that’s truly innovative then I’d consider it AGI. i don’t see how LLMs will ever achieve that.
Always worth pointing out the LLMs require a server farm that has the energy requirements of a small state, whereas the human brain runs pretty effectively on a bowl of cheerios. I think more people should think about this!
While this is true, I think it misses the point of the eventual advantage of deep learning systems. Human brains are fixed in size right now, mostly due to the evolutionary pressure of the size of the birth canal. Even if deep learning is multiple orders of magnitude less data and compute efficient than human brains (excluding the horrible compute efficiency of evolution to get us where we are), we can still scale the models to run on ever more power hungry data centers to surpass human brains. At the same time we can do this, our algorithmic and data sample efficiency gets better too, improving the ceiling that we can achieve.
@@CyberKyleall of that advancement will lead to great things but at its core foundation a LLM cannot achieve AGI. Also keep in mind these models are not even scratching the surface of the brains capabilities to apply intuition, rational, morality, and many other things that contribute to decision making beyond just simply data processing.
@@Martinit0 that’s not true, the operations needed for inference can be sharded across many nodes just like training. It’s just that training requires a ton of forward passes to see what the model outputs before you backpropagate the errors, so it requires large clusters to complete training in a reasonable timeframe. It is conceivable that you could make a ginormous model with many trillions of parameters that you’d shard across many GPUs.
@@CyberKyle. Although the capabilities will possibly reach AGI even without radical efficiency improvements, AI will always be greatly limited in it's impact until the energy efficiency problem is solved. Most likely there needs to be a fundamental change to what type of hardware architecture is created for AI to run on that can be sparse computationally and reduce memory transfer costs by possibly combining memory and processing into a unified architecture "processing-in-memory" (PIM) like neuromorphic computing.
@@ZachMeador We have to see who the uses are, we see that the majority were from students who wanted to falsify their work, but really the numbers of users are not significantly high, we must also take into account that it is possible that they even inflate the user base thanks to the mandatory implementation of LLM in internet browsers and operating systems
@@eprd313 We can know that DNN's are probabilistic adjusters, they can help us find the most likely possible answers, but they are not intelligent, nor is a search engine, in fact their implementation in search engines has been catastrophic, especially for Google where they prioritize the answers from Reddit, whether or not these are true does not matter
I don't get all these comments saying that there was a lot of repeated questions. The way I see it, the subject was interesting and "tricky" enough to talk about it in depth like you guys did here, yes it would seem that you repeat the same question everytime, but the answers and explanations from Chollet were super interesting and every time we had a new way to look at it, nice interview.
Voldemort asks all the right questions, but rarely more than two in a row, before he rests his throwing arm while firing off a bunch of nerf balls. His best probes are episodic rather than sustained. This stuff is simply too hard to command on an episodic basis.
interesting thought experiment: if you had all the data of everything that happened in medieval economic systems, before the industrial revolution: every conversation spoken, and lots of multimodal data, trained the biggest LLM ever on all that, and then jumped forward to today: how well would it do?
I am not so sure that humans do much more then using pattern recognitions to solve the ARC-problems. When I look at them I very quickly recognitions patterns I saw somewhere else. Our brain has been trained on millions of image patterns by evolution.
This is really insightful from francois - great articulation of memorization/benchmarks gaming, true generalization, intelligence and a good working definition of intelligence from Piaget - I recall the other frenchman yann quote him once too. I'd love to see the day when google/Francois ' team creates a new System 2 based Combinatorial search (discrete search) based engine that can do program synthesis on the fly with LLM embedded in them in the future!
How is the brain not interpolating or probabilistic? The only addition that the brain has is qualia and input from the cns, how significant they are for generalized agi is unclear yet. For reference: Oxfords Shamil Chandaria's lectures on the Bayesian brain.
@@mythiq_ It goes beyond interpolation and probabilistic solutions. Our brains are fundamentally able to abstract concepts with very few data points, proving that we are very sample efficient even when exposed to an entirely new set of data. LLMs are just really fancy estimators, capable of understanding the semantics of a given problem and generating an outcome based on similar problems its faced before. The semantic understanding of a problem enables interpolation. It does not abstract the given problem and then deal with it with it's own sense of understanding.
It's such a fascinating discussion! It definitely deserves more views! I can disagree with Francois Chollet about LLM potential in general, but I must admit that his approach is extremely refreshing and novel. We need more people with nontrivial ideas to build True AGI because just scaling LLM is a risky approach: if we fail the new great AI winter is waiting for us.
More interviews about the importance of system 2 thinking would be awesome, for instance John Carmack (of Doom and MetaAI fame) is also working on this... Your channel is becoming so popular it could easily lead to a technical breakthrough at this point
@@falklumo if that's true, why did Dwarkesh' ML friends who are working on AGI not know about the ARC benchmark and why were they surprised that the frontier models failed?
@@jeroenbauwens1986 The German guy (interviewed in immediately previous podcast) is young (inexperienced) and not a ML researcher. He stated that he was extrapolating straight lines derived from existing data. DP also recently interviewed a pair of guys about LLMs who had only been in ML for a year ie inexperienced.
@@bazstraight8797 so your point being that people like John Schulman and Ilya Sutskever are more knowledgeable.. I wouldn't be too sure they know about ARC though, Ilya has said in the past that scaling's essentially all you need. It sounds like this might be a blind spot in all of these companies. I guess Dwarkesh missed the opportunity to ask them
There’s a reason why our biological neural nets are so efficient. The fact that you need to scale an LLM model so much for it to get a grasp of the world model is telling of its inadequacies - it doesn’t diminish the value of scale or the methods themselves, it just signals their flaws. Intelligence may be a mix of a set of skills that include the memorization LLMs can run, and abstract reasoning from little context. System 1 and System 2 together, not one without the other. I’d expect a model that can actually reason to require much less compute, just like our brain does.
Our biological neural nets are not more efficient in terms of complexity. Just the opposite is true. Our largest AI systems are terribly simple compared to the complexity of our brains, way, way less than 1% the complexity.
I would totally invest in Chollet's research. He has a tons of insight and clarity. I have ideas, but I don't have the background to do the work myself - my background is philosophy. I'd love to participate in this challenge, but it would take me years to embed myself in the academic institutions.
The most of time programmer spent on is figuring out what current code base is doing, which is a learning process by itself. LLM can't do it because 1. Entire code base is too large to fit into context window. 2. Code base is changing too fast to be included in training data. 3. Even if you add a snapshot of code base into training data, subsequent changes are still too large to fit into context window. Essentially it's missing the long-term learning ability that encoding knowledge into model weight.
@@falklumo this is 100% philosophy. Science cannot answer what is intelligence. Only when philosophy gives a definition of intelligence, can science go out and measure whether and how much beings are intelligent.
No it's not, it's about the difference between acquire new skills and reasoning on new situations on the fly. Only the interviewer doesn't seem to grasp the difference!
@@AlessandroAI85 yes it is. They are both trying to define intelligence and find the boundaries of what is and isn't intelligence as compared to just memorization. What else to call that but philosophy? Philosophy is not a special thing only done in universities.
@@AlessandroAI85 they both had hard time to understand each other. The main issue is emergent properties. You can't predict if a large enough model have some "true" intelligence or llms are saturaing.
he talks about the importance of synthesising programs on the fly, using pieces of other programs and reassembling them for the task at hand. isn't that process just another form of a program, which could be incorporated into an LLM just like the various existing programs he claims don't represent intelligence?
@@perer005 i'm not saying they're the same thing. just that they're both technically programs potentially able to be embedded into an AI. it's not infeasible that there are patterns within the logic used to piece together a new program which the AI could learn from and improve with.
i think hes completely right. LLMs seem to lack something. they are just doing next token prediction extremely well. scaling that up will give us very good results, but we will never achieve true AGI in that way. something like a multimodal AI on the other hand could quite possibly result in AGI, or at least bring us way closer to it. think about how our brains, or more specifically our cortex work. we have multimodal brain regions where we process multiple senses at once and we have other unimodal brain regions where only a single sense gets processed. the perception of the world around us results from the connection of the multimodal and unimodal brain regions. i believe that if we do something similar to that we can definitely create AGI.
Even Multimodal Transformers not goonna result to true AGI because it will still lack the "Scanning / Searching" system that is necessary for any kind of intelligence and novel Inference. Although it will loook like an AGI or it will give an illusion
Dwarkesh is totally right pressing on the critical claim that humans do more than template matching. Chollet couldn't give a single example. The question was very easy to understand.
@@power9k470 there is a structure to this world that we seem to be able to partially observe. We create models of this structure. Whatever model we create - the model structure can be understood as a template. And many theories share similarities. One could think of the application of modus ponens/ deduction as one such template.
@@power9k470 and it's also not about who is right. I am not sure. The problem is just that Chollet claimed there are many examples but couldn't name a single (besides his test).
@@vincentcremer4235 Chollet probably could not because he is a CS and ML guy. These subjects do not have profound paradigm shifts. This question is suitable for physicists and mathematicians.
Think about art. Generative AI cannot create art in new, never seen styles. If you trained a generative art AI on all the paintings up until 1850, do you think it could develop new styles as original as impressionism or surrealism or comic art or graffiti? If you trained a generative music AI only on music recorded before 1960, could it come up with new styles as original as punk or hip-hop or EDM? Art is constantly evolving. Sure, individual artists do much that is derivative. But the fact that art evolves demonstrates a human capability that cannot possibly be a form of template-matching.
EXCELLENT EPISODE! These types of counter arguments against the LLM hype are SUPER IMPORTANT in this public debate, and both Francois and Dwarkesh made great points for both sides of the debate! The pushback from Dwarkesh was excellent, but we need that type of pushback against the proponents of scale = AGI as well.
Dwarkesh, don't listen to the comments, you did extremely well in this interview, much better than Lex when he challenges his guests. Well done and continue this line!
I think it’s very hard to avoid human biases here. Could any animal besides humans solve questions from the ARC benchmark? Yes we see it as “easy” and can say that it requires only “core knowledge” but that leaves aside the point that we (as humans) constructed this benchmark. I think “core knowledge” itself is a loaded term, which is far too entangled with human intelligence and the specifics of our evolution. Why should we assume that LLMs (or any advanced form of AI) will have the same sort of intelligence?
I agree with Francois that skill aquisition is intelligence, where I disagree with him is where he claims things like number counting is innate core knowledge rather than a trained construct. He apparently doesn't know about the Piraha tribe in the Amazon who do not have any numbers in their language. Neither do they have a conceot of time.
Learning mathematics is such a bad example because we don't even know how we learn it... We know it has something to do with mental representations and Intuition, but our traditional learning method relies more on memorization rather than intuition development
23:04 We synthesize new ideas from learned ideas, it is pattern recognition. We tokenize, compare, recognize patterns, mutate aka synthesize, discover new patterns which work, others which do not. From this we learn more patterns, all at different layers of reality, different contexts, etc. We have machinery like emotions to jump start where to look for patterns, essentially a macro tokenizer. Etc. My point. I believe you severely overestimate human intelligence, and underestimate where AI will soon go.
We operate in 10 countries- and some of the countries have very small language sets on the internet. And we get 18% accuracy for parsing things like street addresses, with one-shot prompting to GPT4. Our internal hand crafted "old school ML" get 50 something percent. But a lora on a fine tuned open source can get 60 plus %.
Dwarkwsh goes into this full on AI bro mode with a clear preconceived bias to prove his point -- though I'm sure he'd say he played devils advocate. Yes, he recently met Leopold so he may be influenced by it, but he did a very poor job here -- and i say it because I'm disappointed because he s done such a consistent good job so far. Had I been Francois i would not have accepted the dismissal so lightly, so kudos to him.
Haven't watched the entire interview yet but I mostly agree with Francois. I think the Tesla's "Supervised "Self driving is a great example. It's like 50 Transformers stacked on top of each other and with probably 1T+ hours of training data, it will still usually fail if it encounters a slightly novel situation. Most humans are able to adapt to those types of situations the second they see them.
You don't need to be able to handle 100% of road situations to be safer and more reliable than the majority of human drivers though, which is their goal.
@@KibberShuriqFrom a product perspective that's true but from a learning perspective, it shows that an LLM has very weak/nonexistent "Reasoning" skills. Tesla Autopilot is at best level 2 autonomy, while a human, after 1-3 hours of practice is already at level 5 (Full) autonomy. If I had practically every conceivable driving situation memorized, it wouldn't be too difficult to achieve 99.999% reliability.
@@Ibbysz A human is only at level 5 autonomy after 16+ years of practicing interacting with the physical world, typically including countless hours of watching other people drive cars. A toddler would NOT be able to drive a car after 1-3 hours of practice. Also, I don't think Tesla is using LLMs in their system, they are using ViTs (similar architecture, but not the same thing).
@@KibberShuriq Lol yeah mb I meant *Transformers. Yeah but that's the difference. A human brain needs at most maybe a few thousand hours of loosely related pretraining then it can use reasoning and related ideas ( physical interactions, movement, etc ) to come up with solutions to novel driving situations. A massive transformer + CNN + RNN cluster cannot even with billions of hours of footage + 9 Cameras. This shows a fundamental issue with our modern AI architecture.
@@Ibbysz It's called "sample efficiency" and it's likely only going to remain "fundamental" until we find new approaches to improve it. And there are a lot of very smart people working on it and they're making progress all the time.
I'm a bit of a scale maximalist, but I agree with some of the stuff fchollet says... I'm still a scale maximalist because the general intelligence here (by Francois') is not the LLM, it's the learning algorithm (Backprop, Grad Descent+,Finetuning+RLHF) the algorithm(s) is decoupled from it's product (LLMs, ViTs, ConvNets etc). I disagree because I think the ability to generate these programs (learning circuits) and use them (inference) are both functions of intelligence. I like the discrete computation graph approach though!
Francois is arguing what, that Humans and LLMs think differently. Yep, we know that. But what he doesn’t want to acknowledge is that LLMs (ie ChatGPT 4O) have some degree of reason and inference that seems to grow organically through brute force modeling of language alone.
But ChatGPT 4o reasoning is weak. When you detect a fault in an answer, ask it to check that fact - it does and comes to the conclusion it was wrong Even lists the reason for why it was wrong! Now ask it to reprint - the correction is gone... (still being in the same session)
Is it correct to say that LLMs encode human knowledge, but not human intelligence? That is, they have memorized the relationships between an enormous amount of things, but memorization alone is not enough to get you to human intelligence?
I think it's not hard to understand his point: Ask GPT to create a novel architecture for AGI, it can't because it can't comeup with an actual novel thing, but a mix of existing ones that looks "novel" but it really is not.
That's not the issue. The issue is that GPT can't think about the problem. Even humans come up with new ideas by recombining existing bits and pieces. Look at Jules Verne's "rocket" to the moon which was a big big cannon ball ...
@@falklumo okay but it's not the same to combine things to make a new thing than actually proposing a completely novel idea right? So what I understood from his claim was that actually models currently use what they already know and combine that, but they can't take a broader look and redefine, let's say quantum mechanics with a truly novel approach, isn't like that? Maybe I got that wrong IDK 🤔
I might have missed something then, I just thought that's what this test was all about, perform over completely unseen problems that require to actually abstract more from the details and into the bigger picture to try and find the underlying logic of it. I am not an expert on any of these things, I just thought that was it.
I dont understand why hyperscale people are so stubborn…. It is obvious this architecture does not possess the ability to reason… Lets scale it up and use it but lets also spend equal resources on finding new architectures…
Excellent walk-through of that sobering paper! I want to contribute, however I can to making AI and AGI work for the betterment of humanity. Also I hope to continue enjoying and contributing to life on Earth as I’ve come to know it, and embracing and helping others with the transitions ahead.
I'm having a hard time with the first comparison Chollet makes between insects & humans. He states that insects have hard-coded behavioural programs that map stimuli to appropriate response, and they can navigate their environments in a way that is very evolutionarily fit - without needing to learn anything. And then he goes on to say that if the human environment was static enough, predictable enough - evolution would have found the perfect behavioural program, and hard-coded it into our genes. The issue I'm having is that insects & humans both exist in the exact same environment. Why is it that the environment is static enough for this to work with insects, but not static enough that it works the same way in humans?
@@Tokterin other words, the environment is static enough for insects to survive this way, given their reproductive mechanism, but not static enough for humans to survive in the same way.
We definitely don’t exist in the same environment as insects. Consider the enormous biodiversity of animals (all specialize around different traits and strategies) that arise because of differences in environment, and note that insects share the most diversity. This is likely because their strategies are least adaptable, so any variation needs to be implemented as a whole new insect with its strategy hard coded.
You are constructing "environment" incorrectly. In ecological terms, your environment is your specific ecological niche. William H. Calvin wrote a bunch of supremely nerdish books long ago on a skein of related subjects, uncontaminated by modern relativist revisionism. If perhaps now somewhat dusty, they remain an excellent mental gymnasium. The human visual cortex has a tremendously high burn rate, and our enormous neocortex doesn't help matters. Our ecological niche is to survive our elevated burn rate. Our opportunistic strategies to gain enough calories must pay for the very expensive cognitive resources invested in cataloguing, recognizing, and coordinating those opportunities. Lions invest metabolic resources in growing giant teeth. Humans invest metabolic resources in growing giant brains, which are then employed to knap obsidian with great finesse. Two great ways to make lunch leak blood. In the second case, woe betide you if it passes from mind where you last saw a viable obsidian deposit. Once you have obsidian knives, or metal edgecraft, they turn out to have a second set of good uses: reshaping the local environment in novel ways to take lessen the burden of maintaining our incredibly costly minds. As a trite example, it's truly astounding how much more a person can reliably recall with a good library nearby. A journey of a thousand miles begins with a single step. A library of a thousand books begins with a single underground brush stroke. Very different niches when you parse things properly.
The link between memory and generalization and one shot learning is abstract thought. Both the interviewer and the guest sort of overlooked this fundamental topic. The problem we have is that LLMs don't exhibit much if any ability in abstract reasoning. That's why it feels like I am talking to an hi-tech parrot when chatting with AI. It utterly sucks at creating novel ideas. Abstract thought is the machine language of thought and it has a grammer of it's own.
He doesn’t say that passing the test equals AGI, just that LLMs by themselves will never reach it. P.S. I know that it is Pride month, but nobody cares what you do with your a$$
Correct! their also about wholesale Theft of novel ideas & works from 'actual intelligent' people & copywritten materials - without any credit, citation or due compensation ...& then passing those thefts off to the dimly educated & unintelligent as novel proofs of intelligence!
He’s probably one of the salty ones who screwed up their career and now desperately hopes AI will ruin everybody else’s. 99% of the AGI sheep are like this.
Regarding the path to AGI, I think there is a similar theme in the work of Ben Goertzel. If I understand correctly, he also suggests combining symbolic AI and neural AI to leverage the strengths of both approaches.
I wonder what Chollet will say when LLMs 'solve' ARC. "My benchmark was not properly built up. Sorry guys. I have this new benchmark, this is the good one"
taking 50th human percentile as a threshhold is faulty - schools damage our ability to do abstraction and forming new problem solving algorithms on the fly, it's punished in favor of ... well, you guessed it, memorization - so most people's abilities are heavily geared toward memorized skills and underperform relative to what the real (but as of now, unknowable) baseline would be otherwise, if memorization was not forced so hard as to overload natural problem solving ability the broken structure of school leads to this entire field failing to understand what knowledge is vs intelligence, and that learning is not a thing you do only during a set period and then you're done for life
After attempting to brute force maximalism for 10 straight questions, would have been nice to switch to having the guest teach us more about the classic who/what/when/why’s of the Arc test, so we as the audience walk away with a new frame and lens to use in this.
Knowledge is like memes. They persists because they get copied because they are useful or hold value. Memorized opaque solutions don’t have that quality. Discrete programs are meme-like, they can be copied criticized and improved.
The metaphysics of LLMs, Intelligence, Problem Solving is seeing massive development because of ChatGPT. So for sure there are many opinions, definitions, and interpretation of the technical terms. I have some notion on what Fracois Chollet is trying to mentioned. Basically generality comes in when an intelligence can layout all the problem solving templates it memorised, meet with a novel problem, select and sequence the templates in a way that it can solve a novel problem. To test generality, it should not be a single template to be recalled but a set of templates instead. I think that is what Francois Chollet is trying to explain here. It's similar per Marvin Minsky's book . I will be interested to know more abt the design of the ARC test. Coming up with such tests is not easy but it is a good exercise to have a deeper look on human's problem solving competencies. :)
I am at 26 mins, and already, I just have to answer to Dwarkesh's question of whether larger models could have enough program templates within it's datasets for it to master any problem and to very quickly master it: yes, but if such a model cannot do something we expect it to do, then we are going to look for what program templates are missing and this cycle is going to go on infinitely; and over time this will allow us to detect patterns in what continues to be missing which will settle this question one way or the other. Whatever the pattern is will be judged to be necessary or unnecessary to qualify as having intelligence. But this is what is so exciting is whether general reasoning can happen as a result of more and more program templates. How would a large model with training on sufficient program templates to appear to us to have general reasoning be different from one that in addition to this, also had sufficient program templates to start making new program templates custom made to solve a prompted problem? I think the difference would be time that it takes to train and whether humans can train the former model or humans run out of time before they train the model to train itself. If a large model is achieved that can appear to us to have general reasoning adequate without said model having the training to train itself, then when we add training on program templates for building its own program templates to suit a never seen before prompt, the model will have something new that may require a new name like ASI. This is semantics. Dwarkesh's definition of AGI is that it will have an adequate data sets with enough programs to appear to reason because of its ability to choose the set of programs to suit a problem. Francois's definition is that AGI will not only have the ability to choose from its memory the set of programs to suit a prompted problem, but it will also have the ability to write new ones to suit a prompted problem. Dwarkesh then asks whether there could be a spectrum between having the ability to write new programs to solve a prompt and not having the ability. I suppose in a way, you could think about the ability to choose what programs from memory to solve prompted problem as being a degree of of creating a new program. So Dwarkesh raises a great insight that there is no distinct line between the two abilities and they have to be considered to be on a continuum with infinite gradations in between such that all examples of choosing programs from memory are degrees of ability to write more or less new programs to solve a prompt. Using Dwarkesh's definition it then follows that intelligence is rated by the degree of jump or in other words any example where the written programs outweighs the programs already stored in memory by training. This jump can be all the self-written programs over years, or the ability to accumulate programs that do not overlap those in memory over shorter durations of time. But the real answer to whose definition is more correct, Dwarkesh's or Francois's, depends on what happens in practice: at what point will the programs the large models are trained on focus on the writing of new programs not in memory to solve a prompt? The answer to whose definition is correct is a judgement call about how much novelty exists in an answer and in the answers in stored in memory, and a judgement call about whether the model is performing in general reasoning at a high enough level. The reason we probably lean towards agreeing with Francois's definition is because current models are being judged to have neither novel answers, nor to be performing at a high enough level of general reasoning. But as models in the near future are performing better in these two areas we could find ourselves surprised that we begin to use Dwarkesh's definition. To answer the question of whether you can "brute force" intelligence? No, you can brute force the test and if you do, then a better test needs to be built. The definition of intelligence requires an analysis of the so called, enormously large dataset, to measure the degree of novelty and I argue that you will have to admit there exists a high degree of novelty in memory that is the result of experience over years of training which includes all of the choices that the model made; hence, novelty. In Francois's definition of AGI he believes, and I think he is correct to do so, that AGI must be measured to have a degree of jump in novelty where new programs can be made that are not present in memory that solve a problem presented in a single prompt. All of this is to say that there is a question of the degree of jump in a single prompt to qualify as AGI, and the answer lies in a general consensus about what measure would match a smart doctorate level human. The very definition of "brute force" is that it uses minimal degree of novel jump to solve a single prompt; meaning the measure of AGI, by definition, rules out the possibility in question.
Designing intelligence comes down to developing an adaptive interface between truth and provability of the system in consideration. Edit : No one exactly knows how to do that.. yet!
Great guest to have on this podcast. I entirely agree with him. Dwarkesh: I think you need to slow down and really listen to what he is saying. He has a different perspective to you, so try to lean in first snd then figure out where it doesn't make sense. LLMs have so many parameters and so much training data that it's hard to see that they get stuck until you play around with the API and see that if you build a program iteratively with it, it will easily get stuck and lack the most basic ability to problem its way out. I think they are amazing and mathematically elegant but the attempt to brute force AGI is not the final step. We will need another breakthrough. That said, it will change the world, but not as far as we would expect if you believe the hype. In any case, try to agree with him first before you find counterpoints.
Dwarkesh, thank you for bringing such high profile guests to podcast, but please make the podcast a platform for them to express their views. I kept fast forwarding your part. It was a pain to see you keep going back to prove your point correct.
