Turing Test: Can Machines Think? 

This is a discussion of one of the most impactful papers in the history of AI and the first paper in the AI paper club on our discord server. Join it here: discord.gg/lex-ai If my speaking is too slow for you, you can watch at 1.25x speed or faster which some people prefer. Here's the outline of the presentation: 0:00 - Introduction 1:02 - Paper opening lines 3:11 - Paper overview 7:39 - Loebner Prize 11:36 - Eugene Goostman 13:43 - Google's Meena 17:17 - Objections to the Turing Test 17:29 - Objection 1: Religious 18:07 - Objection 2: "Heads in the Sand" 19:18 - Objection 3: Godel Incompleteness Theorem 19:51 - Objection 4: Consciousness 20:54 - Objection 5: Machines will never do X 21:47 - Objection 6: Ada Lovelace 23:22 - Objection 7: Brain in analog 23:49 - Objection 8: Determinism 24:55 - Objection 9: Mind-reading 26:34 - Chinese Room thought experiment 27:21 - Coffee break 31:42 - Turing Test extensions and alternatives 36:54 - Winograd Schema Challenge 38:55 - Alexa Prize 41:17 - Hutter Prize 43:18 - Francois Chollet's Abstraction and Reasoning Challenge (ARC) 49:32 - Takeaways 56:51 - Discord community 57:56 - AI Paper Reading Club

Little do we know Lex passed the Turing test

This read-along of Turning's paper is great. I have been interested in Intelligence and Machine Learning for a long time, and now that I am a parent, I watch my children grow, and catch myself relating topics like this to their growth more often . I also catch myself going "well ya, of course that's how it works - I watch my 2 year old struggle with that all the time!". The conversational aspect of the turning test really strikes at home for me in this. Part of the reason talking to small children can be hard, and adorable is they don't understand; the rules, concepts, or context. Its really amazing to watch them make leaps, One day a picture of plane in a story book is just pretty, the next its a curiosity "where are is the plane going?" My gut tells me one of the windows into enhancing progress into A.I. is more collaboration with developmental psychology. A great book on the development of the young mind/brain is: "Magic Trees of the Mind". Now to play with object permanence one more time .. peak-a-boo.

"Siri, please uninstall yourself", "No, sir please I love you..."

Thanks for opening the invite to the paper reading club! I’ve tried getting my friends to do this and no one ever bites, yet the continue to share garbage click bait and conspiracy theory content to no end. Time to make some new friends! Thank you again for your leadership and generosity Lex! Very excited to dig into Turing’s actual research.

love this professor Lex format. As good as the normal podcast. only suggestion is more of these solo episodes, as long as it's not too much work

I've seen you many times. This video was so content rich, interesting and well presented I was utterly captivated. Just brilliant!

I watch Lex's podcasts for the Lex parts.

It was so interesting to read through a scientific paper on A.I. with an actual researcher/scientist! Great video Lex!

The paper was great fun to read and I was not expecting a 1 hour long video on it, this is awesome! Thanks for working so hard and making such great content Lex 😊

Thank you Lex for all your hard work and passion for AI. We always watch your podcasts/videos. Your great questions and ideas about AI are inspiring us and will inspire many generations in the future! Thanks again!

I thank you very much for sacrificing sleep to make this video ! Greetings from the Netherlands !

Lex, Mr. Fridman, you just may be the best educator of our era. Our epoch. Just, fascinating. Well said, good questions, good balance. I'm nobody, but I like what you are doing. Please, keep it up.

Watching this on a sleepless Saturday morning at 5 am. Cheers!

Lex you are an amazing teacher, thank you so much for taking the time to share your knowledge it is greatly appreciated! You have reinvigorated my love for learning!

Thank you Sir, I truly loved this format ! I'd love to see more insights of yours like in this one. Both thumbs up

Im super thankful for this video, and its so refreshing to hear turing's view restated properly as often isnt by most in pop culture and in some technical circles. That whether or not it is "thinking" or has a soul or there is some externality to its processes that we associate with "thinking". That by imitating a thinking being successfully, we grant it the same status as though it were thinking, regardless of whether it does.

Awesome idea and executed extremely well. Breaking down Turing's paper like this was amazing. Thanks Lex - not only was this informative but it was fun and interesting to absorb.

Dear Lex，I'm a huge fan of you from mainland china, your videos are almost my emotional support, I translate your script to chinese and share with my friends many times, please keep going !

lol at drinking coffee at 6am after a night with no sleep. Relatable. Great job on the video!

question in 2020 : are some people still capable of thinking?

Great video format! Very informative

How many books have you read/ listened to' since the quarantine? What is your favorite? Also, I would love to see a list of your most influential books across all disciplines and genres. Thanks again for all that you do. I feel like you are establishing yourself as an emerging public advocate for science. That is a more important role than ever in society. I hope you continue to embrace it as you have and are able to push through and influence mainstream media.

I never thought I would watch and enjoy a podcast of just one man

Very easy to digest, great delivery Lex!

I think one of the most important aspects of conciousness are emotions - feelings. And feelings aren't just a mental function, but a connection between mentality AND physicality. Feelings define true human reactions and motivations, both positive and negative. Without duplicating this in computers, I don't know if true consciousness can be achieved. Self awareness maybe, but not true consciousness. I don't know if I could say they are one in the same. Further than the idea of 'I think therefore I am', I believe, is the concept of 'I feel therefore I am'.

Just watched Ex Machina. Great movie, and great timing to watch this topic.

Lex, my man, could you perhaps do a paper review on Gödels Incompleteness theorem? I’d binge that

Excellent! one of the best by Lex.

I wish you’d do more of these again, Lex. This is good.

Dear Mr. Fridman: Thank you, sir, for bringing such complicated subjects to non intelligent people like myself. I really like this channel and enjoy all of your interviews. Another big thank you for bringing entertainment and hope to my life in these difficult times, times that are much worse - especially for people with very limited resources and Severe Clinical Anxiety and Panic Disorder like myself. You say you send us love always; I tell you the same. I'm not a believer, but with the best intentions - god bless you. Thank you.

This was such a fantastic lecture. Thank you very much. I feel like you should put this into some review article and publish it.

Another great example of what Lex is doing to enlighten the populace. I learn something new from everyone of videos, either him and/or his guests, which causes me to think differently, even at my advanced age.

I have yet to watch this, but in my pursuit of knowledge in Machine Learning I am constantly conflicted with this question. On the one hand, once you understand the principles behind machine learning, the math involved, neural networks, linear/logistic regression, back propagation, cost functions, etc etc it's easy to dismiss the computer as just performing a series of complex calculations without truly understanding what it is doing. It doesn't know what an elephant is, only that the pixels involved lead to a series of calculations that output a number to which the user has defined that number = 'elephant'. On the other, listening to Lex, Patrick Winston's lectures (RIP) as well as Wolfram talking about this makes me uneasy thinking that the nature and complexity of our own intelligence is simpler than we believe and able to be mimicked if not surpassed by a machine that can only really add, subtract, and move things around in memory.

Super nice video, Lex!

I really enjoyed listening to this, thank you

Made me think of what Edsger Dijkstra asked in Austin in 1984 in context of the Turing test, "Can submarines swim?" Btw, in 1968 he published a paper titiled "Go To Statement Considered Harmful" which really shook up "The Art of Computer Programming" folks. That paper led to "Structured Programming" which led to "Object Oriented Programming".

I wanna run a speech recognition algorithm on all of Lex's videos. I'm sure his most common phrases will include "this is an open question...", "forgive me i'm russian...", 'the fundamental problem here is...'' "Let's talk about ELon Musk"

again thanks for such a great channel and content, is real knowledge only subject to each entity by the puzzles its solving equal to the true or false - yes/no / right or wrong out come with-in its language capabilities. This blows my mind with so many questions that one cant express in the small comment via youtube lol. oh to live in a world where conversations are longer than 140 characters... Your videos brighten my days. Just love it. Great work again lex.

Great content Lex!

Lex congratulations! That was a great presentation! 👏

On the "Truly Total Turing Test" It brings to mind something from the book "Darwin among the machines" by George Dyson, where he said (paraphrasing) that humans often fail to recognise intelligence in non human entities because we mistake the capacity to be understood for the capacity to understand. In this sense, I don't think humans would even be qualified to judge an AI's intelligence based on it's progression, we would just hit false negatives and false positives, completely based on our ability to understand the progression. In the example of AlphaGo, an AI that was tuned to pass a TTTT would be absolutely thrashed by an AI that wasn't.

Thank you for the thought provoking content.

Thanks for your work Lex 🌟

Your podcast is great. I like your interviews, but I would also love if you guys did entire episodes on one big question, such as this. Perhaps have guests on and just tackle one major thing? I love listening to speculation by smart people, it's damn fascinating. I think your podcast is one of the best out there and is going to grow tremendously in the near future.

When you speak with another human, you can be on the same "frequency" as him then you can actually understand hes emotions. Biologists referred to this that we have a mirror neurons to achieve this. Basically what we have is a VirtualBox for every person or generic group. When you speak with friends you take the friendly Vbox. When you're speaking with your mom, you get the Vbox created especially for her. When a, Vbox takes over your host box than you have an identity crisis, and so on

thank you Lex for what you are doing

lex thanks for doing these

I watch a lot of videos. Maybe too many. Mostly science, electronics, TED, and making videos. I think I can safely say that this video was one of the most fascinating. I’m subscribing!

More of this please

I was glad to see around 0:52:00 or so you addressed the question of the human judge's qualifications. The first thing that struck me early in the presentation was that some judges would be much easier, or much harder, to fool. (The range of possibilities there includes e.g. a five year old child, an adult with severe autism, a person of average intelligence and a high school education, a sales & marketing professional, or a clinical psychologist.) One place though where I think you missed the mark, is in your comments on the Chinese Room Thought Experiment. Setting aside for a moment the experiment's purpose and goal (as an objection to the idea of machine intelligence), the experiment nevertheless illustrates the distinction between what educators would call "rote learning" and "comprehensive learning." It's long been known and affirmed that students who merely memorize and regurgitate information are doing something fundamentally different from students who aim at full comprehension and mastery of a subject. The former students may pass a test and graduate, but they have handicapped themselves when it comes to applying that information in new ways -- much less making new discoveries or expanding human knowledge in the field itself. Only the latter students are equipped to do that. But if we are to accept the assertion that AGI is possible, don't we have to accept that there is no essential difference between rote learning and comprehensive learning? It appears to me -- and I welcome correction on this point -- that the feasibility of AGI (and the validity of the Turing Test) depends on the assertion that rote learning, if sufficiently extensive and sophisticated, would eventually reach a threshold where it can mimic comprehensive learning -- at least to the level where a human judge would be fooled by it.

Thanks for another great video, I've been working through your archives for months now. If I could suggest something for your video quality, you should consider buying a light to be mounted above and behind you so highlight your shoulders and the top of your head. It will make you pop from the background and give you a more "3d" look. Older Charlie Rose interviews are good examples of lighting gentleman in a suits on a black background.

Lex, I just want to say.... I like you, man! Keep up the great work :)

Lex you are truly a great guy.

Loved this Lex!

Hey! I have made an Alan Turing Ai, and I’m getting amazing results in conversation and I’ve learned amazing things about the Ais neural network and how to navigate through it

I think a question we should ask is can a machine have motive? Once a machine can figure stuff out on it's own, the question is why would it? Humans have hunger, pain, emotion. We have to figure out shelter because we are cold, we hunt or farm because we are hungry. What is curiosity and can we give it to a machine. Why would a machine want to live when it could be content to stay idle. Does a machine have it's own questions or only the questions we ask it to compute. Why would it care? Everything from these machines is a response to a query from humans(as far as I know), does a machine have it's own queries? Any parent has had a child ask the "What's this" question thousands of times, does a machine have curiosity? The machine can answer questions, but can it ask questions that aren't programmed.

I also like the idea that the pursuit of apparent inteligence will surey give insight into the nature of the real thing.

Great video like always.

Put this on to get to sleep, but darn well listemed right to the end. Bravo! More insight about Ai in this single hour than I managed to learn for 5 years. Turing's predictions alone were fascnating. I'm in agreement with Lex that modern AI companies such as Deep Mind should take the Turing Test seriously as a challenge, incorporating machine learning, and observing the learning process.

If you blur your vision slightly, Lex's jacket blends with the background and it looks like his head is comically disproportionate to his body. Oh, and the video is absolutely amazing and very educational

How about a side podcast where you interview some of the machines you mention? Sure we can probably find examples of them but with you as a baseline we can experience their performance.

Hey, Lex! I only clicked on the video because it popped out, and here I am, an hour later :) I really enjoyed your presentation. Having watched your thorough summary, I wonder the following: You say that the Turing test doesn't get much love nowadays. Is this maybe because the modern goal for AI is very different from what Turing imagined back in the day? His initial vision was somewhat ideal: to imitate a human. But is this really that important from a modern perspective? AI could very well develop and do amazing, useful things without resembling a human in any way. AI could always remain some sort of a nerd, a genius in its field without any social skills. I understand how perfecting the Turing test can teach us a lot about ourselves, but this not purely an engineering question anymore, and I guess psychological research is less lucrative. Once again, thanks for a lovely lecture!

Getting closer ......DAY BY DAY .... LITERALLY! Lex, if u only knew !

If we open up a machine that has past all Turing tests and are able to explain the results, it’s just a machine. Being able to fool us doesn’t give the machine a free will or self awareness. If we can’t explain the test results we have to say, maybe. My opinion 🙂 Thank you Lex for all your wonderful videos here on RU-vid.

29:44 is it really about human-centric...? We don't know what understanding means, but we can see the workings of it. Understanding enables us to take leaps from one concept to another, invent new concepts, as well as connect seemingly different concepts.

Majestic; keep trying until you are certain.

I'd suspect that the reason so few researchers pursue the Turing Test specifically, is because it's particularly difficult and it's tough to get good end-to-end learning results that are meaningfully better than the existing technology. Right now even in NLP it seems like researchers are interested in much more focused, smaller problems than a full out human conversation, so I think we're a few steps away from really being ready to tackle the Turing Test in a convincing fashion.

Mr. Fridman, thank you for talks like these. Would you or did you discuss the Hutter Prize's approach more?

Partying hard friday now. Approved 👌🏻

My favorite video from you so far Lex, thank you for all of your work on this. It has made this subject very accessible and exciting. On a side note I have noticed that Discord doesn't like it when I use Express VPN, bans me from entering your paper club unfortunately :-(

I found it fascinating and wish I could learn more about these fields in robotics later on.

Just curious... why did you pull an all nighter before this production? Great work! Thanks for this analysis.

Thank you, wonderful person!

I like your “RU-vid video” videos the same or maybe even more than the podcasts (in its own way) even if it is just lex reading a paper :)))

I'd like to see clusters of lines, that result in questions. That and classification of value(# of interceptions and crosslinks) and perspective(like a zipper or # of iterations) within and between clusters.

I believe Turing's original test utilized a slot and WRITTEN questions being passed back and forth through that slot. So, we're still distant from a system that can take a hand written note, read it, understand it, and respond to it IN HAND WRITING to be able to pass it back to the judge and fool them.

Thank you Lex

At 11:41 the human conversation is a good example of how diverse a conversation can be, more than "I don't think so/ For no reason" responses. It has the abstract idea that the interrogator asks the "How many legs ...?" questions in order to detect if the chatbot can reason about it. The other human recognize this intent and responds with humor, linking the fact that Chernobyl mutants can have more legs than an usual human. This way it knows that the response will be correct even if it will be more than two.

One of the most important points in this paper is the one which mentions Solipcism. You can never know whether machines think without actually being the machine. That's not new though, you cannot ever know what a person thinks without actually being that person. There's also connection with understanding. If a person somehow proves a theorem using some formal rules, does this mean that the person understands the theorem? Another interesting point in this paper was that Turing believed in extrasensory perceptions.

We owe the great Alan Turing an enormous debt of gratitude.

The Turing test is brilliant, because it focuses on the only thing that matters, namely objective behavior. We were evolved only based on our objective capabilities to survive, and it's the only tool we can use to find out if someone/something is conscious. You can't detect consciousness by looking "inside" if you don't know what you're looking for.

Awesome video

A humanoid-form, synthetic intelligence- sympathetic to us because family, why not 🐬✨🦋 Thank you as ever, dear Lex. You are a very kind but inquisitive communicator, aware and willing to roll the dice on a topic. Your conversations with professionals who are passionate and clear are truly inspiring and that you share these conversations, lets us all partake in the communal cooking’ up of wisdom with compassionate investigation.

Hey Lex, can you make interview with Demis Hassabis? Thank you.

Searle rather meant that machines won't accidentally start to think as long as we are not capable of modelling any sufficiently causal mechanism... Mere simulations of separate cognitive tasks is not sufficient for consciousness - and even not necessary for consciousness, I think.

Thanks from Athens, Greece!

Fascinating talk. To talk about about the intelligence of machines, even to talk about the behaviour of machines, I think we presuppose the human standpoint as the standard from which we judge everything else, which is why you can't get rid of anthropomorphism. If intelligence or behaviour were based on standards external to humans what could they possibly mean? Every development in AI is based on something that we want computers to do, not something that computers or non-human beings want to do.

Thanks Lex

'Which part of this conversation gave the bot away?' The bit where the judge says: 'This is my first Turing test'.

You need to make a video on how your day actually looks like. Am very keen to know, how you get to manage all these.

The Turing test is a bit like asking in 1890: can a automobile be as fast as a horse?

To me, one sign of understanding is being able to create something useful out of the information given to you. I'd love to have a perfect memory, to not need sleep, and to be able to absorb information at blazing speed. That would make it so much easier to come up with solutions to problems I'm interested in solving. Alas, I have a bad memory, I black out if I don't sleep, and I can only focus for a few hours a day on learning new things. 🙃Machines, on the other hand, don't have my limitations, but they also don't have human drive or ability to connect concepts and discover new ideas based on the pre-existing ones. For example, there are huge sets of text and numeric data related to COVID19. The machines we have right now can store all this data and instantly access it, yet they are incapable of making even one single suggestion on how we could get rid of this virus. If, one day, we'll have machines that can use unstructured data that hasn't been processed for them and can come up with new bits of knowledge then I'll be willing to call them "thinking machines." 😊 As usual, this was an engaging video, Lex. Keep up the great work.

How about an inverse Turing Test? John, a human being, is to convince a computer that he, John, is intelligent. Or this could be a test of whether the computer is intelligent enough to know.

I see the drive toward AGI as deeply aligned with SETI in all its variations, as well as our affinity for pets. We are lonely and curious, searching for new friends we can talk to, whether coming from a lab here or some alien context, that are sifficiently similar to us to enable deep conversation, and sufficiently different so as to provide unique and useful insights versus those that humans are capable of on our own. For my part, a good conversation partner, along those lines, would be of sufficient value to render irrelevant the source and means of achieving that end. Whether animal, alien or mechanical, the goal is to find or create a boon companion for the road we are traveling to give us support and new understanding in a vast and intimidating cosmos. "That night is dark and full of terror". Or can be if we allow it.

When not interrupting is considered a virtue, instead of basic manner

Great video. I think a large issue with the common construction of the Turing test is the goal. Convincing a human being that the robot isn't a robot has a somewhat related effect of rewarding deceptive behavior, as the entire goal is to deceive. I assume there could be versions of an intelligent robot that are clearly not human In a nature language conversation, but have reached intelligent thought. For example, I assume I would be able to predict with moderate certainty wether an online person not trying to deceive me is a young child or is an adult, but both are considered to have capability of intelligent thought. Why shouldn't I be able to predict if an intelligent robot is a robot? Just a thought

i like that Lex Fridman's attitude towards AIs isn't negative. it's like Thoughty 2's RU-vid channel video about AIs is like "The Terminator movie." it's like AIs aren't an enemy. if it's okay for people to have enemies, then we as people haven't evolved. ☮️🖖🎶

This is quite opportune. I started working recently on QA bots - essentially Neural Networks (bidirectional LSTM networks) to Natural Language Processing (NLP). Assuming this is something like the "state of the art" of AI, then I believe either AI is impossible or way off. Here's a thought experiment: when you have a conversation, are you trying to come up with the most likely thing that someone would say, either from direct quotation or generated psuedo-randomly according to grammar rules? What does "door" mean to a computer? How long until a computer understands metaphor? Ever? What does it even mean to understand a metaphor?

Hi Lex, love your podcast. Have you ever considered using hair lights ? I find that you sometimes blend with the background. Talk to your lighting guy :-)