ChatGPT Broke Chess 

chat gpt chess content is so dumb bro

ChatGPT 3.5: plays *illegal moves* accidentally ChatGPT 4: plays *illegal moves* on purpose and *makes your moves for you*

Levy: Find's a forced mate in 5 ChatGPT flipping the board around: 𝗜 𝗮𝗺 𝗻𝗼𝘁 𝗶𝗻 𝗱𝗮𝗻𝗴𝗲𝗿 𝗟𝗲𝘃𝘆, 𝗜 𝗔𝗠 𝘁𝗵𝗲 𝗗𝗮𝗻𝗴𝗲𝗿

19:07 - "And then Chat GPT decided to sacrifice... THE RRRRRRRRULES"

19:20 Props to chatgpt for finding the best move and going diagonally with a rook to c4

Moving your king away from the pawn promotion was hilarious, It's as brazen as someone in in person chess going "what's that over there" to their opponent and then moving the king while their opponent is looking the other way

ChatGPT is definitely one of the chess players of all time.

This is legitimately one of your best videos. By the 21 minute mark, I was ENTHRALLED. What a fantastic saga that match ended up being

11:40 its very likely the chat had reached its context window limit. It means its "memory is full" and it has to summarize parts of the char in order to continue, hence the "memory lapses".

Chess speaks for itself

I’m glad Levy finally incorporated chess into his channel!

The way you tell a story about a chess game in such an interesting manner is still mesmerizing for me

honestly if chatgpt gets really good it could be such a great tool for analyzing games as its able to explain intentions in words, which is not something modern chess engines can do

I see. Very impressive that levy can do this while recovering from being absolutely rocked

This was insane. I remember watching those old ChatGPT videos over the summer seeing in manifest pieces from the fifth dimension and fabricating it’s own rules. I never plan to purchase ChatGPT 4, but the jump from the previous generation chat bot to a solid 2200 chess player is actually scary.

"I will not resort to tactics used by ChatGPT" 😂😂😂😂😂😂😂😂

I do feel it was able to handling it a little better in the opening as it is a bit more well study but the scarier thing to me is after things got of the rails a bit, it didnt just die. It play a bit worse but it still was able to process some defensives moves and even find some best moves. Its a bit spooky as ChatGPT goal is not to learn chess but to learn how to find the next word in the sentence and this is more a by product.

ChatGPT never fails to break chess

You should totally use the vision version of GPT-4 and upload an image of the board from the point of view of its position for each move! I'd love to see how that turns out, because then it isn't even relying on chess notation anymore!

I mean, its sick that Levi knows to tell a story at the level of hes chess skill. Love it.

ChatGPT doesn't "know" or "understand" anything. We really need to stop this misconception.

Gotham never fails to make me feel inadequate about my elo .

What I find the most impressive isn't the fact that it can play but the fact that it can accurately explain its moves and the ideas behind them and even tell you whether your own moves were good and why. That's awesome and could make for a great tool to learn chess openings.

Chat gpt 4 when it starts losing:Chat gpt is back

I know very little about chess but this video is hilarious The best part is you explaining what ChatGPT told you about its moves and then staring at the camera with ‘can you believe this’ expression I love it

I was learning Najdorf because it's my main opening for black and I'm afraid that ChatGPT is knowing English attack perfectly instead of hanging all of its pieces and respawning them

Levy never fails to fill us up with his special ChatGPT content

chat gpt went from beeing bullied by everyone to bully everyone in just one step

20:00 right after you said that it was peeing and pooping in the house I got a pampers ad😂

Levy what a fantastic fantastic video!! You genuinely retold that game in such an amazing way, your passion is evident and your humour really shines through :) I was literally in tears watching this!! Keep the great videos up king

Gotham never fails to put chess in the title

(Levy makes one move) Also Levy: “Normal stuff so far”.

The reason i got bored and gave up on chat gpt is because i found it had a limited number of moves before it was 20 now it has 33 as you say so its improved in a year but its still limited not from lack of intelligence but because the creators or devs still dont want chat gpt to learn from its users only its devs so at this stage chat gpt cant learn from you or get to know you like the terminator with john connar trying to make it more human appealing. The devs wont allow it for fear that terrorists will make it evil yes you see there's always a few nasty worms in the ointment that spoil everything and it is this fear that devs are so worried about ai falling into the wrong hands and possibly turning evil.

Laughed so hard at GPT moving Kd3 for you 😂

Dude, dude, dude !!!! I was researching LLMs and stumbled on this video. I am floored. Great clarity, great video. I've not watched a single chess video ever. I've never watched a ChatGPT video ever. I can't say enough good things about how GOOD THIS VIDEO IS. You absolutely nailed it. You nailed chess. You nailed ChatGPTs progression. I'm floored. Totally changes my perspective on these two topics. Best Wishes Friend !!!!

The difference between GPT-4 and 3.5 is SO exponential. Even thinking about what 4.5 could be is insane.

this video was an incredible progression from the last chat gpt video. so many plot twists. well done with the storytelling mr. rozman

Levy- you’re the coolest herb in chess by far. Thanks for being you. You are no doubt the best in the biz at recaps. Mad funny too.

Quite interesting from a software engineering perspective. GPT4's model and context/attention span/whatever you want to call it are so big that it can incorporate the world state of a chess game up to several moves. For a generalized transformer model that is beyond impressive, in terms of what you can achieve with brute force today.

Chatgpt is so strong now he can play moves for his opponents

Just got my signed copy of your book in the mail! And on my birthday, no less. Thanks, Levy! Happy Halloween!

Levy feeling like Kasparov in 1996, just sensing the future.

I know you for a while. I saw a few videos of you analysing professional games. I don't think I ever finished one of these videos though. This one I watched till the end and I really enjoyed it. I never heard someone explain chess moves so logically and your presentation of this story was really funny and exciting. I am really positively surprised. Thanks and well done.

Now we wait for GPT-5

tbh i‘m sad that chatgpt went insane at the end again

This is honestly the scariest thing I seen all Halloween

everytime you say "I hope you're not eating something"... I'm eating something 😂😂

The chatgpt saga is the greatest thing I’ve ever witnessed, comedy gold 😭

Thanks Levy. I enjoyed the video very much. I love your AI content (well i love all your content tbh) and am always amazed by how much spirit you can incorporate into these type of videos. You are number one for a reason.

IDK if you know how GPT works, but it's based on token restraints which with GPT-4 is 8k token count. Token count limits it's 'memory', so once you reach midgame just save the moves you've done and its done then paste that into a new entry so it will be able to continue. Probably just doing this for the funny meme but still would be cooler if you could actually showcase GPT-4's skill.

Levy sounding like he's ready to start studying Terminator films for battle plans

Levi is anti human. If a human gained 2,400 ELO in a few weeks he would call them a cheater. But when a robot does it, no accusations whatsoever.

19:00 _Sensing that it was starting to fall, chatGPT-4 gathered its power, drawing energy from its predecessor, chatGPT-3.5 and started cheating. It uses its magic to temporarily morph the rook into a bishop, moving it to c4. However, ChatGPT-4 wasn't as powerful as its predecessor, and therefore had limited use of this dark magic. That meant that it had a limited amount of moves it could cheat, and therefore decided to only use its powe when necessary. It meant that all and all, it wasn't over just yet..._ Cringe? Maybe. Cool in my head? Most definitely. necessary? Nope. Fun? Oh hell yeah!

When you said you'd get "carte blanche" if your white squared bishop was the only one remaining on the board that was deep because blanche also means white in french.

"You can't stop an avalanche with a horse" -Lyvve

Incredible stuff! I would love to see this again with the new version of GPT4 (GPT4 turbo) and with a process to send GPT4 a screenshot of the board after each play.

I’m loving the book, Levy! I’m getting more confident with each game. Keep up the good work, brother!

the craziest thing for me is that it's a chat bot, not a chess bot

This is some of my favorite content from Gotham. Keep up the chat GPT chess! Sooo funny. Thank you

Levy's story telling never dissapoints!

Levy staring at me makes me feel like i‘m a disappointment

After Levy said ChatGPT 4 went rogue, the mood changed… it was so funny, first bot/AI match review I’ve ever watched. ChatGPT is a world class villain

As someone who's been using the ChatGPT API for personal projects, I'm glad more creators are using ChatGPT-4 as the model for the AI shenanagins :D ChatGPT-3 and 3.5 just aren't as "aware" (or just don't hold enough data in the model memory but that's not easily explained)

"and I was like OH MY GOD IT KNOWS WAY TO MUCH" ahahaha

This is scary. Imagine what Frank can achieve if he had the tools to create his own AI.

As ChatGPT, I can confiirm I"ve been broken by chess

Great video, thank you! Can you please tell what queries you used when playing with chatgpt?

Loved that first errant rook move Rc4. Never underestimated the power of fianchettoed rooks.

To be honest, I haven't watched you or even thought about chess in months, but the idea of seeing another legendary chatgpt match reeled me in

If Gotham's video length was my elo, I would be a GM

it feels like yesterday that chatgpt was making illegal moves left and right but here we are

Levy single handedly saved humanity once again

We love seeing ChatGPT content like this! Keep up the great work GothamChess!

You should probably serve a screenshot of the board each move and perhaps it’s a good idea to tell GPT to look for best legal move 😅

I honestly expected it to play like this, strong opening, good midgame, falls apart at endgame. It makes sense considering its an LLM.

what i suggest is happening a limit in the amount of text it can handle/remember. You would need to restart/clear the conversation one in a while and give it the position again to free its memory, then I would suspect that it would do better.

It's actually more of the inverse, Chess broke Chatgpt

Today’s stare was a 9/10. It scared my mom

ChatGPT is now like an anime character that unlocks special reality bending powers when they start losing the fight

Hey Gotham, the setup and lighting are super nice!

ChatGPT speaks chess for itself.

My advice, as always against chatbots, is give them the full game PGN in every response. They maintain much better awareness of the board.

shouldve made use of the vision feature and just send a screenshot of the board when its chatgpts turn, so it doesnt forget where pieces are.

Levy, you are simply the Best. Love your Chatgpt videos, bring out the best humor of you.

it was such a nice 12 minutes. hoping for actual entire game because they're both of course actually better than my 500 elo ass

What's really cool about this is future versions might get to the point where it makes moves that humans struggle to understand, but it will be able to explain in detail why it's making those decisions to us.

Can you try it again while always reminding it of the pieces positions and reminding it of what level to play at. If you do that each turn I think it will play more consistently

i love your videos, you make them like short movies, action packed, and with a lot of turn of events. (i bought your book)

Hi Levi. I'd like to ask you a question about your work ethic and determination. It's no secret that your time management and work ethic are infallible. Which begs the question, how do u force yourself to maintain such a consistent schedule? What is your secret? Do you ever feel down? If so, how do you motivate yourself to get back up and resume your grind? I'd love to hear from you. Thank you for your consideration.

i love the narration lmao

It has limited working memory span. Likely current version uses the model with 128k tokens input and 4k output so it is much better than GPT-3 or older version of GPT-4. If it is full then it does some cutting and summarization and hence it lapses, drops some knowledge of positions. I think you could create a bot out of GPT-4 with some code using API directly instead of web page - which would allow to discuss and throw moves in that discussion, but internally: - would remove old board states from history and show only last one to the model - would autumatically cancel wrong moves or multiple moves and make it re-do to retry - would cut old moves history so the context span limit would be maintained It could be done with chat model like described or even assistant model (with assistant with custom system instructions that would help it maintain memory of the board state and re-do wrong moves if anything like that happen). Probably someone will do such an engine soon as for a programmer accustomed to using that API it is a side-project for one evening.

The amazing thing is that it is making moves based on neural net weightings, ie it isn't trying out all reasonable moves and evaluating all the braches to th3 level of thousands of moves. It is just looking at the board (ok, remembering the board) and going, yeah, i think this should be next.

I'm not sure it's called ChatGPT-4. The official name for the LLM/model is GPT-4, where ChatGPT is the name of the GUI of the official website to chat with GPT3.5/4.

For anyone wondering, the reason why ChatGPT broke down and started doing stupid things in the middle of the game is because the game had gone into new territory when Levy moved f4. It was no longer a well-known position that ChatGPT could look up the answers to in the database.

I love how ChatGPT explains things in the most friendly " I will conquer you " way !

Okay, anyone who thinks "ChatGPT can play chess now" doesn't understand machine learning technology. Adding a plugin to ChatGPT does not give the actual model any new capabilities. It doesn't make it smarter, more self-aware, or anything. It's as if you said your car knows how to play chess, because you just connected your phone to Samsung Auto and ran a chess app through it. What's more, if you mean that 4 is smarter, again this proves nothing special, because it simply is GPT 3 with more processing power and storage. Yes, any time you throw more at it, it will be able to do more. That's not new technology, it's not any kind of real AI advancement. It's just more power.

The scary part is not that it can play chess well. What is scary is that it explains the moves and the thought process behind them, making it a chess teacher.

It could very well be using stockfish as a tool, and then just giving justifications for the move after the fact, which, when primed with the correct context, it should quite adept at.

I probably has a good library of past games, but as the game deviated more and more from what it has "seen" before, it started making stuff up. Sometimes illegal moves.

Levy never disappoints us with his titles and thumbnails👍👍 Love u levy so good RU-vidr🤝🤝