How Good Is ChatGPT's Spatial Awareness? 

"Where does the cube go?" "That's right, in the circle hole!"

ChatGPT is like if you skipped preschool and grades 1 through 8 and went straight to high school

You can see how it can recognize basic shapes, but has no idea what a hole is.

0:43 ChatGPT finally drew a circle!

"here's a rabbit shaped hole" "no thats a hole in a rabbit" "my apologies" *draws a bigger hole in the rabbit*

chatgpt is better at astrophysics then putting basic shapes into their correct holes...

"Draw a hole for this object to exactly fit through" (Okay Chatgpt, all you have to do is draw an outline of this object, not that hard...) "Here's the hole for the object." **Shows said object but with a hole inside of it**

I think the way the AI renders the images is simply with a text prompt generated by the language model, so there’s an extra layer of communication there which often can fail.

To be fair, 1:14 is accurate. Nothing better to fit a rabbit than a rabbit mommie's tummy.

1:07 Why is nobody appreciating the fact that he's doing this just for "rabbit hole" pun?

The problem is that ChatGPT itself wasn't drawing the rabbit shaped hole. It saw that it was a rabbit, and sent a prompt for a 2D rabbit shaped hole to an image generator, which then failed because it doesn't know what the object itself looked like from the prompt. I think.

ChatGPT crumbles under no pressure. You say “you are wrong” and it’s like “I’m terribly sorry, I’m a moron”

Technically the size of the cube compared to the hole was never given, so it might be able to fit through the cicular hole Edit: Nvm I didn't notice it said "exactly"

To be fair to the AI: I saw the wooden cube and the instruction being something about making a hole "exactly" "through" and immediately imagined the same wooden cube with a circular hole drilled through the middle.

i like how stupid chatgpt is

2040: *someone on AI powered-life support* “Can you turn off the air conditioning?” AI: “Sure, allow me to do that.” “I said the air conditi-“ AI: “I apologize for the confusion. Turning off the air conditioning.”

Yo that first rabit hole art is fire tho

It's because chatgpt isnt directly hooked up to DALLE (the AI art program it uses). What it's actually doing is prompting DALLE behind the scenes and returning the result. Chat GPT actually did an amazing job considering this restriction. It realized after the first one that dalle wouldn't know how to parse "an X-shaped hole", so it switched to "a 2D outline of an X" instead. That's some out of the box thinking.

"Oh no! Ai is taking over the world!" AI:

That last one was unfair, Kool-Aid man travels exclusively through non-holes

1:01 That hole rabbit is adorable

I love how all these AI seem so cool until you really start pushing their limits then the whole illusion of "Intelligence" breaks down

I love how chatGPT is better at apologizing than most of youtubers

This is not how ChatGPT and Dalle work. They are two separate models with only very limited communication between them. If this was an actual multimodal model you’d get very different results

Just a reminder that even though the AI says it knows the answer to something, it’ll assume the non-sentient image generation algorithm has full context of the conversation thus far.

I subscribed for a month of chatgpt 4 again this month just to check on the new context length upgrades. it really helped. It vastly improved in its ability to play games. it was able to solve a "escape the room" puzzle i gave for it with only two minor prods (i just said: "Maybe you should investigate another area you still haven't investigated for more clues" because it was kinda hyper-focusing on one piece of the puzzle). it did not lose context all the way through, it did not forget its inventories, or what objects were in the puzzle and where they were. Nor did it forget it was a game and just start roleplaying a story towards the end. It was also able to get a passing grade on the "battleship game" test it failed miserably at before. Its only negative marks were that it failed to understand one of the rules: that two out of the four ships were sized at 2 squares, and two only one square - it made all four ships only one square. and it shot twice one the same place once, but when i alerted it of its mistake, it never happened again. before, it would ruin the game two moves in. the escape the room too - before it would focus on one puzzle and literally forget the rest of the room, it was unable to keep track of its inventory, and the room's inventory, and it started roleplaying like it was an RPG instead of playing the escape the room game as intended. Now it can probably solve a much more complex room than the one i gave it.

The first rabbit would make a good game character

You have to be aware that ChatGPT is just prompting DALL-E 3, it isn't making these images itself really. However, there is some interesting work in this direction I think, for example I think there is open source model that is fully multi modal in this regard but haven't tried it

GPT-4V is only multimodal for its inputs. When it "draws", it's really just making an API call to DALL-E 3 with a prompt, which is why it struggles to produce exact results. Also for some reason, they don't show it the images so if you want it to see what "it" drew you have to reupload them.

1:03 Minos Prime's bunny

I mean *technically* the first one was correct

I thought the Kool-Aid man one was gonna be a brick wall

0:36 If you look at it from a perspective from below, the cylinder actually fits there.

I love how it suggested a spherical hole for the sphere. ChatGPT is thinking in 4 dimensions.

I think it fed "rabbit-shaped hole" to Dall-E and Dall-E gave it a hole that has been transformed into a rabbit rather than a hole with a rabbit silhouette.

ChatGPT understands spatial reasoning just fine. What this video is showing is a breakdown in communication between ChatGPT and DALL-E. A better test would be to present ChatGPT with a shape and a hole, and ask if the shape can fit in the hole.

"ai will replace artists!" the ai in question:

The problem is more in dall-e. Dall-e does it's own thing and chat-gpt can only make vague suggestions to it.

This is a bad experiment, because ChatGPT doesn't actually draw the drawings - it tries to figure out what you're describing, and sends a prompt to another images generating AI. That's why it generated images that make no sense in context - the guy doing the drawing has no idea what you said, just what ChatGPT asked for.

They should have done a wall for the kool-aid guy

You see it that it didn’t establish scale, that cube it drew could be 10x bigger than the earth. That cube definitely would be able to fit through it then.

What's the song?

1969: "I BET youtube will let us play music with screen off In 2000!!" 2024: "haha. AI dumb"

1:00 That's just a rabbit getting getting shot lol

I knew I was right all along as a kid when I tried to put the cube block in the circle hole.

Interestingly the issue appears to be with DALLE. The image interpreter that chatgpt is using to get a description of the image just needs to tell DALLE when it isn't right. Of course the issue is deciding how many image generation calls it is allowed per user prompt.

That rabbit actually looked like some brand mascot

ChatGPT simply couldn't comprehend the unlimited power of the wooden cubes

Casually draws Kool-Aid man pulling off his skin.

Those hole rabbits are really awesome designs ngl

The editing on this was amazing. Also when the rabit appeared I felt bad for the AI and audibly called you mean for switching up the difficulty level.

1:04 NO WAY UNDERTALE REFERENCE?!?!?!?!?11!

Artists: "AI's going to steal our jobs!!!!" AI:

this is some real "put your hand on the lid" type shit

honestly impressive enough considering chat gpt has sn aversion ti recognition of copyrighted materials.

Ngl the first rabbit one looked sick as hell

The rabbit hole with a hole shapes just got me 😂

This is definitely more of prompting issue. I tried the same thing but with a more clear prompt and instead of "hole" used the word "cutout", it worked very well. Any limitation from there on would be due to the way image generators work when using specific keywords and descriptions.

its integrating different ai models which were not trained to be integrated together

with enough force, anything can fit through any hole

0:08 no, it did right. This is a common notatiom used in crystalography and math Except Its really uncommon But it is right.

A couple of those hole-rabbits would make pretty good logos

i was genuinely expecting chatgpt to say "fuck you" in the kool-aid man part

The first rabbit hole pic actually goes hard though.

"This is a rabbit-shaped hole" *Draws a hole-shaped rabbit*

Buddy, everything goes in the square hole

Try this but with svg image generation. By disabling dalle in a custom gpt, it will generate an svg, and you can actually have it edit the image and shift stuff back and forth through the layers.

My new favorite thing I somehow need to incorporate in my everyday life: rabbit-shaped hole

i like how it didn't know how to move to the next phase of the conversation and kept thinking about holes.

bro the editing is f*cking hilarious 😭

1:02 that is god damn minos prime

the music from youtube library is goated

Impressive that it recognizes the pose as cheerful though

This is why you have to distinguish between machine learning and AI. Machine learning like chatgpt can give you information it has already been given itself. It has no sense of logic, intuition, or problem-solving. It is not an artificial intelligence, which would understand that the cylinder goes through the square hole, or that the star also goes through the square hole, or that the rectangle goes through the square hole

the "rabbit holes" look cool

Okay but that rabbit design actually looks like a sick logo tho

I think that ChatGPT itself did good, but it's the image generating module that isn't smart enough to actually create the image that ChatGPT wants to draw. In the background it's ChatGPT prompting Dall-E to generate an image based on your prompt.

Chatgpt understands fine. Dall-E is the one messing it up, it looks like. No way to know for sure without asking for the prompt it used

The Quake menu select sound does something to my brain...

why's no one talking about how clean the star was, and idc that this is the 8th comment

People out there using chatgpt for their schooling and/or job anyway: "Sure, hole in a rabbit. A rabbit could probably fit through there, whatever."

The image generation and natural language component of this model are distinct. Hopefully in a few years the steering will improve

To be completely fair, I also assumed the first task is to draw a circular hole with a radius equal to half the cube side's diagonal.

That's because chatgpt doesn't intrinsically know what a "cube", an "hole" or a "cylinder" are. It works just by searching on the internet everything related to the subject and approximates the result by combining them. Someone did a logic experiment about it, by typing the prompt "what is the 21th letter of this phrase?" And chatGPT gave always a different answer, cause doesn't know what a letter is. There's no intelligence in AI and everyone is losing their mind for what essentially is just a sorting and approximating algorithm, this is literally one of the biggest frauds in history

"Matching its cheerful pose" AI thought it ate 💀💀💀

1:02 bro took rabbit hole too seriously

How is your chatGPT creating images? I tried but it said it cant generate images

That's right! It goes in the square hole.

I just want to know why both pictures of the rabbit have some sort of liquid flying off the rabbit

the rabbits look really cool imo, even though they weren't what you asked for

one thing to think about is that the images it is trained on are 2D. they may seem 3D to us, but they're only illusions, they're 3D enviroments projected onto a flat 2D surface. they have no depth. we live in a 3D enviroment with depth and stuff so we understand them, however AI does not have this, everything it knows is 2D. fun fact! humans only learned to use perspective and space in paintings relatively recently. until a certain point, paintings were just 2D.

Well, this is really unfair because chatgpt is not doing the drawings, Dalle, is doing them, chatgpt only writes the prompt.

where did you get the wood shape images from?

It started off so well

"Oh no ai will replace artists" they said

these really highlights how little commoners understand LLM. someone with a middle school level education can figure out the correct prompt to consistently get accurate results for this problem format, makes you wonder where the educational system failed for the uploader

that chatgpt rabbit would make a sick af creature tho

1:00 rabbit minos prime

I love the music and sound effect choice

That's right, it goes into the square hole!