AI Learns to Park - Deep Reinforcement Learning 

Watch two AI Agents fight for the same parking spot: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-CqYKhbyHFtA.html

This is the definition of "don't practice until you get it right, practice until you can't get it wrong"

Actual footage of me trying to park after 2 years of driving

It looks like it's having to fight the voices in its head telling it to run full speed into those other cars

This kinda makes me appreciates the NPC's driving in GTA V.

*Pushes through car to get to parking spot" "Boom! nailed it"

I like the strategy at 2:45, sometimes I wish I could do that when I park...

Came for the car going around randomly, now I can go to bed happy

2:48 "Leonard that's my spot"

"What is my purpose?" "You park cars." "Oh god...."

“AI will take over humanity” 300k iterations later, the car still gets a parking ticket.

I wonder how much the learning process would be affected by feeding in not just the current sensor input, but the last couple of seconds of sensor inout (so 2 seconds * 60 FPS * 8 sensors additional neural inputs), and maybe the last 2 seconds of the AI’s own output decisions as well. The network would be much bigger and training would take much longer, but I wonder if past input and decisions would feed forward in time to produce smoother results, preventing the AI from just going for a bad solution as often.

4:17 okay, now its just showing off.

"It has 8 depth sensors" *breaks the pole* Is the sensor learning too??

What the- how did I never realise that my parking practice was being recorded??

Since the initial position of the car for every learning try was random, I imagine that the resulting model with 300k or so learning tries should be able to park on the remaining available parking slots. Checking if so, and if not why not would be great tbh. beautiful work, I feel more than lucky to be able to see such nice results presented in such an interesting manner!

I love it, when ai parked perfect to the wrong parking spot.

If there's an artificial intelligence apocalypse, we'll know that this guy left his computer on for too long.

2:49 my favorite attempt. " 'Scuse me, that's my parking space. Move over."

More penalties should of been used, ie, driving on the curb, and instant failure on crashing. Also a segmented training might of helped, with the car spawning closer to the space in the beginning, maybe even changing the spot to park after a success

The little speed boost it does whenever it is nearly there is so cute imo

“After 5k attempts” *oh shit he’s getting close* “After 15k attempts” *alright how long is this video now?*

Multiple issues i spot: Sometimes the raycasts see nothing (it is plowing a car, yet raycast is green and max range), Raycasts are raycasts, i suggest boxcasts to be more realistic at least for the front side. The parking spot never moves, nor does the other cars/environment, so this does not generalize well at all either (if it learned this spot, a different spot will likely not work at all). Hitting an obstacle or invalid driving location (sidewalk) should instantly end the simulation of that episode to iterate faster on problematic attempts.

It's kinda like watching a bee under the affect of alcohol trying to fly into it's hive

This is pretty much a real life depiction of my data analyst job experience

Now i wanna see multiple ai on trying to park at the same time.

I’d love to see it have its parking spot selected randomly from the 6 open ones, then it’ll have to learn more generally how to go to any location rather than a specific one

This makes me feel a lot better about when I learned to parallel park.

At around 15-20K it started looking like one of those Russian security cam videos

If (going to crash) Then (don't) boom, programmed

Still better than me

When you compare the beginning to the end it's just awesome!

Car AI: **Tries to park** Samuel: "I haven't programmed that path yet"

I love how at 20003 it got the first taste of that sweet parking reward, then it needed a fix so badly it rammed the white pickup out of the way.

Even my mother can do it in just 15k attempts!

"After 10000 attempts..." _Already drives better than my sister._

It would be interesting to see how it would manage with random cars in space placement and a random space to park in. My guess is that in its current form it would really struggle, but I wonder what the difference in number of iterations would be :)

Me with 4 overdue assignments at 3 am *I N T E R E S T I N G*

Deep reinforcement learning sounds more like you hit the AI on the hand with a ruler every time it does something wrong

Feels like watching a kid growing up x1000 faster Succeeding, failing, still moving no matter what This is so cute 🥰

my friend: "im not THAT bad at driving" also her:

Overfitting? Should've given different parking slots

I love how the AI tries to park in all the different spots.

I like how it almost successfully parked in the wrong spot a few times.

I love these AI training videos. It’s hilarious contrast seeing how childish and bad at it the computer is, contrasted with knowing how brilliant everything that goes into making this happen is.

"AI is penalized for crashing" 10:07 spawns AI on top of car AI: One day you will pay, humans...

So you're saying all those 3 AM Walmart parkers are really AI'?

a couple things i noticed: - occasionally the car will crash into things in front of it because of the one-dimensional lines of sight. - car appears to prefer to turn left. because of practice perhaps? very cool stuff!

Love how the music switches to the lofi hiphop music when the AI parks (near) flawlessly and switches back when it doesn't lol

after 20000th iteration: yeah I got it! I finally got it! 20001st iterations: rams into truck again

The parking lot configuration should’ve been randomly generated each trial

The car: "oh yea, you like that baby?", The spot: "It's not even in yet 🙄".

This is actually a simulation of me trying to park

AI: Learns to park User: cool, (proceeds to change parking location)

AI typically takes around 1 million try’s to get something and they can try 1 million times in like a few minutes... this is insane

It’s literally like putting a 5 year old under the wheel 🤣

Where I live, even the first couple simulations were pretty damn accurate.

I don't know why, but the first 5 minutes of the AI failing and just crashing everywhere was absolutely hilarious to me

3:22 When you try to back your car into a spot for the first time

Me normally: doing stuff almost flawlessly Me whenever others are around: *THIS VIDEO*

That instructor has a lot of patience

I wonder how well a HUMAN would perform with those 10 inputs!

People: AI will take over the world!!!! AI after thousands of (essentially) bruteforce attempts: 1:20

2:49 The AI went full Mr. Bean on that parked car!😂

Conclusion: AI won't control the world yet

2:55 "Bro what the fuck are you doing" "It's still learning....."

it got there in the end but man that first 90% it was just ruthless lol smacking cars and poles left and right

I hope one day i can fully understand stuff like this one and I can actually do it. Excellent video thanks for sharing

Leaked footage from Tesla R&D center

I found it very interesting to see how AI learns if you give it no information exept for the penalty and the reward. Nothing the AI does is intuitive. For example, a human would have understood very quickly that it is essential do drive a curve. The AI does not see such patterns as „curves“. It only sees the numbers. It only sees if it has gotten closer and if it has made mistakes.

What a generous driving instructor. I only got 1 chance.

I don't know why, but it's really cute to watch this AI car learn. As if watching a toddler learn how to park a car. Almost as if watching a real human being!

I was expecting a drift parking after so much attempts and watching AI Hide & Seel video.

I love how the cars bounce every time the program is initialized.

Imagine you would drive like this in driving school 😂

It's so satisfying seeing a naive robot make a clean park with no mistakes. Good model!

And we learn from AI to NEVER GIVE UP!!!! 😎👌

I appreciate the choice of music.

I'd like to see the enviornment change with every attempt. Despite spawning at a random position it might still have overfit. Amazing work regardless, keep it up.

Tested it with my 3 year old nephew... he mastered it after 5 attempts... We have a long way to go ;)

The biggest flaw seems to be the 8 'depth' sensor having huge blind spots, what if instead of 8 raycasts, you only had a single rotating one (maybe a few degree per frame). Let the AI know about the angle of the sensor relative to the car and the hit distance. That way it would probably be more 'aware' of his surroundings quicker.

So this is an example of something called “overfitting”. Though the “over” part assumes generalization, which isn’t the goal of this application, but I still think it’s a good talking point. The AI is learning a lot. But notice that it’s in the same map, with the same layout. So it’s mostly learning how to park in this EXACT situation. How does the AI learn to turn left? Well it could be something like if the front left depth sensor suddenly cuts in half, and all the other ones are at max length, then it knows to turn left. (I.e. driving past the first light post) A.I. Can figure out all kinds of tricks like that. However, it doesn’t have predefined cognition (humans do. Your brain is largely pre wired. You could argue that, through your life, your brain really only changes marginally with respect to it’s original construction) the predefined structures in our brain focus on learning generalizable ideas. This is hard to do with AI. A human could run this simulation in this scenario, and probably, after getting it right, be able to park in another situation. That’s because a) the sensors we use to perceive the world are super sophisticated, and kinda come pre-installed with the capacity to understand their input b) we focus on learning things like “stay on the road” not “turn left when front left depth sensor gets short suddenly”. The key difference between generalizable intelligence, and an overfitted system. If you shifted the starting point 3 feet to the left. It’s entirely possible that the AI would completely fail. However, if you put the car in a box, it would, after training through thousands of iterations, probably find a way to glitch the car so it pops up in the air and lands outside of the box. Systems like these are bad at generalized stuff, but really REALLY good at solving specific and difficult tasks. There’s a famous study where researchers wanted to make some NN for controlling walking. It had 6 legs, and they set a rule that it got more points for spending as little time on its feet as possible. After seeing that it learned to go 100ft with 0 seconds of foot contact, they found out that it learned to flip over and walk on its shoulders. What a time to be alive Edit: I didn’t really watch the video. The car changes it’s starting point, so it’s kinda generalized. But the map stays consistent, so it’s not really. If you moved an item around it’d probably freak out. Though idk, I’m mostly talking about AI for you the reader, not about the video.

Imagine if it drifted to the parking spot in the most badass way

So interesting to see that due to how it's trained, the AI will never attempt to park in reverse, even in a situation where that would be easier

On attempt 310013, it spawned on top of the truck!

Wow, this ai learnt to park perfectly in half the amount of attempts a normal person takes :O

After 310k attempts, now the AI can park perfectly in that one spot for that specific parking lot with that specific arrangements of cars!

Imagine you're the driving coach and this car is your student

2:40 when someone takes your paid parking spot

Its like watching my mom trying to park a car

After 3 million attempts it learned the urban inefficiency of surface parking lots and switched to a bicycle

At the end: he a little confused, but he got the spirit.

As an amateur ML programmer, this is inspiring! I may do a project as a result of watching this.

I could imagine some random guy in apartment watching this guy trying to park over 300,000 times

imagine if you left this until the 1mill iterations, the Ai would be making world record times for each random position

You should also track how well they're following the road as well as the driving direction.

The issue you might have with having only one parking space is that you could be training it just to park in that spot rather than any spot. Which kinda renders the ai somewhat useless in 99% of parking situations.

Like a baby doing it's first steps.. the dev must be so proud..

Medium-aged man: “If you can't do it well immediately, don't try it at all.”

I swear this is better than an actual let's play.