I Made a Self Driving FPV RC Car 

I like the idea of "Quiet study space" with an RC car zooming by blasting a laser and smashing into most walls 7:10

02:12 you perfectly captured and displayed that feeling that a programmer feels when his super complicated code actually works

The RC car knows where it is at all times. It knows this because it knows where it isn't. By subtracting where it is from where it isn't, or where it isn't from where it is (whichever is greater), it obtains a difference, or deviation. The guidance subsystem uses deviations to generate corrective commands to drive the RC car from a position where it is to a position where it isn't, and arriving at a position where it wasn't, it now is. Consequently, the position where it is, is now the position that it wasn't, and it follows that the position that it was, is now the position that it isn't.

Would love to see how you built and programmed everything.

Hey, quick suggestion! I noticed the vehicle was tending to take corners too fast, resulting in it having to realign itself back onto the determined path on the next straight. To improve efficiency on your turns I would add a deceleration/ braking element to that portion of the path. By determining the maximum velocity your vehicle can take that 180 degree turn at without skidding or flipping (Ekin, centrifugal < Fm, gravitational Force on vehicle?) then you can create a brake slope starting at the point of highest centrifugal force (1/4 PI of turn circle, as the 180 degree turn is a 1/2 PI and you want the center of this section as it is the outer most point of the turn) and then setting up brake points along the run-up to the turn that will reach your target speed (the maximum achievable corner speed) by the time the car hits that outer most point in the turn. Then you could even reverse this brake slope on the tail end of the turn into an acceleration gradient returning the vehicle to it’s top speed for the straightaway. Just a thought though, I have no experience in any of this and I don’t know if this is even a feasible solution, just thought I’d throw the idea out there in case it’s worth anything!

Compute the sum of the curvature of the N points ahead of the vehicle. Perhaps extend the lookahead by a factor of the current velocity. Then use the result as an input to a throttle PID feedback control loop. Low curvature should result in a high target velocity, and high curvature in a low target velocity. This should make the vehicle decelerate sharply in anticipation of upcoming corners and keep a tighter racing line. I would compute the curvature like this: let summed_angle be 0 loop 0...N let vector1 be point[index + 1] - point[index] let vector2 be point[index + 2] - point[index] normalize vector1 normalize vector2 let cos_angle be vector1 dot vector2 let angle be inverse cosine of cos_angle let summed_angle be summed_angle + angle end loop return summed_angle

It’s either self-driving or remote control, not both. Also slapping a go pro onto something does not qualify as FPV.

As a software development student, I can confirm this guy is leagues ahead of a normal programmer.

This would be a great Setup for an AI-Based Driving FPV RC Car. Use a neural Network for the Controller-Points with Attributes like Speed, Brakes and Steering and let it develop with small changes in the Attributes to the wich Evolutionstage gets the best Laptime...

That's awesome! Now make it want to explore unseen parts of the map so it can map whole buildings!

1 letter can cause 1 month of delay. Really makes you appreciate programmers.

Hi, how do you use the LIDAR and create the map? I'm doing something similar but I don't know where to start

This video is really well made, keep going like this and you’ll have a million subscribers in no time. Awesome project!

Can you tell me about programming language and interface that you use for controlling car ?! Good job !!!

if you only the lidar sensor,I suggestion you can try this algorithm named hector

what lidar did you use?

banger

6:00, Love that the clock is just in time seek mode where it turns like that until it hits the correct time (radio controlled). Albeit sped up a bit.

lmfao im doing the same thing, but we have a slightly bigger car

1:15 You just captured in film a phenomena that happens with every developer in this planet 🤣

Have you seen the scifi movie "Screamers" ? 😀

Good to know those Nvidia boards can take a beating, haha.. I've always been super careful with mine 😁

0:13 Literally every programmer in the world

Great insights into how challenging it is to make an RC car automatous. I noticed is at higher speeds the car seemed to overshoot it’s intended path when cornering. Not sure if this a latency between the car knowing it’s position and when steering response is applied, or something else (like a slippery floor). An easy fix is to base the speed of the car on how straight the upcoming path is, this way it will slow as an expected curve in its path tightens. This should then allow for higher top speed in the straighter sections of the course. Thus an overall faster speed on laps. ;) Look forward to when having multiple cars racing, and all the improvements being worked on!

You are awesome I can create a blog for you and Rate my Blog my friend. Unleashing the Speed: Building a Self-Driving FPV RC Car Introduction: Are you ready to drive into the world of autonomous technology and high-speed racing? In this blog, we'll explore the fascinating journey of designing and building a self-driving FPV RC car that can reach speeds of up to 40 mph. From overcoming technical challenges to fine-tuning controls, this is a ride you don't want to miss. Simultaneous Localization and Mapping (SLAM) To enable our self-driving car to navigate autonomously, the author implemented Simultaneous Localization and Mapping (SLAM), a crucial technology for real-time environmental awareness. However, the initial implementation of the sensor-based SLAM algorithm faced significant challenges, leading to an incomplete map. the author had to drive the car around to gather more data and refine the map, eventually overcoming the obstacles to create an accurate map generator. Overcoming Technical Hurdles One of the key technical hurdles the author faced was the issue with the sensor-based SLAM algorithm, which relied on laser measurements to detect obstacles. We encountered difficulties in ensuring precise measurements and faced initial problems with an incomplete map. However, with perseverance and innovation, we were able to resolve the issues, leading to a functional and reliable map generation. Optimal Control and Racing Line Generation Implementing optimal control techniques was an essential part of our journey to create a high-performing self-driving car. Generating a racing line and utilizing a look-ahead algorithm for precise car control required ingenious strategies. We encountered challenges in discretizing the racing line for effective representation in code, eventually leading to the car's ability to pick the farthest point within a certain radius and align its wheels in the desired direction. Challenges in Controller Tuning Tuning the car's controller posed significant challenges, as over-correction of waypoints resulted in the car's instability and wiggling behavior. It became evident that driving the car at higher speeds would lead to potential crashes if the controller was not appropriately tuned. Through meticulous adjustments and problem-solving, we aimed to fine-tune the controller for autonomous racing, ensuring stability and precision in the car's movements. Embarking on a Racing Journey Our journey culminated in the successful creation of an autonomous car built for high-speed racing, marking the culmination of four months of research and development. Through collaborative efforts with a professor and relentless dedication, we rewrote the controller and generated waypoints to bring our vision to life. Looking ahead, the prospect of head-to-head racing projects and the potential for multiple cars racing, reminiscent of F1, excites us for the future. Conclusion: The journey of building a self-driving FPV RC car has been both challenging and immensely rewarding. From overcoming technical hurdles to fine-tuning the controls, the process has been a testament to innovation and perseverance. The thrill of autonomous racing and the potential for future projects hold promise for an exhilarating journey ahead.

dont use an Xbox controller, use what actual RC Race Drivers use. there's a reason they all use the conventional wheel controller and trigger, as opposed to anything else. trust me you will be ale to control it much better. I fly planes and Race 1/8th scale and 1/10th scale off-road buggies. As much as I like game controllers you can never have the same control over a car as you can with an actual RC Car controller and you want to use a quality one because the latency makes a HUGE difference. If if you cant at first think it will make a difference, it shows up in your lap times Big Time!!!

Could you put a bumper with pressure sensors on the car and tune your algorithm to be More or less carful in a turn depending on if it hit the wall or not?

okay... i understand its generating a flightpath on the fly, and all inputs are generated in realtime, but what function is the map providing? shouldnt it also be generating its knowledge of the world on the fly??

Bathrooms? NOPE! That echo is not only annoying it makes some words you say a little hard to understand instantly (Also annoying)

To be honest, it's not self driving if you yourself are programming (not literally) each turn and speed at each turn

Really good, Steven? Would you be interested in wokring on a joint project with an Aerospace Company for autonomous drone in Brazil(remotely)?

Your first mistake was trying to program such a monumentally complex behavior manually. Should've used reinforcement learning and have the AI simply to through checkpoints in the hallways as fast as possible. With weights distillation, it should be able to learn proper racing techniques that require some foresight, rather than just being a reactionary state machine. Of course there is no hope to do this in real time, this requires a very fast simulation. Luckily, your setup's simulation would only require the lidar map which you already have.

actually... the logic should it be better: if going to turn right, then take the most left lane first before making the right turn. So this will decrease the angle of turn, which enable the car to run faster during turning or reduce the need of braking. at least this is the way I would drive on a race.

_SLAM_ One single "-" character, lol. In racing, it helps to brake early, then power the gas out and into corners.

Do you like that LiDAR sensor? I’m about to buy one, but for the price I want to know I’m making the right choice.

can we do this with a drone? I want to go around a gated track and it remember my stick inputs and repeat it.

So essentially you design a vehicle that get inputs which are used to build a virtual environment. You then use this virtual environment to run simulations to train a model which you can apply to your vehicle and it will use the sensors and input methods to rebuild the virtual training environment at runtime allowing it to essentially operate in the real world. That seems like one of the most exciting industries to be getting into especially as someone interested in deep space exploration as AI seems to be the future to long term multi generation exploration missions which are going to become a thing in the not so distant future as we start to apply them to explore the remote ice moons in our solar system like Europa.

Time to upgrade your Ubuntu, 18.04 is EOL. :) Great work on the car!

What .. it works??? Then tell me, why is on earth using an an Antenna at rear?? Itu Seems using radio controller

1:48 Be lucky this wasn’t an actual rocket you would work on with a space agency. Otherwise, the rocket would have crashed

What are the program you used to write/edit the code? I usually just use MS code but this seems lighter.

Why not use Reinforcement learning? I saw some good examples of the same thing but for drones!

It seems like it needs something to compensate for its weight, drift and stuff like that. Lol

Damn Rice is that you?

you should focus on off-road AI basically "sh**ty-road-no-markings". i fing that self driving cars in my opinion are meant for San Francisco and the likes and when they are places in "dirt-poor Africa" where im from ... uuuuuuuuh the tech is still a million miles of, if the AI can navigate africa then the world is your oyster. BUT HEY GREAT SUPER FANTASTIC WORK AND KEEP ON GOING

C'mon man..............:(n even i can do better than that. Provided i could afford an RC car, let alone a lidar. Programming, however, could be improved. Ultrasonic sensors could be added....

Do you use Nav2 in this project? I am working on SLAM, seem we are using the same SLAM toolbox packages. However, I couldn't make it.... the map frame is not exist when I launch the it. seem the TF from map to odom cannot be published... do I have to add the base_laser in my URDF so that to make it works?

Wow, this things goes off when its working "as intended", the head to head part should be more like "real" RC racing, the SAME RC cars and the "racers" are the developed algorythms, the best algos win :D Thats what we need to get forward in AI self driving cars, not Teslas recording the nonsense happening on the streets. I miss that brushless sound, i had once also a monster truck going 60mph+, but in the end it was just too expensive (killed batteries like 2-3 kg LiPos a year, replacement/tuning, maintainance, service of diffs, new bearings,...) Fun, but expensive.

Now that you have the algorithm working well with the physical car, can model the environment. It's time for you to take it to the next level! You can train a neural network to build a generalized, machine-learning algorithm capable of all the "optimal control" stuff inherent in the fact that it will optimize for getting to major features (start A->corner B->destination C) as well as lap (start A->destination C) as fast as possible. Hitting the wall naturally slows you down, so the algorithm would theoretically not want to do that on its own and it would optimize its angle of approach to maintain speed. I would imagine it learning to ideally by move OUT as LOOSE as possible before taking the corner IN as TIGHT as possible. All while remaining on the correct exit trajectory. When you said that you'd have other cars against you, I immediately thought of a generative adversarial network (GAN) where A's loss is B's gain and vise-versa. ML is mostly training of a simple algorithm. Anyway, if you want to collaborate, I'd love to help you get started.

Consult with actual pro or semi-amateur RC Race Car Drivers. in your case road buggy Race drivers. you really need to have a talk with them about what controller you are using. they will all tell you the same thing. DITCH THE GAME CONTROLLER!!! LOL, They will also tell you that latency is important to make you feel connected to the car. I think you crashed because of latency and possibly you could not see around the corner properly. OMG, I just realized you are using Wifi too. dude seriously get rid of that stuff for when it's under human control. that's fine for communication for autonomous driving, but for human driving just get a regular Transmitter and receiver.

lol running in the 90s xD

Amazing project, i doing some related project autonomous vehicle in Indonesia, but i think i need to build f1teenth my own, but the BOM is restricted now, may you give me the BOM of F1teenth? Thanks

Amazing Work, Keep At It, Hope I Can Read Your Name On Some Paper In The Not So Distant Future, Even If It's An "Et al." hahaha have you though of implementing this on an fpv drone?

Reminds me of my first big EE project on a team at DigiPen in 2003-2004!

6:10 with all these yellow lights, the car is definitely running in the backrooms

Ah E5-E7 dull and boring, but home to so many poor students))) p.s. how many lidars did you go thru? my Hokuyo didn't survive even one crash

Just one thing I didn't understand.... I know you specifically noted that bathrooms are the best place to record RU-vid videos but I still can't wrap my head around it. 😂

My guess about this rc code See's line of points = run straight. See's end of straight line of points = slow down. See's points (any) angle = Steer to it. Im no pro coder, I just used the monkey sees action, neurons activated meme. Edit: if the line intersect with another line, the code will set time dial to points already passed so the rc car won't suddenly break

What don't you use game engines for you test ground before going live? Unity, unreal.... The driving Algo is find a point in the middle of the walls and auto-adjust the steering of the wheels? Or something more elaborated? How to make it do U-turn ?

What's up with the guy talking loudly in the bathroom? Oh that's just Steven recording RU-vid Videos!

Are you joining IV F1Tenth? Good luck!

Self-driving, doing surrounding surveying and mapping... Isn't this what the sweeping robot is doing?

How much is the lidar on that thing? Do you think lower quality lidar (around 100-300$) would still work?

Precisely why people tell me they don't want to code. Finding your own mistakes is 80% of the job. LOL

Bro, the hardest errors to fix are the logic errors caused by one single non-compilation-breaking character lol Sorry you had to deal with that. Good job :)

for self driving cars, can they be controlled by a bunch of if then type code lines? like logic? or does one have to use learning algorithms and generative ai stuff that tries again and again and has parameters like not crashing/not experience excessive acceleration/accelerate other objects by colliding into them minus plastic bags. ? why cant self driving cars me made to work by having some low skilled person writing the logic that is needed to avoid each accident experienced by an rc car, then later a simulation and then a full size car?

WTF is the point of an FPV "self-driving" RC car? Supposed to be similar to a rollercoaster?

I wonder if security cameras will ever be mouinted on moving vehicles that run on a track in the ceiling of buildings and offices. They can ru n high speed and stop and reverse with a camera on both ends front and back. Talk about tracking a subject.

Vision would’ve been better than LiDAR, no?

(-) I felt your pain there! imagine a zero pixel space! I spent weeks refactoring code with the same space!!!!

I think you have some roll-induced steer happening. You can fix this by modifying the steering arms a bit.

A vehicle using a pre-generated map and following a series of waypoints is not self driving. This is just a more sophisticated version of a line following robot. A real autonomous vehicle would use sensors to detect and avoid obstacles in real time.

You are brilliant. Keep up the great work! PS love your recording "studio"!

Nothing impostor new is like you put some sensor and GPS like a drone or something similar 🤷🏽‍♂️

For part 2, please make both of them update their positions on the same minimap

what simulation software did you use? How were you able to genearte the map from the Lidar Reads? I'm assuming the maps generate in real time

Anyone knows what is the best budget rc for the project?

PID Control still has room for improvement, the best way is using AI learning to get the optimum value.

It's braking to compensate during a return when it should probably slow down before the turn and regain speed coming out.

you need to adjust your code to seek the fastest racing line through those turns.

One thing, I don't know why It doesn't even matter how hard you try Keep that in mind I designed this rhyme To remind myself how I tried so hard

couldnt you make it so that it just runs off the lidar in real time? and then just bias it so when it comes to a junction in the hall it will always go left? would have thought that would be more accurite for driving as you have to deal with under/oversteer and all that fun stuff.

is it making determinations for the acceptable speed for each corner? or is it just following the route and then you are manually tuning it to a speed that it doesn't crash??

Here's a hint X^2 So it doesn't always need a map.

Hey Steven, What all things did you used to make this Thanks, Jaidev

Ohh dude if you could perfect this for 1/10th or 1/8 scale Touring cars first, then Off-Road buggy racing, and actually get it up to a reasonable racing speed against humans that would be Legendary. Do it mate! GO FOR GOLD!

Sometimes when dealing with Ros, one thinks it might be easier to remake your own ros than deal with the shenanigans of ROS. But we are all lazy so we deal with it

use a genetic trained NN to make it self optimizing. It will hit the wall 99% of the time in the beginning. But the other 1% it will go zoom

It’a sad beauty bloggers get more views than this for bitching about one another… hope the video gets some traction !!

Glad I'm not the only one that could spend a month debugging a single keystroke. #NoobLife

as its a smooth floor and you had eurobeat playing, try tuning your program so the vehicle drifts around the corners!

This was so fun to watch. Thank you for the inspiration! By the way, bathrooms, lol

Well nice build and good idea. But I don't think it is considered " self driving" since it can drives pre determined routes.

Bathrooms are terrible for narrations esp on video. But singing is another matter! Please repost this video all on song recorded in that bathroom.

if you handle breaking properly and factor it in your algorithm, you can get better results

@steven I am keen on learning more I am trying to do this for my daughter’s ride on vehicle since she has low vision

Lol, an errant negative sign. 😆 Sounds like a unit test or few would have helped!

well done... can't wait for your next video... keep up the great work.

Programming. You gotta love/hate how much time it can sometimes take to find a coding mistake.

Nice. You already caught up to tesla. They are at this stage in self driving.