All credit to comma.ai team for uploading original high quality video on their official ru-vid.com | comma.ai/shop/products/three ($2199) | comma.ai/jobs | Buy comma.ai/shop (Merch) | Follow the official ru-vid.com for more videos. Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop. Chapters: 00:00:00 Intro Harald Schäfer 00:00:39 What is a real self driving car 00:02:21 The classical approach to self driving 00:03:54 Classical perception 00:04:41 Classical planning 00:08:02 The flaws of classical approach to self driving 00:09:17 Remembering how to drive 00:10:02 Why is end to end better 00:11:23 The new approach to self driving 00:11:55 Is all starts with data 00:12:45 So much diversity 00:14:12 What is the output? 00:14:44 Localization 00:15:18 GNSS 00:17:08 Sensor fusion 00:19:58 Time to train 00:20:48 Validation 00:21:48 Quote about software engineering 00:22:09 Simulator 00:24:16 Problem: cloning 00:27:10 Solution: train in simulator 00:28:25 Testing Testing Testing 00:29:33 Does model know it's speeds? 00:29:47 Is model accurate? 00:30:05 Does it drive well? 00:31:09 Does model predict swerve 00:32:05 Problem 2: Desire where do you want to go? 00:33:19 End-to-end that works 00:33:57 Technical difficulties 00:34:28 End-to-end lane changes 00:34:40 Technical difficulties 00:36:22 Some questions 00:36:27 Training the model components 00:37:06 End-to-end lane changes 00:37:53 End-to-end lane less 00:38:49 Longitudinal end-to-end 00:39:24 End-to-end forward collision warning 00:40:10 End-to-end stop lines 00:40:48 What is next for comma research? 00:41:32 Q&A 00:42:11 Driving without GPS data 00:43:22 Good data for training 00:44:35 Bad stop sign human driving data 00:45:10 Training out unwanted behavior 00:46:19 End-to-end lateral control 00:47:03 Explaining the model behavior 00:48:37 Larger training models and edge cases 00:49:44 Driver alertness 00:50:49 Community contributions 00:52:15 Evolution of end-to-end approach 00:53:31 Bounding boxes 00:54:40 Adding more complexity 00:55:20 VR sensor fusion 00:56:00 How big does the models have to get 00:57:00 End-to-end not fully end-to-end 00:58:20 Hardware limitations for Level 5 00:59:13 Data visualization of the model 01:00:02 Driving data utilization and benefits for the user 01:00:42 Path prediction on curvature 01:01:43 Pedestrian crossings 01:02:15 Lane changes and blind spot monitoring 01:02:47 Simulator scaling to more complex scenarios 01:03:37 Self-supervised learning 01:04:07 Hand labeling and comma-pencil usecase 01:05:19 Model mistakes and failure prediction 01:06:09 Artificial general intelligence 01:06:49 Failures of localizer 01:08:08 Model reliability and scaling to city driving 01:08:56 Differences between drivers in different cities 01:09:18 Humans teaching the car how to drive 01:10:00 End-to-end lateral planning model cheating 01:10:46 Training process and incremental learning 01:11:58 George Hotz More GPUs 01:12:02 Biggest limiting factor of improvement 01:13:11 Visual geometry lane lines bug how it was fixed 01:13:49 Harald's birthday
Look at this. Free 3 years of googling and countless Jupyter notebooks on cool machine learning ideas all in one video. Imagine googling this and this guy just puts everything in one video. Nice.
You thought the 30 under 30 list was rough but Harald is leading a successful open-source self-driving project and he’s barely 2 years old! My nephews could barely eat without putting most of their purée in their hair at that age.
Would be interesting to know how far a super human driving agent could navigate blind from one snapshot of it current position, velocity, location, inferred object physics/behaviour, conditions and other inputs using its "imagination" before it fails, to see what margins exist in various scenarios and how much more anticipatory such systems can be compared to consensus human driving models which are bereft of these electronic data points.
Given there are limited resources aboard the comma devices, I think the efficientnet should be fine tuned based on the country you're in. (And it should be further fine tuned based on the driver.) It will be tricky to know how many layers to retrain to gain some additional accuracy.
@@kamalmuradov6731 Not the same. This is a single neural net that takes raw sensor input and spits out driving commands. Tesla's system is far more complex, it has many stages and modules, some are even hand coded. There are similarities too, that differentiate these to from the rest, like sourcing data from customers and not using lidar. The ends state of the two systems will likely be the same, but they are taking different approaches. Both has it's advantages and disadvantages.
@@moracabanas Sorry I forgot the context of my comment, and the video is too long to watch it again, my only guess is that I'm talking about a 2 year old AI maybe Idk
Not even close. Tesla has some of the top AI experts. Andrej Karpathy for example. And Tesla is far ahead and has far more resources, so they will win, that's a given. Even George Hotz says Tesla already won this game.
@@andrasbiro3007 Well that's fair but IMHO being able to get more than 50% profit researching this "for fun" and open sourcing it for people, and also giving us the opportunity to buy comma 3 hardware for under $2k with weekly updates and pretty reliable level 2 system in pre 1.0. is more than insane. I mean Hotz is just now taking some time off comma making livestreams on Twitch porting tinygrad to every possible SoC which could be candidate for running it. They coded tinygrad now used on comma and it outperforms the native Qualcomm ML framework. The last day I've seen him porting tensorflow and tflite models to tinigrad for Google coral in a few hours what the fuck.
This is like what the Tesla team is doing, but it seems to be able to do less things. I'm grateful to see that others are trying hard to make this a publicly available solution though. Nobody should privately possess this kind of technology to sell it. Great talk overall. I think it shows just how complex this open problem is.
