Really wish my experience with 12.4.1 was as good as others I see 😕 Sponsored: Use my special link www.privateinternetaccess.com/AIDRIVR for an 83% discount, 4 months free and 30-day money-back guarantee on the best VPN - Private Internet Access
You are by far my favorite FSD reviewer - I like the fact that you use voiceovers and focus on interactions that demonstrate the abilities and limitations of the software. I also enjoy that you must live near me - at one point in this video you drive right by my son’s apartment 😂
@@usernamenotrequired142 And to be fair to the fsd, it is *nothing* like a human driver. heck, any high school kid is a much better driver than the simpleton beta FSD software.
Dont even own a tesla, nor do I live in the states, but your channel is so relaxing to just sit back, relax and learn all these developments in self driving technologies. keep it up!
1 video from you is more valuable to shareholders than the 100th „no intervention“ drive from other channels. The system can do a lot. But we have to focus on what’s still missing.
As a shareholder I disagree. The other channels don’t have the same problems which means that the problems might be his car (hardware) specific. AI driver says so himself at the end of the video. I have no doubt some things are 12.4 related as well, that is to be expected. Having said that, I don’t think it matters much. Because I don’t think FSD will be unsupervised any time soon. “There is less and less interventions so we cannot improve the neural network” yet there are millions and millions of different once-in-a-lifetime edge cases that are extremely important for self driving to master (safety!) that require human knowledge way bigger than just a driving neural net will ever have. Knowledge about humans, about infrastructure, weather, nature, etc. Good luck training on land slides, wrong way drivers, lost ladders in the road, drunk aggressive pedestrians attacking you, stuck red lights, presidential convoy, accident in the middle of a crossing, getting away from hurricanes, escaping an attempted carjacking, etc etc etc. Hell, I even wonder how they are going to handle reading the thousands of different unique, text-only signs in the usa, let alone all the different traffic rules and traffic signs and infrastructures all over the world… It will take a LONG time.
@@qqleq thanks for your thoughts. I disagree on the once-in-a-lifetime events. Yeah, system wasn’t trained on every thing that might happen, but that’s true for us as well. We can still drive a car. We can react to new situations without specific training. If FSD gets similar capabilities it even doesn’t need to be perfect. The logic would be: Unclear situation > avoid collisions > come to a stop > request tele operator to take over control. So there’s no need to handle every situation but just to identify situations it can’t handle safely.
@@qqleq 1) Blaming the car has always been a common excuse. That is not the problem. FSD has all sorts of problems in my area and you can find many other FSD channels having problems too. It just does not perform the same in different places, nor does it improve as fast. Haven't seen any significant improvement in FSD where I live in years, despite the behavior and problems changing. 2) FSD does not need to be trained on every situation, nor does it even need to be able to handle every situation - that is what remote operators are for. The strength of the E2E Neural Net approach is that it learns to generalize and problem-solve situations it hasn't encountered, like humans do. The goal is to get that good enough that a relatively small number of remote interventions will be necessary.
@@catbert7Neural nets as they are now do not generalize the way humans do. Otherwise, they wouldn't need millions of examples to learn something that humans can learn almost immediately.
I have a theory for what happened at 8:35 where the path planner wanted to go, but the car stayed still. The new city streets model is supposedly trained with video+controls and outputs control directly, but we also know that they're still running the traditional object detection models (traffic lights) and some of the logic for determining which lane you're in (for the auto-canceling turn signals and to determine which light is yours). It's possible that they're still using the results of the older systems to add an extra layer of safety on top of the new "end to end" model. If that's the case, then the tentacle we're seeing is what the city streets model wants to do, and the behavior of it not moving seems like a hard-coded rule, a rule that's got higher priority than the city streets model does. If anyone remembers when Elon live streamed a v12 drive, one of the things v12 did was run a red light for no reason. I don't know about anyone else, but if I were an autopilot engineer, and I had a bunch of proven software (v11) that could help me ship v12 guaranteeing it wouldn't run a red light, I'd just gatekeep v12's ability to drive unless both v11 and v12 agreed about whether the light was green.
came to the comments to share pretty much this exact theory (even the part about running the red light on the live stream) but you explained it perfectly already. i think this is by far the most plausible explanation
Yes, this has been my assumption ever since v12 rolled out. Because if you stop and think about it, a pure "video in, vehicle controls out" neural network ... would not create visualizations on its own. It would be a black screen. The car would still do all of the same things you're seeing - but us human passengers would all freak out. So I assume this is just a "split brain" symptom of having two AI engines running off of the same video feed. One that makes pretty pictures on the UI screen based on all the image training data Tesla has done for years - but that's all it does, just render the world around it. And the other (v12) makes control output signals and that's all.
@@dougtoombs9195 V12 sure does create a "world model" in real time which can be visualized. You can actually see this visualisation when the HiFi Parkassistant view is active. "video in, vehicle controls out" is a simplification and leads to misconceptions. The controll NN (neural network) is not trained on raw video, but rather on the "world model" which is created beforehand. So a more accurate simplification would be: "Video in -> perception (aka "world model") -> control out".
Welcome to Walnut Creek. This drive was facinating to watch! I was thinking for that unseen left turn light at Broadway/Ygnacio was the use of a Programmed visibility (PV) head on the mast arm. It did see the PV light on the thru movement though. The upright on the left had regular LED bulbs but it didnt want to recognize those either. So maybe the issue was it not knowing you were in a dedicated left turn lane.
FSD will work great in northern Europe from day one. No weird left turns, no weird “no left on red”, no “California roll” at stop signs, not too many weird drivers, and a lot less traffic.
8:57 remember it running the light on the first preview we saw of v12? My guess is that at intersections they still have the old stoplight code running and the car isn’t allowed to move unless the old stoplight code says it can and v12 neural net wants to move.
Edit: 6:00 Thank you!!! Big respect Original: You should let everyone know that you misrepresented California law (and 46 other states) about entering an intersection when turning left at a green light. It's legal in California and all states except Illinois, Texas, and one other state I forgot about. Anyway, 3 states with unintelligent legislators.
I actually hope they keep improving multiple different models. It would be nice to have the option to have a ‘chill’ mode that doesn’t try to cut around slow traffic and cross double-yellow lines along with an ‘aggressive’ mode that does. During the free trial I flipped from chill to aggressive and didn’t see a single difference.
21:27 Based on my free month trial, I had tons of interventions and disengagements. Give me a longer free trial and I'll absolutely rake them in in Minnesota where there's of course much, much less data than California.
ai drivr out here risking his drivers license for our entertainment Also i live near walnut creek and its really funny to get some commentary on the drivers around here lol
I have a 2018 Model X and paid for FSD when it was promised to be revolutionary- I’d say that this version is probably the point at which Tesla has delivered on the promises software back in 2018. If you’re waiting for a robo-taxi you’re effectively waiting for 1) government approval of FSD (together with legislation changes to traffic rules and at fault liability claims with fully autonomous self driving vehicles) and 2) further software improvements / AI learning for several billion more kms. I’d also like to see the cars get FSD horns and a loud speaker like KIT
Well, this is interesting. I'm a bit disappointing the progress the past 2-3 months. It doesn't seem like they're able to quickly make improvements to V12, at least not without introducing other regressions.
8:45 I also suspect that there is some second system which regulates continuing at traffic lights. An intersection near me is no-turn-on-red, but from the line you can't see the signs which say so (they are too far back.) And as soon as traffic is clear, the path planner appears as though the car wants to make a turn on red, and the wheel turns like it is about to go. But it waits until the light turns green before accelerating. Possibly some system has memory of the rules of that intersection (or its just based on map data), but the path planner generated from the end-to-end network can't see the no turn signs and so it tries to generate a path to continue and gets overridden.
12:40 - I dunno if I'm helping at all, but there is in fact a human in that car, flashing brakes and everything. I don't see that reflected in the visuals, yet I wonder if its reading that and getting confused? Although you are right next to them so probably not. Although I will say, that the car that popped up afterwards, I'm almost 100% certain the car got scared and did an emergency stop cause it didn't show on screen even before the stop sign so I think that's what that was
I'm sure not a new question but have you performed a camera recalibration recently? There are some recent anecdotal posts (Tesla Motor Club Forum) that problems FSD owners have had with v12 were resolved with a calibration. No guarantees of course but since your experience doesn't seem to reflect other v12.4.1 experiences seems like a prudent thing to do. If nothing more just to tell people like me that you've done it. Watch all your videos and they are excellent by the way.
Great content, sad I was really looking forward to fewer disengagements from 12.3.6. Perhaps an improvement here is that 12.3.6 will sometimes be way too confident to the point of forcing critical safety disengagements. (Happens a lot where I live) Overall I love FSD and still super bullish, but really disappointed in the apparent regression of 12.4.x As a product manager in big tech, I think Elon's claim that there's 'not enough disengagements' needs a qualifier to note this means "validated disengagements" -- it's the problem of identifying signal from noise at scale. I guess the voice feedback feature isnt helping with this problem as much as I assumed it would (I use it every time). We probably need to move past RLHF and go to alpha go style training (AI training AI, reinforcing the model) but this is a hard nut to crack still. I doubt it's a hard limit but general progress in the foundational models has been slowing down and regressing even a lot more than I expected.
That said Anthropic's Sonnet 3.5 is showing initial signs of significant progress so thats bullish; notwithstanding that, its still too early to say we've hit a hard wall with our basic approach as we havent exhausted the work arounds.
top things that bother me about FSD: stop signs, going from one lane to two: it doesn't decisively pick one. The way it slams on brakes to stop at yellows instead of smoothly going through.
@8:40 It appears to me that the Rules of being the Left Hand Lane was consistent with what the driving ai wants to do, although, what I think happened, is the lights for the Left Hand Lane didn't appear on the visual HUD and seems to have been lagging. It was picking up the lights for the right hand lane and perhaps the system is a bit behind. I think your assumptions were correct, it was being held back by that Street Lights parameters even still it fully knew what to do next.
This feels more like my experience. I find myself having to intervene more than other videos I’ve seen. I also probably have less patience than those creating the videos.
12:05 there is a really good Tesla approved auto body shop there. They will fix your car better than when you took delivery. They're called Brooks motors
Totally agree. One of the first things I thought of when this update installed was that it has no confidence. Speed control and maneuver smoothness are also bad.
Makes me wonder what would happen if you had multiple Teslas all with the same version of FSD and dispatched them one at a time right after each other. Would they all behave the same? Would the disengagments be the same? Would it make a difference with no traffic vs. mild or heavy traffic?
15:00 I can't wait until FSD is smart enough to get you into an "empty lane" at stops, as opposed to waiting behind a long line of cars. There are several cases where the will be 2 straight ahead lanes at a stop light. One lane has 10 cars in it and the other has 0. FSD seems to like to pick the lane with 10 cars in it instead of using the lane that will bring you right up to the stop line, being the "first" car to go.
The autopark perpendicular parking feature could be adapted to do three a point turn imo. It might be as easy as the path planner virtually creating a parking space for the feature to work.
It seems there is a pretty direct relation between confidence and safety. It seems to be very difficult to train these systems to exhibit both behaviors depending on the circumstances. It very much feels like the hallucination problem with LLMs where confidence leads to confidently wrong outputs and safety leads to excessive refusal and hedging that makes the output less useful even for simple tasks where a lot more confidence would be reasonable.
I feel like since v12 "seeing" and "making decisions" is trained separately. I mean 1 neural net but on different layers. Maybe this is the reason of camera calibration makes effect. I can imagine that on the confused red/green light situation the car sees the light red, but based on the learned other car movements the decision making logic wanted to move because it felt that time to move.
Yes good point, since the car can’t back up / turn around on its own (yet) I consider any time it needs manual input an “intervention.” It doesn’t seem right to call it a disengagement like the illegal right turn it tried. Open for feedback on this. Maybe my old system was better (using “critical” and “non-critical” disengagements)
8:49 I think this might be the parallel system which monitors FSD continuously, overriding the FSD decision. That's the same system which will deactivate FSD and throw red hands if it detects a malfunction in FSD - true or not. That's why FSD 11/12 sometimes randomly disengages "catastrophically" with red hands in certain areas. Black Tesla made a video about it, calling it "Dead Zones".
Glad to see a more critical take on 12.4, I suspect the polished driving style comes with more updates and a more realistic comparison here would be 12.3.1 rather than later 12.3 updates. Regardless as you said they need interventions to feed the data collection, good review.
The most important thing is if it is safe enough for Robotaxis. We don’t know that your disengagement was critical because we don’t know if FSD would have been able to complete that right turn. Choosing that sharp right turn was a problem with navigation, not FSD, as navigation should have told it to make the prior legal right turn and not the illegal right turn.
On what happened at the intersection: it seems that multiple subnets of the system have to agree on a situation (lights are green) and in this case a part of the system did interpret the lights as red. Probably the visualization is not representing the entire system state, but only of a part of the system.
The best version was 12.3.3. Every version after that was a slow downhill. 12.3.4 introduced the wild lane changes. It would try to go to the right lane when it has a left turn coming up for example.
Idk a couple of points like where it doesn't cross over yellow lines to take the turn lane, honestly I do the same thing. I prefer it being predictably legal.
Question from a non-tech person: Why does Tesla simultaneously develop the next few iterations (e g , 12.4, 12.5, and 12.6) before they're even deployed? If 12.6 is best, why spend resources building out 12.5? Why not maximize 12.6 abilities? And wouldn't you want to build 12.6 with learnings from 12.5? Perhaps there's a good reason, but seems like that might contribute to the 2 steps forward, 2 steps back? It seems like a focusing on building the best FSD would give more linear improvements, rather than multiple separate models in tandem.
Criticism of FSD cautiousness is a stylistic thing, and humans tend to be way to risky and impatient, so I'm cool with the robots taking over with their style.
An option to dealing with irrational drivers - can be to replace them all with FSD Teslas - then it's both safer and faster. But it just needs to be safe and effective not always confident as such things will improve over time..
High density mapping is going to be the next step. That is the closest to human memory. You can drive around your neighborhood with more certainty than anybody who doesn’t live there. I don’t care if they are a race driver and they do it for an living. It’s hard to explain, but that is the type of information is priceless.
I am wondering: does the performance of FSD change significantly in more challenging weather conditions like fog, heavy rain or snow? As far as I remember it’s pretty much always sunny in your videos which I’m jealous about for once but I also am wondering if the camera system would struggle in situations with less visibility 👀
The 12:35 stop thinks there's something in the road. I really believe this is because everyone is insisting that mono vision is just as good as stereo vision. It isn't. Otherwise WE wouldn't have stereo vision. They're making the job way harder being stubborn about using true stereo, which is weird since teslas have multiple cameras a good distance apart AND Elon is a fan of 'if nature does it, we know it works', so I would have thought he'd insist on it.
I seriously wonder whether Tesla is randomising a bunch of parameters (which greentheonly found a while back) for each card each time you updated the software, to stop "supervisors" becoming too complacent? As FSD continues to improve, yet isn't good enough to drive unsupervised, complacency becomes an ever greater risk. Also it helps further diversify the car's actions so that they can get intervention data on a variety of different reactions to each type of situation.
Regarding your confusion about the disconnect between the driving and the visualization: Go back to the famous first v12 live demo. Elon explicitly mentions that, because you can’t really look into an AI’s brain, from v12 on, the visualization will be separate from the planning and driving. E.g. it is at least partially guesswork.
Thanks for your objectivity. I am tired of all the pumpers and their biased reviews because they bought the stock at its all time high and are hoping Elon will share their content to their financial benefit.
Looks like FSD is many years away from being ready. The ticket machine at the parking lot is a major obstacle. It also looks like it always has regressions and disengagements.
8:35 , pedestrians on the both sides of the road, one of them (closer one) is on the road at that moment on the visualization. (Car: “Time to give them time to cross from both sides”)
I think that since FSD is now based upon normal network learning that each new major release (12.X) will need to be regarded as a conscientious NOVICE driver. It's also possible that while the network can identify on a map the protected right lane at a sharp intersection, it wasn't certain how to handle it. It's extremely unlikely that ALL of the data that's being sent to the network for observation is from good drivers. I've been closely observing Teslas in traffic as a prospective purchaser and seen a significant number of Tesla drivers (and others) driving erratically and/or inconsiderately. It would only take a few of them making last minute improper maneuvers to make it uncertain what proper behavior was or how another car might behave. Only interventions flagged by drivers like you will cause FSD behavior to get force-biased in the correct manner. Instead of accepting bad drivers as behavior as inevitable and just dealing with them, for at least one year you need to flag each occurance of observed bad driving for Tesla.
Really surprised Elon said that they weren't getting enough interventions, but its not the first time he's hyped up/sugarcoated something like this. In my area I have to intervene 2-3 times within a mile for 12.4, less for safety and more for embarrassment since the car does something dumb. Hopefully this gets better but I likely won't stay subscribed until the tech gets more confident in future updates
At 9:50, I have that same issue all the time, instead of taking the right to turn right, it goes straight then tries to turn… has never worked in any version
It would be cool to see what would happen if you put only FSDs on the road in a limited area for testing, that were on continuous taxi mode. Also I wonder if it would be possible to in the future quick swap on the fly between a less and more assertive mode.
19:06 You've definitely highlighted the difference between 12.4 and 12.3, that 12.4 is letter-of-the-law and 12.3 is more like a free spirit doing whatever it can get away with. I'm not sure 12.4 is wrong. I wonder if 12.3 had some accidents or tickets?
12:32 Here the car is confused. It thinks, since that car appears to be parked on a sidewalk, which is rare, that it is actually in the road and trying to merge and drive forward. So, the Tesla is yielding. (Zero danger, just being very cautious.)
