5 ms·
What I would find really interesting: If someone drove the same route with every version of FSD and counted the interventions. That would give a nice chart ove
by mg 3y ago
What I would find really interesting: If someone drove the same route with every version of FSD and counted the interventions.
That would give a nice chart over time from which one could see the progress.
Is there anything online similar to this?
Another way to look at it would be to look at the revenue that is generated via FSD. Buyers probably do some due dilligence what FSD offers. If the number of buyers goes up, that could be an indicator that FSD is getting better. But a quick Google search did not bring up much data about this published by Tesla. In their earnings report, there is a chart showing the cumulative miles driven with FSD. Not sure if one can draw conclusions from that. It seems very smoothed out and only goes back to 2021.
- xendipity 3y agoA YouTuber named DirtyTesla has done a series of drives over the years throughout his hometown where he's tracked the intervention and disengagement rates per mile. He often shows those numbers at the end of his videos.
- mg 3y agoIs it the same route every time? If not, they might take harder and harder routes as FSD gets better. Making it look like there is less progress than there really is.
- buro9 3y agoYou just know that Tesla would tune the software for this specific route, if the value of the comparison number became meaningful. Just like you cannot trust benchmarks, Goodhart's law will kick in https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law > Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes
- mensetmanusman 3y agoTheir model architecture don't allow for individual route tuning.
- tsimionescu 3y agoThey claim.
- mlrtime 3y agoSort of off-topic but I believe Openpilot allows this. You can record every route you take, copy it off to your PC and run it in a simulator. You can then tweak the model for this particular issue.
- krisoft 3y agoI think that is not true. They can just take data from the route, annotate it and put it into their training set. That is how you would do route tuning.
- numpad0 3y agoThere is no publicly known archive of Tesla software images, at all. Which, with the benefit of hindsight, do sound wrong.
- jjallen 3y agoWould have to drive many times over different times of day, days of week, seasons and weather conditions for them to be comparable.
- simion314 3y agoThat would depend on traffic, like it could perfectly work for you 100 times and the 101 time when a truck with some weird painting on it is present on the road and it will just hit it, or because of day of the year the shadows are different. Tesla keeps the true data secret and cherry pick the results they want. Would be inserting to data mine that data, like - are there roads where humans always (most of the time ) take over - how many near accidents are caused by FSD/autopilot - are there time of day variations - are there weather variations - are there issues after software updates - what are the good roads and bad roads see if we can identify what is special about them. Probably there could be more the public would like to know if the data would be made public, and IMO makes no sense to keep this public safety data secret.