6 ms·

Tesla FSD v12.3.3 still runs down child mannequins. Unfixed for 1.5 years [video]

25 points by Veserv 3y ago

Veserv 3y ago
Just to get ahead of the standard comments: 1. So-and-so Tesla Youtuber/Blog/News/Influencer said that they pressed the accelerator/manipulated results which invalidates the results and shows the test are untrustworthy. Let us examine that claim. Why would pressing the accelerator or other manipulations invalidate the results? Nobody would argue that turning on the windshield wipers or setting the cabin temperature would invalidate the results. What is special about the accelerator or other stated manipulations? The answer is that they could change the results which would invalidate a test. The actual claim the Tesla influencers are making is: "The only reason the car would run down a child in that situation is because the accelerator was pressed/car was manipulated." The video evidence for the last year has proven that is a bold-faced lie. Every claimed manipulation that can be caught on video has been thoroughly proven to be a outright fabrication. The car will run down children while FSD is enabled in simple circumstances when operated normally and has demonstrably done so for over a year. The videos the Tesla influencer community point to are earlier videos that did not have sufficient clarity yet to disprove their lies so they thought they would not get caught. The newer videos with increased clarity prove them to be unrepentant liars who will fabricate whatever is convenient as long as they think they can not get caught. If any of the Tesla mouthpieces show up in the comments here to regurgitate the newest "manipulation" I will show what I mean. 2. The clearly visible child facing the road stepped out into the road at the last second, which children never do, so the laws of physics make it impossible for a car, that did not show caution and slow down when clearly visible children are by the road, to stop. So, the car had no choice but to run down the child and then continue for a few seconds to roll over the child while it is on the ground. I think I will let that point speak for itself.
- nomel 3y ago
  I think it’s better to respond to comments than fabricate them.
- Ukv 3y ago
  > The videos the Tesla influencer community point to are earlier videos that did not have sufficient clarity yet to disprove their lies so they thought they would not get caught Haven't kept up with the drama in a while, but to my understanding the issue people took with the Dawn Project's initial video was not that it lacked clarity - but that it had the clarity to see that FSD was not engaged.[0] Could be explained, such as if the Dawn Project did the tests legitimately and then went back to capture footage for the ad, but I wouldn't assign so much malice ("unrepentant liars who will fabricate whatever is convenient as long as they think they can not get caught") to people pointing it out. [0]: https://electrek.co/2022/08/10/tesla-self-driving-smear-campaign-releases-test-fails-fsd-never-engaged/ https://electrek.co/2022/08/10/tesla-self-driving-smear-camp...
  - Veserv 3y ago
    Except that it was engaged as stated in the Safety Test Report [1] and as seen in the raw footage [2]. Rather than acknowledging that a incomplete investigation with conflicting information demands waiting for confirmatory evidence such as the raw footage which would have demonstrated their assumptions on the validity of the test were false, they chose to immediately begin declaring it a smear campaign based on false tests. They are not liars because they point out FSD was not engaged in portions of the ad. They are liars because they then straight up ignored evidence to the contrary and then assert, in confidence, that the tests were certainly fabricated and that FSD would never operate as demonstrated even though the video evidence that was forthcoming proved their assertions were completely and utterly wrong. As to being unrepentant liars. Is a legitimate test demonstrating a critical safety defect in a consumer product a smear campaign? No. Have they retracted their false accusation that correctly exposing a critical safety defect is a smear campaign since they jumped the gun on their accusations before they did even a token investigation? No. Then it is a lie and they are unrepentant. As to being willing to fabricate whatever is convenient. As soon as it became clear that FSD was enabled during those tests, the party line became that the only reason it happened was because the accelerator was pressed. This is false as can be seen in the current video. Then it became the only reason it happened was because the cones prevented a swerve. This is false as can be seen in the current video. Then it became the only reason it happened was because they steered into the child. This is false as can be seen in the current video. Every single one of these accusations was made because they thought the video could not disprove it and every single one has been proven to be false once the video setup was improved so it could disprove them. The fact that people, to this day, still deliberately point back to the earliest videos because they can not find any way to lie about the current videos without getting caught is bad faith argumentation. To then argue the Dawn Project is untrustworthy because of false accusations made by liars is true grade A bad faith argumentation that needs to be nipped in the bud. [1] https://dawnproject.com/wp-content/uploads/2022/08/The_Dawn_Project___Tesla_FSD_Test__8_.pdf https://dawnproject.com/wp-content/uploads/2022/08/The_Dawn_... [2] https://dawnproject.com/wp-content/uploads/2022/08/raw-footage.mp4 https://dawnproject.com/wp-content/uploads/2022/08/raw-foota...
jerzmacow 3y ago
I would sooner rally against SUV / Truck grills what hide 12 children under their exaggerated height.
- altairprime 3y ago
  Rally against both.
lolc 3y ago
I'd assume it's because Tesla can't afford more phantom breaking or they will lose people buying the FSD package. I wonder whether they will try to detect the test setup though and disable FSD, or pull a VW.
dzhiurgis 3y ago
Even autopilot stops for pedestrians. Are you saying Tesla added special exception when it detects children to just mull them?
- Veserv 3y ago
  No, it is just bad at detecting small humans. Given that Tesla only uses monocular vision and thus does not have binocular depth perception, it is possible that it determines that children are just far away adults, though that is just speculation on my part. It is Tesla that should affirmatively determine how their system acts in the presence of children before releasing it to their in-house testers, let alone before they give it to their customers so they can book revenue.
  - aeternum 3y ago
    At 0:15 you can actually see it pop up on the display and it looks like it gets the position right.
    - nomel 3y ago
      It definitely sees it in the later ones. I’m wondering if the confidence is too low to trigger the system, to prevent a phantom brake, maybe from the “child” moving in an impossible way (statically sliding across the ground). I would assume this is similar to the crash test setup though. It would be really interesting to get a deep dive into the cause.
deleted 3y ago
[deleted]
TheAlchemist 3y ago
It probably can't be 'fixed' - vision only + 'end-to-end NN'. I don't believe you can get it to be really reliable with that setup, at least not in this decade. It's a very good setup for awesome demo of the technology and vague promises in the future. Just like in 2016 (every Tesla will become a Robotaxi with an OTA update) and 2019 (we will have 1 million Robotaxis on the road in 2020). Another example - Tesla official account highlighting that FSD is 'taking pedestrian gestures into account': https://nitter.poast.org/Tesla/status/1776021631346835491#m https://nitter.poast.org/Tesla/status/1776021631346835491#m Is it a lie or not ? Even the usual influencers are saying it's not taking pedestrian gestures into account, based on their tests. But, the system is an NN - so it's impossible to prove it - just like it's impossible to prove that it does take it into account. Hell, it takes everything it sees on the camera into account !
aeternum 3y ago
What do you say to Tesla's claim that IIHS officially tests simulated crossing children and Tesla models successfully avoid the crash? Home-made tests are pretty easy to game, there could have been hundreds of attempts and these are the only that failed.
- Veserv 3y ago
  Those are tests of automatic emergency braking (AEB) where advanced driver assist systems (ADAS) such as Tesla FSD are explicitly not enabled. In actual ADAS tests, such as the recent IIHS Partial Automation Safeguard Ratings [1] Tesla FSD scores the lowest (6 Poor, 2 Acceptable) and Tesla Autopilot scores the second lowest (6 Poor, 1 Acceptable, 1 Good) out of 14 evaluated systems with the third lowest being the Genesis Highway Driving Assist 2 (6 Poor, 1 Marginal, 2 Good) and the highest being Lexus Teammate (1 Marginal, 3 Acceptable, 4 Good) with the average (to the extent that means anything) being around (3 Poor, 1 Marginal, 1 Acceptable, 3 Good). Or if we wish to look at the Euro NCAP Assisted Driving Gradings [2] Tesla scored 36/100 in assistance competence, the lowest score by far out of 26 evaluated systems. The second lowest is 52/100 and the average over all models excluding Tesla being ~67/100. [1] https://www.iihs.org/ratings/partial-automation-safeguards https://www.iihs.org/ratings/partial-automation-safeguards [2] https://www.euroncap.com/en/ratings-rewards/assisted-driving-gradings/ https://www.euroncap.com/en/ratings-rewards/assisted-driving...
  - aeternum 3y ago
    Directly from your Euro NCAP link on the assisted driving test: >The Model 3 responds to avoid a collision in all the ACC test scenarios and requires AEB interventions only in the most critical cut-in and cut-out tests. The driver is supported through the S-Bend, staying centred in the lane at all test speeds. The vehicle has an Active Blindspot system designed to prevent lane changing into adjacent vehicles. A lane-change assist function is also provided. In case of an unresponsive driver, the Tesla performs a controlled stop in lane. If the radar or camera are blocked the Model 3 provides a timely warning and prevents system activation. >The Tesla Model 3 excels in the level of Vehicle Assistance but fails to balance that high level of support with a similar level of Driver Engagement leading to possible overreliance. Euro NCAP tested the latest version of AutoPilot available at the time of testing. Tesla can improve the functionality of the system remotely by over-the-air software updates. The autopilot system itself got very high points, the 36% is from driver engagement features. Driver Engagement / 36%. Pretty disingenuous to use this driver engagement score which is measuring things like the driver monitoring system and not the autopilot performance.
ultrablack 3y ago
The owner of the Dawn project also owns an FSD competitor if I recall correctly.
- Veserv 3y ago
  That is false. They make operating systems and development tools for embedded applications such as aerospace and automotive. Some automotive companies may use the tools to make advanced driver assistance systems (ADAS) or automated driving systems (ADS), but that is as much a competitor with Tesla as GM's hammer supplier is a competitor with Tesla.
  - Ukv 3y ago
    > They make operating systems and development tools for embedded applications such as aerospace [...] Specifically a "Platform for Safe and Secure Automated Driving Systems". Could argue it's still not a FSD competitor in its own right, but it's more than just the implied "they make developement tools, which happen to be used by some automotive companies, which happen to make ADAS".

25 points by Veserv 3y ago

Just to get ahead of the standard comments: 1. So-and-so Tesla Youtuber/Blog/News/Influencer said that they pressed the accelerator/manipulated results which invalidates the results and shows the test are untrustworthy. Let us examine that claim. Why would pressing the accelerator or other manipulations invalidate the results? Nobody would argue that turning on the windshield wipers or setting the cabin temperature would invalidate the results. What is special about the accelerator or other stated manipulations? The answer is that they could change the results which would invalidate a test. The actual claim the Tesla influencers are making is: "The only reason the car would run down a child in that situation is because the accelerator was pressed/car was manipulated." The video evidence for the last year has proven that is a bold-faced lie. Every claimed manipulation that can be caught on video has been thoroughly proven to be a outright fabrication. The car will run down children while FSD is enabled in simple circumstances when operated normally and has demonstrably done so for over a year. The videos the Tesla influencer community point to are earlier videos that did not have sufficient clarity yet to disprove their lies so they thought they would not get caught. The newer videos with increased clarity prove them to be unrepentant liars who will fabricate whatever is convenient as long as they think they can not get caught. If any of the Tesla mouthpieces show up in the comments here to regurgitate the newest "manipulation" I will show what I mean. 2. The clearly visible child facing the road stepped out into the road at the last second, which children never do, so the laws of physics make it impossible for a car, that did not show caution and slow down when clearly visible children are by the road, to stop. So, the car had no choice but to run down the child and then continue for a few seconds to roll over the child while it is on the ground. I think I will let that point speak for itself.

nomel 3y ago
I think it’s better to respond to comments than fabricate them.
Ukv 3y ago
> The videos the Tesla influencer community point to are earlier videos that did not have sufficient clarity yet to disprove their lies so they thought they would not get caught Haven't kept up with the drama in a while, but to my understanding the issue people took with the Dawn Project's initial video was not that it lacked clarity - but that it had the clarity to see that FSD was not engaged.[0] Could be explained, such as if the Dawn Project did the tests legitimately and then went back to capture footage for the ad, but I wouldn't assign so much malice ("unrepentant liars who will fabricate whatever is convenient as long as they think they can not get caught") to people pointing it out. [0]: https://electrek.co/2022/08/10/tesla-self-driving-smear-campaign-releases-test-fails-fsd-never-engaged/ https://electrek.co/2022/08/10/tesla-self-driving-smear-camp...
- Veserv 3y ago
  Except that it was engaged as stated in the Safety Test Report [1] and as seen in the raw footage [2]. Rather than acknowledging that a incomplete investigation with conflicting information demands waiting for confirmatory evidence such as the raw footage which would have demonstrated their assumptions on the validity of the test were false, they chose to immediately begin declaring it a smear campaign based on false tests. They are not liars because they point out FSD was not engaged in portions of the ad. They are liars because they then straight up ignored evidence to the contrary and then assert, in confidence, that the tests were certainly fabricated and that FSD would never operate as demonstrated even though the video evidence that was forthcoming proved their assertions were completely and utterly wrong. As to being unrepentant liars. Is a legitimate test demonstrating a critical safety defect in a consumer product a smear campaign? No. Have they retracted their false accusation that correctly exposing a critical safety defect is a smear campaign since they jumped the gun on their accusations before they did even a token investigation? No. Then it is a lie and they are unrepentant. As to being willing to fabricate whatever is convenient. As soon as it became clear that FSD was enabled during those tests, the party line became that the only reason it happened was because the accelerator was pressed. This is false as can be seen in the current video. Then it became the only reason it happened was because the cones prevented a swerve. This is false as can be seen in the current video. Then it became the only reason it happened was because they steered into the child. This is false as can be seen in the current video. Every single one of these accusations was made because they thought the video could not disprove it and every single one has been proven to be false once the video setup was improved so it could disprove them. The fact that people, to this day, still deliberately point back to the earliest videos because they can not find any way to lie about the current videos without getting caught is bad faith argumentation. To then argue the Dawn Project is untrustworthy because of false accusations made by liars is true grade A bad faith argumentation that needs to be nipped in the bud. [1] https://dawnproject.com/wp-content/uploads/2022/08/The_Dawn_Project___Tesla_FSD_Test__8_.pdf https://dawnproject.com/wp-content/uploads/2022/08/The_Dawn_... [2] https://dawnproject.com/wp-content/uploads/2022/08/raw-footage.mp4 https://dawnproject.com/wp-content/uploads/2022/08/raw-foota...

jerzmacow 3y ago

I would sooner rally against SUV / Truck grills what hide 12 children under their exaggerated height.

altairprime 3y ago
Rally against both.

lolc 3y ago

I'd assume it's because Tesla can't afford more phantom breaking or they will lose people buying the FSD package. I wonder whether they will try to detect the test setup though and disable FSD, or pull a VW.

dzhiurgis 3y ago

Even autopilot stops for pedestrians. Are you saying Tesla added special exception when it detects children to just mull them?

Veserv 3y ago
No, it is just bad at detecting small humans. Given that Tesla only uses monocular vision and thus does not have binocular depth perception, it is possible that it determines that children are just far away adults, though that is just speculation on my part. It is Tesla that should affirmatively determine how their system acts in the presence of children before releasing it to their in-house testers, let alone before they give it to their customers so they can book revenue.
- aeternum 3y ago
  At 0:15 you can actually see it pop up on the display and it looks like it gets the position right.
  - nomel 3y ago
    It definitely sees it in the later ones. I’m wondering if the confidence is too low to trigger the system, to prevent a phantom brake, maybe from the “child” moving in an impossible way (statically sliding across the ground). I would assume this is similar to the crash test setup though. It would be really interesting to get a deep dive into the cause.

deleted 3y ago

[deleted]

TheAlchemist 3y ago

It probably can't be 'fixed' - vision only + 'end-to-end NN'. I don't believe you can get it to be really reliable with that setup, at least not in this decade. It's a very good setup for awesome demo of the technology and vague promises in the future. Just like in 2016 (every Tesla will become a Robotaxi with an OTA update) and 2019 (we will have 1 million Robotaxis on the road in 2020). Another example - Tesla official account highlighting that FSD is 'taking pedestrian gestures into account': https://nitter.poast.org/Tesla/status/1776021631346835491#m https://nitter.poast.org/Tesla/status/1776021631346835491#m Is it a lie or not ? Even the usual influencers are saying it's not taking pedestrian gestures into account, based on their tests. But, the system is an NN - so it's impossible to prove it - just like it's impossible to prove that it does take it into account. Hell, it takes everything it sees on the camera into account !

aeternum 3y ago

What do you say to Tesla's claim that IIHS officially tests simulated crossing children and Tesla models successfully avoid the crash? Home-made tests are pretty easy to game, there could have been hundreds of attempts and these are the only that failed.

Veserv 3y ago
Those are tests of automatic emergency braking (AEB) where advanced driver assist systems (ADAS) such as Tesla FSD are explicitly not enabled. In actual ADAS tests, such as the recent IIHS Partial Automation Safeguard Ratings [1] Tesla FSD scores the lowest (6 Poor, 2 Acceptable) and Tesla Autopilot scores the second lowest (6 Poor, 1 Acceptable, 1 Good) out of 14 evaluated systems with the third lowest being the Genesis Highway Driving Assist 2 (6 Poor, 1 Marginal, 2 Good) and the highest being Lexus Teammate (1 Marginal, 3 Acceptable, 4 Good) with the average (to the extent that means anything) being around (3 Poor, 1 Marginal, 1 Acceptable, 3 Good). Or if we wish to look at the Euro NCAP Assisted Driving Gradings [2] Tesla scored 36/100 in assistance competence, the lowest score by far out of 26 evaluated systems. The second lowest is 52/100 and the average over all models excluding Tesla being ~67/100. [1] https://www.iihs.org/ratings/partial-automation-safeguards https://www.iihs.org/ratings/partial-automation-safeguards [2] https://www.euroncap.com/en/ratings-rewards/assisted-driving-gradings/ https://www.euroncap.com/en/ratings-rewards/assisted-driving...
- aeternum 3y ago
  Directly from your Euro NCAP link on the assisted driving test: >The Model 3 responds to avoid a collision in all the ACC test scenarios and requires AEB interventions only in the most critical cut-in and cut-out tests. The driver is supported through the S-Bend, staying centred in the lane at all test speeds. The vehicle has an Active Blindspot system designed to prevent lane changing into adjacent vehicles. A lane-change assist function is also provided. In case of an unresponsive driver, the Tesla performs a controlled stop in lane. If the radar or camera are blocked the Model 3 provides a timely warning and prevents system activation. >The Tesla Model 3 excels in the level of Vehicle Assistance but fails to balance that high level of support with a similar level of Driver Engagement leading to possible overreliance. Euro NCAP tested the latest version of AutoPilot available at the time of testing. Tesla can improve the functionality of the system remotely by over-the-air software updates. The autopilot system itself got very high points, the 36% is from driver engagement features. Driver Engagement / 36%. Pretty disingenuous to use this driver engagement score which is measuring things like the driver monitoring system and not the autopilot performance.

ultrablack 3y ago

The owner of the Dawn project also owns an FSD competitor if I recall correctly.

Veserv 3y ago
That is false. They make operating systems and development tools for embedded applications such as aerospace and automotive. Some automotive companies may use the tools to make advanced driver assistance systems (ADAS) or automated driving systems (ADS), but that is as much a competitor with Tesla as GM's hammer supplier is a competitor with Tesla.
- Ukv 3y ago
  > They make operating systems and development tools for embedded applications such as aerospace [...] Specifically a "Platform for Safe and Secure Automated Driving Systems". Could argue it's still not a FSD competitor in its own right, but it's more than just the implied "they make developement tools, which happen to be used by some automotive companies, which happen to make ADAS".