3 ms·

The thing that really grinds my gears about all of this as a roboticist, is that as a community we had a pretty good culture of taking safety seriously. In 2014

The thing that really grinds my gears about all of this as a roboticist, is that as a community we had a pretty good culture of taking safety seriously. In 2014, the idea of unleashing onto the public admittedly beta-quality autonomous car software, on a platform that eschews industry best practices was unfathomable. Then came Autopilot and later FSD, and Tesla threw the concept of safety was thrown out the window. An autonomous Tesla decapitated a man in 2016, and Tesla continued unabated! What actionable steps did Tesla take as a result of that incident to make sure it never happens again. Even today in 2022 you can find videos of Teslas on public roads just completely failing to detect and colliding with stationary objects. The culture at Tesla seems to not care this is happening, as it continues to happen. It used to be that when your robot did a single thing wrong or caused the most minor of injuries, let alone it killed someone, then you engineered the hell out of it and implemented safety protocols to make sure that failure mode would never happen again. Not at Tesla, apparently. I get the utilitarian argument that maybe self driving cars could reduce traffic accidents in the future by being better than humans on average. But safety doesn’t happen unless you really work at it, with intention. Using your customers to beta test your 2 ton robots on public roads is not a safety-focused mindset. Tesla is not a company that values safety first when it comes to their autonomous car project.

belter 5y ago
Why all the euphemisms and diplomatic language? There is one word for it: Criminal.
kodah 5y ago
I haven't worked in robotics, but I've worked in automation for quite some time (digital and real-world). Two examples I can think of: - SCADA always had hot, hardware backups and failure detection. To the extent that I thought this was dogma. - Physical automation (like car construction equipment and robotics, pick and place machines, etc) have safety zones that humans aren't allowed in or near. Same thing for Amazon warehouse robotics. I'm not sure how we leapt to, "beta test cars on the street" from there.
- TOMDM 5y ago
  I think the issue is partly that people don't see cars as large industrial robots because we're just so used to them. Everyone who has ever written code for a robot knows you should treat every pinch point as a finger blender, but the cars look nothing like them. Tesla's branding may make that excusable for the layman, but for the employees at Tesla, that branding is incredibly irresponsible
heavyset_go 5y ago
It seems to me like the lives of countless people are seen as edge-cases that don't matter to Tesla's big picture of profits and growth.
randoglando 5y ago
> I get the utilitarian argument that maybe self driving cars could reduce traffic accidents in the future by being better than humans on average. This keeps being touted (by Elon and others), but is a long way away: https://twitter.com/Tweetermeyer/status/1488673180403191808 https://twitter.com/Tweetermeyer/status/1488673180403191808 Tesla uses misleading stats (like not excluding highway miles, which are inherently safer) to try and make it seem like autopilot is any good.
- ModernMech 5y ago
  Yeah, and the reason Elon is reaching for aggregate statistics is because he can't argue that his platform is fundamentally sound. I mean, the primary directive of any driverless car is: don't run into anything. That's it, that's all it has to do. Yet Teslas routinely run into stationary objects, and not because those objects are somehow particularly pathological and would generally fool most anyone. No, they are specifically confounding to the way Tesla does things, in a way that practitioners in the field aware of and know how to solve, but which Elon Musk stubbornly refuses to recognize. I keep harping on it but the decapitation incident in 2016 is just so illustrative of what kind of culture Tesla is fostering. Here's a situation where reckless engineering choices by Tesla literally caused a customer to lose his head. The solution which would fix this problem would be to add more orthogonal sensor modalities, so that any blind spot in one sensor can be compensated for by another (e.g. LIDAR will not see a glass wall, but SONOR will). But what did Tesla do? They actually doubled down! They removed existing orthogonal sensors and they made everything worse! Now basically the entire thing is driven by vision, and it's still predictably exhibiting the same failure mode of running into broad and stationary objects as it did 5 years ago (which by the way was approximately the amount of time Elon Musk predicted it would take for them to get to level 5 autonomous driving. Instead they've bumbled around for half a decade and failed to make any significant progress).

belter 5y ago

Why all the euphemisms and diplomatic language? There is one word for it: Criminal.

kodah 5y ago

I haven't worked in robotics, but I've worked in automation for quite some time (digital and real-world). Two examples I can think of: - SCADA always had hot, hardware backups and failure detection. To the extent that I thought this was dogma. - Physical automation (like car construction equipment and robotics, pick and place machines, etc) have safety zones that humans aren't allowed in or near. Same thing for Amazon warehouse robotics. I'm not sure how we leapt to, "beta test cars on the street" from there.

TOMDM 5y ago
I think the issue is partly that people don't see cars as large industrial robots because we're just so used to them. Everyone who has ever written code for a robot knows you should treat every pinch point as a finger blender, but the cars look nothing like them. Tesla's branding may make that excusable for the layman, but for the employees at Tesla, that branding is incredibly irresponsible

heavyset_go 5y ago

It seems to me like the lives of countless people are seen as edge-cases that don't matter to Tesla's big picture of profits and growth.

randoglando 5y ago

> I get the utilitarian argument that maybe self driving cars could reduce traffic accidents in the future by being better than humans on average. This keeps being touted (by Elon and others), but is a long way away: https://twitter.com/Tweetermeyer/status/1488673180403191808 https://twitter.com/Tweetermeyer/status/1488673180403191808 Tesla uses misleading stats (like not excluding highway miles, which are inherently safer) to try and make it seem like autopilot is any good.

ModernMech 5y ago
Yeah, and the reason Elon is reaching for aggregate statistics is because he can't argue that his platform is fundamentally sound. I mean, the primary directive of any driverless car is: don't run into anything. That's it, that's all it has to do. Yet Teslas routinely run into stationary objects, and not because those objects are somehow particularly pathological and would generally fool most anyone. No, they are specifically confounding to the way Tesla does things, in a way that practitioners in the field aware of and know how to solve, but which Elon Musk stubbornly refuses to recognize. I keep harping on it but the decapitation incident in 2016 is just so illustrative of what kind of culture Tesla is fostering. Here's a situation where reckless engineering choices by Tesla literally caused a customer to lose his head. The solution which would fix this problem would be to add more orthogonal sensor modalities, so that any blind spot in one sensor can be compensated for by another (e.g. LIDAR will not see a glass wall, but SONOR will). But what did Tesla do? They actually doubled down! They removed existing orthogonal sensors and they made everything worse! Now basically the entire thing is driven by vision, and it's still predictably exhibiting the same failure mode of running into broad and stationary objects as it did 5 years ago (which by the way was approximately the amount of time Elon Musk predicted it would take for them to get to level 5 autonomous driving. Instead they've bumbled around for half a decade and failed to make any significant progress).