6 ms·

> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski Any engineer with this attitude ne

by pdkl95 9y ago

> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski Any engineer with this attitude needs to learn the lesson of the Therac-25. The issues in the Ars article are very similar to section 4 "Causal Factors" of the report[1]. > To get to that better software faster we should deploy the first 1000 cars asap. Is that admitting that they do not have the "better software" and intend to deploy 1000 cars using "lesser software"? That's treading dangerously close to potential manslaughter charges if prove this willful contempt for safety to a court. [1] http://sunnyday.mit.edu/papers/therac.pdf http://sunnyday.mit.edu/papers/therac.pdf

iClaudiusX 9y ago
I get the feeling based on comments here that there is a severe lack of ethical and critical thinking among engineers and developers. I recognize that this is only a vocal minority but the constant mantra of "move fast and break things", where getting rich at any cost is seen as a virtue, has made me extremely disillusioned with this brand of startup culture. Doubly so when people are trading stock tips on how to profit from tragedy by supporting the worst actors in the field.
- KKKKkkkk1 9y ago
  "Move fast and break things" is the motto of Facebook. It means that Facebook engineers are encouraged to make user-facing changes without red tape. If you are claiming that some self-driving car company has "move fast and break things" as their motto, then you are being willfully deceptive.
  - PhasmaFelis 9y ago
    Have you not noticed that "move fast and break things" is essentially how the entire startup space functions? When a social media company does it, it's merely annoying. Do it with dangerous machines, and people die.
- JKCalhoun 9y ago
  I'm not sure the "self-driving car industry" can survive these possible (inevitable?) future scenarios: • Self-driving car kills child • Hackers send self-driving car on wild ride • Empty self-driving semi found in warehouse lot, GPS pirates make off with entire cargo
- archagon 9y ago
  The dark, unspoken sentiment behind such comments is: "the end justifies the means". These programmers treat lives as currency for their vision of the future.
wdr1 9y ago
Is it better for self driving cars to have a flawless record & low adoption, or to have a 100x improvement over human drivers & broad adoption? Creating the perfect self-driving car, with redundant systems, safety everything & so on will certainly help its safety records. But it will also drive up the cost. And put it out of reach for a lot of people. If the goal is to save lives, the bar self-driving cars should be held to is what humans do driving today, not perfection.
- discodave 9y ago
  From the article: > But zooming out from the specifics of Herzberg's crash, the more fundamental point is this: conventional car crashes killed 37,461 in the United States in 2016, which works out to 1.18 deaths per 100 million miles driven. Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed people at roughly 25 times the rate of a typical human-driven car in the United States. I don't think there's enough evidence to say that self-driving cars are as safe as humans.
  - jon_richards 9y ago
    To be fair, you have to keep in mind that deploying 1000 cars would quickly make self-driving cars safer than humans. Yes, you lose some life in the interim, but you accelerate the creation and adoption of something that saves much more life in the long run. But that is human experimentation, something we as a culture generally agree is abhorrent.
    - fourthark 9y ago
      This is assuming that the learning actually happens.
    - AlexandrB 9y ago
      > To be fair, you have to keep in mind that deploying 1000 cars would quickly make self-driving cars safer than humans. This seems unreasonably optimistic. First, this particular crash is an egregious counter-example. The car doesn't even seem to slow down when it first sees the pedestrian's foot. Nor does it try to swerve. This is basic stuff for a human driver, never mind more complex avoidance and risk mitigation a human driver can perform. Second, we've had years of training various AI content curation algorithms on social networks, videos, blogs, etc - and the most advanced AI and search company in the world still can't keep adult-oriented conspiracy videos off of Youtube Kids. And while you might counter that content is a human problem, driving is too! Dangerous driving situations happen at the periphery of traffic rules, where someone is doing something the drivers around him don't expect. I've seen people run solid red lights, drive the wrong way down a one-way street, pedestrians start crossing when their light turns red, etc. Short of creating special roads for autonomous cars only - how does an autonomous car deal with all of this successfully?
PinkMilkshake 9y ago
Agile methodology applied to safety-critical systems engineering, god help us.
- pjc50 9y ago
  "Move fast and break things. Also break people, but if they're homeless or ride bicycles nobody cares"
  - cmsj 9y ago
    To quote a friend... "disrupting traditional life expectancy models", or to quote Webshit Weekly... "A/B testing vehicular manslaughter"
deleted 9y ago
[deleted]
phlo 9y ago
> willful contempt for safety To play Kalanick's adversary, he might be arguing for more real-world data collection. Tesla famously equipped most of their cars with more sensors than were required at the time of delivery, using the data to drive development of the Autopilot function that was later added to the cars.
- PinguTS 9y ago
  Please, remember that the original technology of the so-called Autopilot was bought from MobilEye. Please remember, that MobilEye is a long-time player in this area and have deployed their technology first with BMW and later with Volvo way before it was used in Tesla. Tesla only activated features that were not ready for primetime. That was the reason why MobilEye backed off from the partnership with Tesla. Btw. all this of collecting sensor data and improving on this, is an original idea of MobilEye that they presented at conferences before the so-called Autopilot was available from Tesla.
IshKebab 9y ago
> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski He is clearly right about that. Human-driven cars are safety critical and already do fine without redundant brakes and steering. How many crashes are due to brake or steering failure? I'm guessing it's well under 10%. Most human crashes are due to bad driving, and for computers it will be the same. I mean, even this fatal crash probably could have been prevented with better software. It's not like the brakes failed. They just weren't applied. > To get to that better software faster we should deploy the first 1000 cars asap. This is where he is totally mad.
- davidgould 9y ago
  Human driven cars have redundant brakes. There is one brake assembly for each wheel. The hydraulic system is split into two separate circuits to guard against leaks. Additionally, cars are fitted with an "Emergency brake".

by pdkl95 9y ago

iClaudiusX 9y ago

I get the feeling based on comments here that there is a severe lack of ethical and critical thinking among engineers and developers. I recognize that this is only a vocal minority but the constant mantra of "move fast and break things", where getting rich at any cost is seen as a virtue, has made me extremely disillusioned with this brand of startup culture. Doubly so when people are trading stock tips on how to profit from tragedy by supporting the worst actors in the field.

KKKKkkkk1 9y ago
"Move fast and break things" is the motto of Facebook. It means that Facebook engineers are encouraged to make user-facing changes without red tape. If you are claiming that some self-driving car company has "move fast and break things" as their motto, then you are being willfully deceptive.
- PhasmaFelis 9y ago
  Have you not noticed that "move fast and break things" is essentially how the entire startup space functions? When a social media company does it, it's merely annoying. Do it with dangerous machines, and people die.
JKCalhoun 9y ago
I'm not sure the "self-driving car industry" can survive these possible (inevitable?) future scenarios: • Self-driving car kills child • Hackers send self-driving car on wild ride • Empty self-driving semi found in warehouse lot, GPS pirates make off with entire cargo
archagon 9y ago
The dark, unspoken sentiment behind such comments is: "the end justifies the means". These programmers treat lives as currency for their vision of the future.

wdr1 9y ago

Is it better for self driving cars to have a flawless record & low adoption, or to have a 100x improvement over human drivers & broad adoption? Creating the perfect self-driving car, with redundant systems, safety everything & so on will certainly help its safety records. But it will also drive up the cost. And put it out of reach for a lot of people. If the goal is to save lives, the bar self-driving cars should be held to is what humans do driving today, not perfection.

discodave 9y ago
From the article: > But zooming out from the specifics of Herzberg's crash, the more fundamental point is this: conventional car crashes killed 37,461 in the United States in 2016, which works out to 1.18 deaths per 100 million miles driven. Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed people at roughly 25 times the rate of a typical human-driven car in the United States. I don't think there's enough evidence to say that self-driving cars are as safe as humans.
- jon_richards 9y ago
  To be fair, you have to keep in mind that deploying 1000 cars would quickly make self-driving cars safer than humans. Yes, you lose some life in the interim, but you accelerate the creation and adoption of something that saves much more life in the long run. But that is human experimentation, something we as a culture generally agree is abhorrent.
  - fourthark 9y ago
    This is assuming that the learning actually happens.
  - AlexandrB 9y ago
    > To be fair, you have to keep in mind that deploying 1000 cars would quickly make self-driving cars safer than humans. This seems unreasonably optimistic. First, this particular crash is an egregious counter-example. The car doesn't even seem to slow down when it first sees the pedestrian's foot. Nor does it try to swerve. This is basic stuff for a human driver, never mind more complex avoidance and risk mitigation a human driver can perform. Second, we've had years of training various AI content curation algorithms on social networks, videos, blogs, etc - and the most advanced AI and search company in the world still can't keep adult-oriented conspiracy videos off of Youtube Kids. And while you might counter that content is a human problem, driving is too! Dangerous driving situations happen at the periphery of traffic rules, where someone is doing something the drivers around him don't expect. I've seen people run solid red lights, drive the wrong way down a one-way street, pedestrians start crossing when their light turns red, etc. Short of creating special roads for autonomous cars only - how does an autonomous car deal with all of this successfully?

PinkMilkshake 9y ago

Agile methodology applied to safety-critical systems engineering, god help us.

pjc50 9y ago
"Move fast and break things. Also break people, but if they're homeless or ride bicycles nobody cares"
- cmsj 9y ago
  To quote a friend... "disrupting traditional life expectancy models", or to quote Webshit Weekly... "A/B testing vehicular manslaughter"

deleted 9y ago

[deleted]

phlo 9y ago

> willful contempt for safety To play Kalanick's adversary, he might be arguing for more real-world data collection. Tesla famously equipped most of their cars with more sensors than were required at the time of delivery, using the data to drive development of the Autopilot function that was later added to the cars.

PinguTS 9y ago
Please, remember that the original technology of the so-called Autopilot was bought from MobilEye. Please remember, that MobilEye is a long-time player in this area and have deployed their technology first with BMW and later with Volvo way before it was used in Tesla. Tesla only activated features that were not ready for primetime. That was the reason why MobilEye backed off from the partnership with Tesla. Btw. all this of collecting sensor data and improving on this, is an original idea of MobilEye that they presented at conferences before the so-called Autopilot was available from Tesla.

IshKebab 9y ago

> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski He is clearly right about that. Human-driven cars are safety critical and already do fine without redundant brakes and steering. How many crashes are due to brake or steering failure? I'm guessing it's well under 10%. Most human crashes are due to bad driving, and for computers it will be the same. I mean, even this fatal crash probably could have been prevented with better software. It's not like the brakes failed. They just weren't applied. > To get to that better software faster we should deploy the first 1000 cars asap. This is where he is totally mad.

davidgould 9y ago
Human driven cars have redundant brakes. There is one brake assembly for each wheel. The hydraulic system is split into two separate circuits to guard against leaks. Additionally, cars are fitted with an "Emergency brake".