6 ms·

You didn't design the aircraft that you fly on, which are incredibly computerized. Also the drugs that your doctor prescribes. Etc. You put your faith in myria

You didn't design the aircraft that you fly on, which are incredibly computerized. Also the drugs that your doctor prescribes. Etc. You put your faith in myriad bureaucracies and trust networks that you have no hope of understanding. Driverless cars are just one more added to the list.

ethanbond 5y ago
Neither aircraft nor drug development norms look anything like the norms of software development. I actually find a substantial difference in my trust level between a traditional car company that's building self-driving capabilities (e.g. Mercedes) than a company that fashions itself as a software company that happens to build cars (e.g. Tesla) for exactly the reason of the norms and behaviors that each org would encourage.
- woah 5y ago
  Didn't aircraft development norms lead to several planes plunging out of the sky with zero mechanical issues, due to software failure?
  - gouggoug 5y ago
    It's actually _failure to follow aircraft development norms_ that lead to the crash of multiple 737-Max
- dogman144 5y ago
  Generally agree but also knew people who worked on Boeing QA/SWE and the internal picture sounds dicy. I'm a bit disconcerted flying after learning about it.
  - skylanh 5y ago
    Boeing's issues were written on the wall about 2 decades ago when it moved it's hierarchy from an engineer-centric one to a Chicago accountant-centric one. That alone wouldn't have been an issue, but they had an immediate brain drain as half their HQ staff didn't follow the relocation. Imagine a thousand employees, half your HQ staff, that leave. There are several possible reasons: - they're older -> more corporate knowledge and knew the "magic recipes" - skill based mobility -> they're so good they can go anywhere, and Chicago isn't Seattle - etc https://hbr.org/2001/10/inside-boeings-big-move https://hbr.org/2001/10/inside-boeings-big-move This was widely thought on ... what was I browsing back then? Reddit? as a fore-warner of bad stuff to come The fore-warning came to fruition when the issues at various Boeing plants started developing over the next ten years (insider videos talking with an employee about being on drugs during crucial assembly work and shift mates being similarly high, "get it through your station" attitudes) and the Dreamliner issues that were hand-waved away (no training, 1-hour tablet self directed training, certification delays, and eventually crashes).
    - HeyLaughingBoy 5y ago
      > what was I browsing back then? Reddit? I remember it being discussed on SlashDot back then.
  - foolinaround 5y ago
    generally, you dont want to know how the sausage is being made... its pretty much all industries, some worse than the others.
ratmice 5y ago
Aircraft is a vastly different problem at a much smaller scale. Where did our ground traffic controllers go?
slg 5y ago
Furthermore, at this stage the driverless cars are unlikely to be creating demand. These are replacing human drivers. Most of us have no hand in verifying the safety of other drivers on the road. I happen to believe it is bonkers that we generally test drivers once at the age of 16 and somehow that earns a person driving privileges for at least 50 years before being retested if not their entire life.
- kylecordes 5y ago
  Imagine if cars were invented today. The bar would be much higher, probably halfway to how pilot licensing works. (Of course this had some substantial downsides.)
  - njarboe 5y ago
    If cars were invented today, they would be too dangerous to be allowed to exist. Almost nothing has been invented since 1970 that can cause any harm. Safety first and all that.
    - notreallyserio 5y ago
      They'd have to be built with cow-catcher like devices to push all the horseshit out of their way while traveling down the road. It'd be a different kind of awful.
    - tomjakubowski 5y ago
      Personal computers cause all kinds of harm (in addition to the benefits).
      - njarboe 5y ago
        Not much physical harm and very little liability. I think computers and software was the one place people could make what they wanted and sell it without much worry of getting sued out of existence and that is why there was so much advancement. Software licenses are pretty amazing with the "as is" clause that seems to hold up.
monadmoproblems 5y ago
Presumably, you choose whether to ride that airplane or take those drugs. You probably don't get to choose when someone else's driverless car runs you over.
- helloworld11 5y ago
  Nor do you get to choose when someone's airplane possibly crashes into your street from above.
  - gouggoug 5y ago
    There were 9 airplane crashes, worldwide, in 2021[1]. It's estimated 31,720 people died in car crashes, in the US, in 2021[2]. [1] https://en.wikipedia.org/wiki/Category:Aviation_accidents_and_incidents_in_2021 https://en.wikipedia.org/wiki/Category:Aviation_accidents_an... [2] https://www.transportation.gov/briefing-room/nhtsa-estimates-traffic-fatalities-continued-rise-record-pace-first-nine-months-2021 https://www.transportation.gov/briefing-room/nhtsa-estimates...
    - helloworld11 5y ago
      True, but you miss my point in referring to this. You still don't get to choose when someone else's mistake might harm you. Also, we as the public in general routinely trust planes with millions of risky flights despite not knowing anything about how well or safely they work, or how they work at all in detail. We accept that accidents sometimes happen nonetheless because that's how it goes with human technology that we've come to depend on. The important thing is that precautions are taken as much as reasonably possible. The same standard can be applied to driverless cars without getting fearful of them simply because of their newness.
      - gouggoug 5y ago
        But driverless cars do not go through the same requirements as aviation. Before an airplane goes into circulation, it has gone through extremely rigorous testing. An airplane also always has a pilot. The same can't be said for driverless cars. They don't go through extreme rigorous testing, they don't have a driver, and they're controlled by software that is most likely a far cry from aviation standards.
xyzzy21 5y ago
Except safety has only ever been improved for either aircraft or drugs by people dying from exactly this kind of failure. Apparently there is ZERO learning or forethought about risks that can be anticipated or suggested from other technologies!
akhmatova 5y ago
Driverless cars are just one more added to the list It's a qualitatively different change than any of the above (driven by a messianic view that AI is here and now and about to change everything), plus an instance of the https://en.wikipedia.org/wiki/Boiling_frog https://en.wikipedia.org/wiki/Boiling_frog apologue.
Veserv 5y ago
The difference is that airplane software standards are by far the most rigorous in the world and so rigorous that most software developers are shocked at the level of rigor employed where as driverless cars have exactly zero software standards with respect to their safety or fitness for purpose. Comparing the processes in play is a outrageously fundamental category error; the systems have standards that are like 6 orders of magnitude apart. As a example, just look at the 737 MAX, a airplane viewed as a literal deathtrap and a complete indictment of airplane software development, which had 2 crashes in 400,000 flights. At a average flight distance of ~500 miles, the 737 MAX, a airplane literally multiple orders of magnitude worse than any other and a complete failure of the approval process, only resulted in 1 fatality per 100 million person-miles. That is safer than the driving average of 1.1 fatalities per 100 million person-miles. What is viewed as by far the most atrocious outcome in the airplane sector, a system literally hundreds or thousands of times more dangerous than the average, is above average in the car sector; that is how far the gap is. And this is ignoring driverless car software which is being developed to even lower standards than regular car software and which has no actual independent quality requirements. In contrast, far more lives are at risk with respect to driverless cars, so they should be developed to standards probably around 1000x better than airplane software which puts them around 0.0001% of the way there to a acceptable level of quality even if the software was already as reliable as regular car software.
- jklinger410 5y ago
  > The difference is that airplane software standards are by far the most rigorous in the world and so rigorous that most software developers are shocked at the level of rigor employed... You mean the standards set by the FAA, which regularly defers to the private industry to regulate itself, due to complexity that it can't afford to keep up with? Those standards?
  - olyjohn 5y ago
    Flying is still the safest way to travel. So I'm not sure what you're complaining about with the FAA.
jjav 5y ago
> Driverless cars are just one more added to the list. As others have pointed out, aircraft certification (while it has its problems) is very rigorous. Driverless car certification, up to date, is basically nothing. But the other aspect is that aircraft autopilots are solving a far easier problem. Developing a car autopilot for busy streets is many orders of magnitude more difficult than an airplane (or boat) autopilot.

ethanbond 5y ago

Neither aircraft nor drug development norms look anything like the norms of software development. I actually find a substantial difference in my trust level between a traditional car company that's building self-driving capabilities (e.g. Mercedes) than a company that fashions itself as a software company that happens to build cars (e.g. Tesla) for exactly the reason of the norms and behaviors that each org would encourage.

woah 5y ago
Didn't aircraft development norms lead to several planes plunging out of the sky with zero mechanical issues, due to software failure?
- gouggoug 5y ago
  It's actually _failure to follow aircraft development norms_ that lead to the crash of multiple 737-Max
dogman144 5y ago
Generally agree but also knew people who worked on Boeing QA/SWE and the internal picture sounds dicy. I'm a bit disconcerted flying after learning about it.
- skylanh 5y ago
  Boeing's issues were written on the wall about 2 decades ago when it moved it's hierarchy from an engineer-centric one to a Chicago accountant-centric one. That alone wouldn't have been an issue, but they had an immediate brain drain as half their HQ staff didn't follow the relocation. Imagine a thousand employees, half your HQ staff, that leave. There are several possible reasons: - they're older -> more corporate knowledge and knew the "magic recipes" - skill based mobility -> they're so good they can go anywhere, and Chicago isn't Seattle - etc https://hbr.org/2001/10/inside-boeings-big-move https://hbr.org/2001/10/inside-boeings-big-move This was widely thought on ... what was I browsing back then? Reddit? as a fore-warner of bad stuff to come The fore-warning came to fruition when the issues at various Boeing plants started developing over the next ten years (insider videos talking with an employee about being on drugs during crucial assembly work and shift mates being similarly high, "get it through your station" attitudes) and the Dreamliner issues that were hand-waved away (no training, 1-hour tablet self directed training, certification delays, and eventually crashes).
  - HeyLaughingBoy 5y ago
    > what was I browsing back then? Reddit? I remember it being discussed on SlashDot back then.
- foolinaround 5y ago
  generally, you dont want to know how the sausage is being made... its pretty much all industries, some worse than the others.

ratmice 5y ago

Aircraft is a vastly different problem at a much smaller scale. Where did our ground traffic controllers go?

slg 5y ago

Furthermore, at this stage the driverless cars are unlikely to be creating demand. These are replacing human drivers. Most of us have no hand in verifying the safety of other drivers on the road. I happen to believe it is bonkers that we generally test drivers once at the age of 16 and somehow that earns a person driving privileges for at least 50 years before being retested if not their entire life.

kylecordes 5y ago
Imagine if cars were invented today. The bar would be much higher, probably halfway to how pilot licensing works. (Of course this had some substantial downsides.)
- njarboe 5y ago
  If cars were invented today, they would be too dangerous to be allowed to exist. Almost nothing has been invented since 1970 that can cause any harm. Safety first and all that.
  - notreallyserio 5y ago
    They'd have to be built with cow-catcher like devices to push all the horseshit out of their way while traveling down the road. It'd be a different kind of awful.
  - tomjakubowski 5y ago
    Personal computers cause all kinds of harm (in addition to the benefits).
    - njarboe 5y ago
      Not much physical harm and very little liability. I think computers and software was the one place people could make what they wanted and sell it without much worry of getting sued out of existence and that is why there was so much advancement. Software licenses are pretty amazing with the "as is" clause that seems to hold up.

monadmoproblems 5y ago

Presumably, you choose whether to ride that airplane or take those drugs. You probably don't get to choose when someone else's driverless car runs you over.

helloworld11 5y ago
Nor do you get to choose when someone's airplane possibly crashes into your street from above.
- gouggoug 5y ago
  There were 9 airplane crashes, worldwide, in 2021[1]. It's estimated 31,720 people died in car crashes, in the US, in 2021[2]. [1] https://en.wikipedia.org/wiki/Category:Aviation_accidents_and_incidents_in_2021 https://en.wikipedia.org/wiki/Category:Aviation_accidents_an... [2] https://www.transportation.gov/briefing-room/nhtsa-estimates-traffic-fatalities-continued-rise-record-pace-first-nine-months-2021 https://www.transportation.gov/briefing-room/nhtsa-estimates...
  - helloworld11 5y ago
    True, but you miss my point in referring to this. You still don't get to choose when someone else's mistake might harm you. Also, we as the public in general routinely trust planes with millions of risky flights despite not knowing anything about how well or safely they work, or how they work at all in detail. We accept that accidents sometimes happen nonetheless because that's how it goes with human technology that we've come to depend on. The important thing is that precautions are taken as much as reasonably possible. The same standard can be applied to driverless cars without getting fearful of them simply because of their newness.
    - gouggoug 5y ago
      But driverless cars do not go through the same requirements as aviation. Before an airplane goes into circulation, it has gone through extremely rigorous testing. An airplane also always has a pilot. The same can't be said for driverless cars. They don't go through extreme rigorous testing, they don't have a driver, and they're controlled by software that is most likely a far cry from aviation standards.

xyzzy21 5y ago

Except safety has only ever been improved for either aircraft or drugs by people dying from exactly this kind of failure. Apparently there is ZERO learning or forethought about risks that can be anticipated or suggested from other technologies!

akhmatova 5y ago

Driverless cars are just one more added to the list It's a qualitatively different change than any of the above (driven by a messianic view that AI is here and now and about to change everything), plus an instance of the https://en.wikipedia.org/wiki/Boiling_frog https://en.wikipedia.org/wiki/Boiling_frog apologue.

Veserv 5y ago

The difference is that airplane software standards are by far the most rigorous in the world and so rigorous that most software developers are shocked at the level of rigor employed where as driverless cars have exactly zero software standards with respect to their safety or fitness for purpose. Comparing the processes in play is a outrageously fundamental category error; the systems have standards that are like 6 orders of magnitude apart. As a example, just look at the 737 MAX, a airplane viewed as a literal deathtrap and a complete indictment of airplane software development, which had 2 crashes in 400,000 flights. At a average flight distance of ~500 miles, the 737 MAX, a airplane literally multiple orders of magnitude worse than any other and a complete failure of the approval process, only resulted in 1 fatality per 100 million person-miles. That is safer than the driving average of 1.1 fatalities per 100 million person-miles. What is viewed as by far the most atrocious outcome in the airplane sector, a system literally hundreds or thousands of times more dangerous than the average, is above average in the car sector; that is how far the gap is. And this is ignoring driverless car software which is being developed to even lower standards than regular car software and which has no actual independent quality requirements. In contrast, far more lives are at risk with respect to driverless cars, so they should be developed to standards probably around 1000x better than airplane software which puts them around 0.0001% of the way there to a acceptable level of quality even if the software was already as reliable as regular car software.

jklinger410 5y ago
> The difference is that airplane software standards are by far the most rigorous in the world and so rigorous that most software developers are shocked at the level of rigor employed... You mean the standards set by the FAA, which regularly defers to the private industry to regulate itself, due to complexity that it can't afford to keep up with? Those standards?
- olyjohn 5y ago
  Flying is still the safest way to travel. So I'm not sure what you're complaining about with the FAA.

jjav 5y ago

> Driverless cars are just one more added to the list. As others have pointed out, aircraft certification (while it has its problems) is very rigorous. Driverless car certification, up to date, is basically nothing. But the other aspect is that aircraft autopilots are solving a far easier problem. Developing a car autopilot for busy streets is many orders of magnitude more difficult than an airplane (or boat) autopilot.