4 ms·

The whole auto factory thing sounds completely misinformed to me. Just because a machine made it does not mean the output isn't checked in a multitude of ways.

The whole auto factory thing sounds completely misinformed to me. Just because a machine made it does not mean the output isn't checked in a multitude of ways. Any manufacturing process is subject to quality controls. Machines are maintained. Machine parts are swapped out long before they lead to out-of-tolerance work. Process outputs are statistically characterised, measured and monitored. Measurement equipment is recalibrated on a schedule. 3d printed parts are routinely X-rayed to check for internal residue. If something can go wrong, it sure as hell is checked. Maybe things that can't possibly fail are not checked, but the class of software that can't possibly fail is currently very small, no matter who or what generates it.

red_admiral 1y ago
On the auto factory side, the Toyota stuck gas pedal comes to mind, even if it can happen only under worst-case circumstances. But that's the (1 - 0.[lots of nines]) case. On the software side, the THERAC story is absolutely terrifying - you replace a physical interlock with a software-based one that _can't possibly go wrong_ and you get a killing machine that would probably count as unethical for executions of convicted terrorists.
- rightbyte 1y ago
  THERAC was terrible. And intermittent to for extra horror. I am a strong proponent of hardware level interlocks for way more mundane things than that. It helps a lot in debugging to narrow down the possible states of things.
jasonwatkinspdx 1y ago
Additionally production lines are all about doing the same thing over and over again, with fairly minimal variations. Software isn't like that. Because code is relatively easy to reuse, novelty tends to dominate new code written. Software developers are acting like integrators in at least partly novel contexts, not stamping out part number 100k of 200k that are identical. I do think modern ML has a place as a coding tool, but these factory like conceptions are very off the mark imo.
- insane_dreamer 1y ago
  But they are processing data 100k out of 200k in identical ways.
lambdasquirrel 1y ago
It’s been said that every jetliner that is in the air right now has a 1in hairline fracture in it somewhere? But that the plane is designed for the failure of any one or two parts? Software doesn’t exactly work the same way. You can make “AI” that operates more like [0,1] but at the end of the day the computer is still going to {0,1}.
- bumby 1y ago
  You can build redundancies into software similar to what you’re alluding to in airplanes. Some safety critical applications require it.
jcims 1y ago
A buddy of mine was a director in a metrology integration firm that did nothing but install lidar, structured light and other optical measurement solutions for auto assembly plants. He had a couple dozen people working full time on new model line build outs (every new model requires substantial refurb and redesign to the assembly line) and ongoing QA of vehicles as they were being manufactured at two local Honda plants. The precision they were looking for is pretty remarkable.
0xEF 1y ago
> Any manufacturing process is subject to quality controls. A few things on this illusion: * Any manufacturer will do everything in their power to avoid meeting anything but the barest minimums of standards due to budget concerns * QA workers are often pressured to let small things fly and cave easily because they simply do not get paid enough to care and know they won't win that fight unless their employer's product causes some major catastrophe that costs lives * Most common goods and infrastructure are built by the lowest bidder with the cheapest materials using underpaid labor, so as for "quality" we're already starting at the bottom. There is this notion that because things like ISO and QC standards exist, people follow them. The enforcement of quality is weak and the reach of any enforcing bodies is extremely short when pushed up against the wall by the teams of lawyers afforded to companies like Boeing or Stellantis. I see it too regularly at my job to not call out this idea that quality control is anything but smoke and mirrors, deployed with minimal effort and maximum reluctance. Hell, it's arguably the reason why I have a job since about 75% of the machines I walk in their doors to fix broke because they were improperly maintained, poorly implemented or sabotaged by an inept operator. It leaves me embittered, to be honest, because it doesn't have to be this way and the only reason why it is boils down to greed and mismanagement.
- bugtodiffer 1y ago
  > * Any manufacturer will do everything in their power to avoid meeting anything but the barest minimums of standards due to budget concerns in capitalistic countries, yes
  - sanderjd 1y ago
    Communism doesn't exactly have a great reputation for quality either. This is a human problem.
- idiotsecant 1y ago
  You don't work in a job with real QC. Do some pharma work and then get back to me.
  - 1y ago

red_admiral 1y ago

On the auto factory side, the Toyota stuck gas pedal comes to mind, even if it can happen only under worst-case circumstances. But that's the (1 - 0.[lots of nines]) case. On the software side, the THERAC story is absolutely terrifying - you replace a physical interlock with a software-based one that _can't possibly go wrong_ and you get a killing machine that would probably count as unethical for executions of convicted terrorists.

rightbyte 1y ago
THERAC was terrible. And intermittent to for extra horror. I am a strong proponent of hardware level interlocks for way more mundane things than that. It helps a lot in debugging to narrow down the possible states of things.

jasonwatkinspdx 1y ago

Additionally production lines are all about doing the same thing over and over again, with fairly minimal variations. Software isn't like that. Because code is relatively easy to reuse, novelty tends to dominate new code written. Software developers are acting like integrators in at least partly novel contexts, not stamping out part number 100k of 200k that are identical. I do think modern ML has a place as a coding tool, but these factory like conceptions are very off the mark imo.

insane_dreamer 1y ago
But they are processing data 100k out of 200k in identical ways.

lambdasquirrel 1y ago

It’s been said that every jetliner that is in the air right now has a 1in hairline fracture in it somewhere? But that the plane is designed for the failure of any one or two parts? Software doesn’t exactly work the same way. You can make “AI” that operates more like [0,1] but at the end of the day the computer is still going to {0,1}.

bumby 1y ago
You can build redundancies into software similar to what you’re alluding to in airplanes. Some safety critical applications require it.

jcims 1y ago

A buddy of mine was a director in a metrology integration firm that did nothing but install lidar, structured light and other optical measurement solutions for auto assembly plants. He had a couple dozen people working full time on new model line build outs (every new model requires substantial refurb and redesign to the assembly line) and ongoing QA of vehicles as they were being manufactured at two local Honda plants. The precision they were looking for is pretty remarkable.

0xEF 1y ago

> Any manufacturing process is subject to quality controls. A few things on this illusion: * Any manufacturer will do everything in their power to avoid meeting anything but the barest minimums of standards due to budget concerns * QA workers are often pressured to let small things fly and cave easily because they simply do not get paid enough to care and know they won't win that fight unless their employer's product causes some major catastrophe that costs lives * Most common goods and infrastructure are built by the lowest bidder with the cheapest materials using underpaid labor, so as for "quality" we're already starting at the bottom. There is this notion that because things like ISO and QC standards exist, people follow them. The enforcement of quality is weak and the reach of any enforcing bodies is extremely short when pushed up against the wall by the teams of lawyers afforded to companies like Boeing or Stellantis. I see it too regularly at my job to not call out this idea that quality control is anything but smoke and mirrors, deployed with minimal effort and maximum reluctance. Hell, it's arguably the reason why I have a job since about 75% of the machines I walk in their doors to fix broke because they were improperly maintained, poorly implemented or sabotaged by an inept operator. It leaves me embittered, to be honest, because it doesn't have to be this way and the only reason why it is boils down to greed and mismanagement.

bugtodiffer 1y ago
> * Any manufacturer will do everything in their power to avoid meeting anything but the barest minimums of standards due to budget concerns in capitalistic countries, yes
- sanderjd 1y ago
  Communism doesn't exactly have a great reputation for quality either. This is a human problem.
idiotsecant 1y ago
You don't work in a job with real QC. Do some pharma work and then get back to me.
- 1y ago