5 ms·

Are you trying to us that less stringent controls on manufacturing software is in any way acceptable? Manufacturing process (including mfg software tools) are

Are you trying to us that less stringent controls on manufacturing software is in any way acceptable? Manufacturing process (including mfg software tools) are a huge potential source of product failure. I don't know about automotive QMS specifically, but I work in embedded software for safety critical systems. If an unreleased procedure or unreleased software is used to build the hardware, or unreleased software is run on the hardware, it's not even suitable for QA (let alone going to the field).

nordsieck 8y ago
>> Production line software is almost certainly handled separately from the software that runs their vehicles, and isn't "production" in the usual web sense of being customer facing. > Are you trying to us that less stringent controls on manufacturing software is in any way acceptable? Yes. It is simply uneconomical to develop all software in a no-bugs-allowed NASA style process. One of the classes of software that can easily be placed in a lower scrutiny bucket is industrial software where the the result of the industrial process will be separately checked by a quality control process. This is in dramatic contrast to safety critical software in cars, for example the battery management software or the fly-by-wire systems. Defects in these systems, in contract to industrial software, are much more difficult to discover by inspection, and failures are much more costly.
- finnthehuman 8y ago
  Yeah, in that case the controls are moved from the development process to the downstream verification. Sorry if I wasn't clear, I originally had some wording about how known unknowns are handled in my post, but it was getting unnecessarily long and felt like a diversion from the hypothetical I was responding to. I was focused more on documentation are repeatability of process than the quality of the tool themselves. >> I find it concerning that one person was able to push malicious code to 'production'. >Production line software is almost certainly handled separately [...] This sounds more like someone messed with their factory automation setup. For a production line, I would still expect a level of control over exactly what software was running and how it was configured, even if it was SOUP. edit: SOUP not SOAP. I need to stop posting in the morning.
- rjdagost 8y ago
  No one is saying "no bugs" NASA level code is required. Basic SQA must however be conducted- factory automation software is not something where you can settle for low quality as you seem to imply. Something tells me you've never worked in factory / production environments. There are critical safety requirements that must be checked, especially in environments where machinery is operating in proximity to workers. Not only that, but there are huge financial incentives in place to ensure high quality. A bug in an early step of a manufacturing process can cause massive amounts of rework or even scrap, leading to huge losses.
  - givinguflac 8y ago
    Chill out. Every system that relies on human input can be fooled and broken. How do we know this employee wasn’t the one responsible for loading new code into the machines once validated? Anyone looking to spend years infiltrating a place for a goal will do whatever it takes to circumvent security measures.
taneq 8y ago
Yes. It's not "manufacturing software" (if I'm right about it being the factory control system), it's more like glue code tying all of the different bits and bobs of the assembly line together. This kind of system is usually built in ladder logic or function block diagrams rather than as source code, is typically a whole lot of very simple bits side by side, and often needs to be tweaked in realtime during production. It's likely that quite a few people (definitely more than 2-3) have direct access to modify this sort of code and are probably monitoring and fine tuning it continually. It's not really comparable to your work where you're writing the OS or system code for a safety PLC or something, and you quite rightly have to maintain a very strict process where everything is fully documented and triple-checked so you can prove that it meets your target performance level. All of the stuff that does need that kind of precision is locked away into modules (like the aforementioned safety PLC) which just gets plugged together like Lego by most of the controls team. (All this is kind of moot, though, because it says that the culprit used a few different logins to make their mischief. Depending on the permissions set for those logins they could have bypassed pretty much any formal workflow, at least temporarily.)

nordsieck 8y ago

>> Production line software is almost certainly handled separately from the software that runs their vehicles, and isn't "production" in the usual web sense of being customer facing. > Are you trying to us that less stringent controls on manufacturing software is in any way acceptable? Yes. It is simply uneconomical to develop all software in a no-bugs-allowed NASA style process. One of the classes of software that can easily be placed in a lower scrutiny bucket is industrial software where the the result of the industrial process will be separately checked by a quality control process. This is in dramatic contrast to safety critical software in cars, for example the battery management software or the fly-by-wire systems. Defects in these systems, in contract to industrial software, are much more difficult to discover by inspection, and failures are much more costly.

finnthehuman 8y ago
Yeah, in that case the controls are moved from the development process to the downstream verification. Sorry if I wasn't clear, I originally had some wording about how known unknowns are handled in my post, but it was getting unnecessarily long and felt like a diversion from the hypothetical I was responding to. I was focused more on documentation are repeatability of process than the quality of the tool themselves. >> I find it concerning that one person was able to push malicious code to 'production'. >Production line software is almost certainly handled separately [...] This sounds more like someone messed with their factory automation setup. For a production line, I would still expect a level of control over exactly what software was running and how it was configured, even if it was SOUP. edit: SOUP not SOAP. I need to stop posting in the morning.
rjdagost 8y ago
No one is saying "no bugs" NASA level code is required. Basic SQA must however be conducted- factory automation software is not something where you can settle for low quality as you seem to imply. Something tells me you've never worked in factory / production environments. There are critical safety requirements that must be checked, especially in environments where machinery is operating in proximity to workers. Not only that, but there are huge financial incentives in place to ensure high quality. A bug in an early step of a manufacturing process can cause massive amounts of rework or even scrap, leading to huge losses.
- givinguflac 8y ago
  Chill out. Every system that relies on human input can be fooled and broken. How do we know this employee wasn’t the one responsible for loading new code into the machines once validated? Anyone looking to spend years infiltrating a place for a goal will do whatever it takes to circumvent security measures.

taneq 8y ago

Yes. It's not "manufacturing software" (if I'm right about it being the factory control system), it's more like glue code tying all of the different bits and bobs of the assembly line together. This kind of system is usually built in ladder logic or function block diagrams rather than as source code, is typically a whole lot of very simple bits side by side, and often needs to be tweaked in realtime during production. It's likely that quite a few people (definitely more than 2-3) have direct access to modify this sort of code and are probably monitoring and fine tuning it continually. It's not really comparable to your work where you're writing the OS or system code for a safety PLC or something, and you quite rightly have to maintain a very strict process where everything is fully documented and triple-checked so you can prove that it meets your target performance level. All of the stuff that does need that kind of precision is locked away into modules (like the aforementioned safety PLC) which just gets plugged together like Lego by most of the controls team. (All this is kind of moot, though, because it says that the culprit used a few different logins to make their mischief. Depending on the permissions set for those logins they could have bypassed pretty much any formal workflow, at least temporarily.)