9 ms·

> Consider these stats : the last three versions of the program — each 420,000 lines long-had just one error each. What exactly do they mean by this? If each o

by dpedu 2y ago

> Consider these stats : the last three versions of the program — each 420,000 lines long-had just one error each. What exactly do they mean by this? If each of the 3 versions had exactly one bug, isn't this just a weird way of saying the first 2 fixes either didn't work or introduced a new bug?

daemonologist 2y ago
Or maybe between one version and the next they only found one bug (there may have been bugs in the first version which weren't fixed until the third or later) - this seems more plausible to me since it's... rather difficult to count bugs until after you know about them. Of course now the greatness of the feat depends on how much testing there was between versions, but given that it was the shuttle there was probably a lot.
- sqeaky 2y ago
  And then we find a new category of bug, consider how we ran millions of different programs for many billions of CPU hours on all of our x86 CPUs before we learned about Spectre and meltdown.
- swiftcoder 2y ago
  Not just testing - line-by-line code review of the entire system by a panel of experts. Outside of aerospace/defence/nuclear this style of review is not very common.
  - Symbiote 2y ago
    I don't know about now, but software verification would also be used in consumer electronics. Fixing a bug in 10,000 washing machine control boards is very expensive when it entails sending a technician to every house to replace the circuit board.
    - swiftcoder 2y ago
      Yeah, I guess for devices that can't receive OTA updates that makes sense. Though I fear that segment of the market is rapidly shrinking - televisions have ubiquitous software update capabilities now, and even washing machines are increasingly internet connected. We didn't apply anywhere near that kind of quality control to smartphones or VR headsets. Once users are trained to install OTA updates to fix issues, most of the impetus for extreme quality control outside of the bootloader->OTA update path is gone
CodeWriter23 2y ago
Regressions are not implied by that statement. A bug doesn’t exist in the human realm until a human observes it.
- senorrib 2y ago
  Oh, the Schrodinger's Bug.
  - deleted 2y ago
    [deleted]
  - p-e-w 2y ago
    Schrödinger's Hubris, actually. The claim that a codebase of 420k lines contains "only one error" is of course absurd, and the members of this forum would laugh anyone out of the room who made such a claim about any other project, pointing out how they cannot possibly know, actual logical contradictions in the claims as described by GP, or just plain ridiculing it without further elaboration. But since the code in question cannot meaningfully be tested by the public, and people have been indoctrinated to believe the myth that aerospace engineers are an entirely different species that doesn't make mistakes in the sense that the rest of the workforce does, the hubris is accepted until disproven, which it probably won't be for various practical reasons. Nevermind that the Space Shuttle was a death trap that killed two crews, and that the associated investigations (especially for Challenger) revealed numerous serious issues in quality and safety management. People will continue to just nod their head when they hear nonsense like this, because that's what they have seen others do, and so Schrödinger's Hubris can live on.
    - avs733 2y ago
      maybe just labeling it journalistic license would be simpler and more accurate. I doubt any of the people who actually write the code would stand by when that claim was made and not clarify it to 'we only found one bug'.
- deleted 2y ago
  [deleted]
akira2501 2y ago
Known bug. The SRR (software readiness review) process happened after development but prior to certification for launch. Most of the bugs were found here and were found to have existed in the code since the beginning of the program. These were overwhelmingly low severity discrepancy reports. If I recall correctly, there was a time when they were finding lots of bugs through SRR, so the main development team started their own "continuous review" designed to catch bugs before going to SRR. This made the SRR people angry because they were finding fewer bugs and felt the development team was focusing on competition over bug numbers rather than the code itself.
- ChrisMarshallNY 2y ago
  > This made the SRR people angry because they were finding fewer bugs and felt the development team was focusing on competition over bug numbers rather than the code itself. This reminds me of the Quality culture, at my last job, which was a famous Japanese optical corporation. It was deliberately set up, so there was an adversarial relationship between QA, and Development, with QA holding the aces. As a Development manager, it used to drive me nuts (Ed. Well, it wasn’t much of a “drive.” More like a short putt). It did result in very high-Quality software, but at the cost of agility. It reflected their hardware development methodology, which regularly resulted in stunningly high-Quality kit, but I think caused a lot of issues with the usability and effectiveness of their software.
  - giancarlostoro 2y ago
    Reasons why I might not think twice when buying anything built in Japan. I know they care about quality. I rather it be built right, than quickly. I wish we held quality to higher standards in the software industry.
    - tmcdos 2y ago
      Japans cultivate this through their entire culture - starting from young children. We, the Westerns, are already at least 2 decades behind, sometimes even 4-5 decades ...

by dpedu 2y ago

daemonologist 2y ago

Or maybe between one version and the next they only found one bug (there may have been bugs in the first version which weren't fixed until the third or later) - this seems more plausible to me since it's... rather difficult to count bugs until after you know about them. Of course now the greatness of the feat depends on how much testing there was between versions, but given that it was the shuttle there was probably a lot.

sqeaky 2y ago
And then we find a new category of bug, consider how we ran millions of different programs for many billions of CPU hours on all of our x86 CPUs before we learned about Spectre and meltdown.
swiftcoder 2y ago
Not just testing - line-by-line code review of the entire system by a panel of experts. Outside of aerospace/defence/nuclear this style of review is not very common.
- Symbiote 2y ago
  I don't know about now, but software verification would also be used in consumer electronics. Fixing a bug in 10,000 washing machine control boards is very expensive when it entails sending a technician to every house to replace the circuit board.
  - swiftcoder 2y ago
    Yeah, I guess for devices that can't receive OTA updates that makes sense. Though I fear that segment of the market is rapidly shrinking - televisions have ubiquitous software update capabilities now, and even washing machines are increasingly internet connected. We didn't apply anywhere near that kind of quality control to smartphones or VR headsets. Once users are trained to install OTA updates to fix issues, most of the impetus for extreme quality control outside of the bootloader->OTA update path is gone

CodeWriter23 2y ago

Regressions are not implied by that statement. A bug doesn’t exist in the human realm until a human observes it.

senorrib 2y ago
Oh, the Schrodinger's Bug.
- deleted 2y ago
  [deleted]
- p-e-w 2y ago
  Schrödinger's Hubris, actually. The claim that a codebase of 420k lines contains "only one error" is of course absurd, and the members of this forum would laugh anyone out of the room who made such a claim about any other project, pointing out how they cannot possibly know, actual logical contradictions in the claims as described by GP, or just plain ridiculing it without further elaboration. But since the code in question cannot meaningfully be tested by the public, and people have been indoctrinated to believe the myth that aerospace engineers are an entirely different species that doesn't make mistakes in the sense that the rest of the workforce does, the hubris is accepted until disproven, which it probably won't be for various practical reasons. Nevermind that the Space Shuttle was a death trap that killed two crews, and that the associated investigations (especially for Challenger) revealed numerous serious issues in quality and safety management. People will continue to just nod their head when they hear nonsense like this, because that's what they have seen others do, and so Schrödinger's Hubris can live on.
  - avs733 2y ago
    maybe just labeling it journalistic license would be simpler and more accurate. I doubt any of the people who actually write the code would stand by when that claim was made and not clarify it to 'we only found one bug'.
deleted 2y ago
[deleted]

akira2501 2y ago

Known bug. The SRR (software readiness review) process happened after development but prior to certification for launch. Most of the bugs were found here and were found to have existed in the code since the beginning of the program. These were overwhelmingly low severity discrepancy reports. If I recall correctly, there was a time when they were finding lots of bugs through SRR, so the main development team started their own "continuous review" designed to catch bugs before going to SRR. This made the SRR people angry because they were finding fewer bugs and felt the development team was focusing on competition over bug numbers rather than the code itself.

ChrisMarshallNY 2y ago
> This made the SRR people angry because they were finding fewer bugs and felt the development team was focusing on competition over bug numbers rather than the code itself. This reminds me of the Quality culture, at my last job, which was a famous Japanese optical corporation. It was deliberately set up, so there was an adversarial relationship between QA, and Development, with QA holding the aces. As a Development manager, it used to drive me nuts (Ed. Well, it wasn’t much of a “drive.” More like a short putt). It did result in very high-Quality software, but at the cost of agility. It reflected their hardware development methodology, which regularly resulted in stunningly high-Quality kit, but I think caused a lot of issues with the usability and effectiveness of their software.
- giancarlostoro 2y ago
  Reasons why I might not think twice when buying anything built in Japan. I know they care about quality. I rather it be built right, than quickly. I wish we held quality to higher standards in the software industry.
  - tmcdos 2y ago
    Japans cultivate this through their entire culture - starting from young children. We, the Westerns, are already at least 2 decades behind, sometimes even 4-5 decades ...