5 ms·

Those who do not read the history of Therac-25 apparently are doomed to repeat it. If the hardware can be destructive under the control of software then you mu

• kwantam 13y ago
  Huh. I learned a different lesson from Therac-25: "never write mission-critical software." :) For those who haven't read it, http://sunnyday.mit.edu/papers/therac.pdf http://sunnyday.mit.edu/papers/therac.pdf
  • mikeash 13y ago
    There are three kinds of programmers: 1. Those who read about Therac-25 and decide never to work on any systems where bugs could threaten lives (your chosen course, and mine). 2. Those who read about Therac-25, get frightened, and work very carefully on such systems. 3. Those who read about Therac-25 and think, "Those idiots! Good thing I'd never make a mistake like that." The prevalence of (3) in all professions and walks of life is kind of frightening.
    • hga 13y ago
      Your 3 is a bit more complicated, I think: If you dig into the details of why its programmers and their managers killed people, it's very easy to say "I'd never be that irresponsible", but still screw up with much more subtle bugs.
      • mikeash 13y ago
        They may have been (2) and just weren't sufficiently careful. The inclusion of (3) isn't meant as a criticism of the Therac team, but rather humanity in general.
        • logicallee 13y ago
          "My comments were not aimed at the Therac team, but at humanity in general." :)
          • mikeash 13y ago
            Don't be offended, I'm insulting all of you equally!
    • awj 13y ago
      You should walk through a building construction project sometime. Your "all professions and walks of life" part is ... disturbingly accurate.
      • mikeash 13y ago
        It no doubt applies to doctors too, which is even more disturbing.
        • masklinn 13y ago
          Not as disturbing as finding this attitude in hospital construction projects.
  • superuser2 13y ago
    I learned not to write life-critical software. There is no bringing back the dead. Corporate mission-critical software (unless said corporation is power, weapons, transportation, or healthcare related) can usually only lose money.
    • orting 13y ago
      The upside to writing software for healthcare is that your code might help people recover faster or in some cases survive where they otherwise wouldnt. But it can be stressful to write software that has a very direct impact on peoples lifes. While I was rolling out an updated version of an inhouse application a serious bug was discovered. The application was used as part of chemotherapy follow-up/planning and on some occasions it gave the wrong answer for a test. I discovered that the bug was old and that wrong answers had been given for a long period. Personally I was relieved that I had not caused the bug, but from a medical perspective it would have been better if the bug was new and had only affected very few patients. My takeaway from that experience is that care should be taken when writing critical software, but also that we will make mistakes and it is very important to always focus on how we can make fewer mistakes instead of pointing fingers.
  • drivebyacct2 13y ago
    It was a bit hard to read from a technical aspect and then I got to the "Events" section. I think I'm going to put myself in the "I choose not to work on things that can kill people" column.
• noir_lord 13y ago
  Researching the Therac-25 was on the list of topics one of my university lecturers assigned. Also on that list (from memory). * AT&T's switch cascade failure * Mars Climate Orbiter * Ariane 5 He also took us through some bugs in code he'd written when he was doing massive telecomm's systems. I came away with a few lessons. * If it can't fail, test it twice. * If it's designed so it can't possibly fail, test it three times. * All code is buggy.
  • masklinn 13y ago
    > If it's designed so it can't possibly fail When faced with this kind of requirements, I always lift my eyes right above my screen, where it is written: > The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair. And for emphasis, next to it is a poster of a nuclear island cut-out.
    • mpyne 13y ago
      The stereotypical example of something that "couldn't possibly go wrong" for our electrical division was the interconnecting wiring embedded in the case that simply connected point A to point B. Sure enough, we once had a motor controller break where the problem was interconnecting wiring...
• scotty79 13y ago
  Stories like this make me want for all comercial software to have source code published before anyone is allowed to earn even a dollar form it's sale, support or related activities.
  • csense 13y ago
    If companies are required to publish their source code before selling their software, what keeps a large portion of their market from simply pirating it instead?
    • scotty79 13y ago
      What keeps them now? If you publish software it will be pirated. Hiding source code doesn't help with that. Even if obligation to publish source code were to harm companies (which might even not be the case) the benefit it would bring to the consumers and whole ecosystem is more important. Companies don't need special protection. The most important advantage of a company is that it can die without killing all people who work there and destroying all its assets. This allows companies to take risks no sane person would take if he were to be responsible with all his money (both savings and potential debt).
• gnaffle 13y ago
  In some cases (like avionics) you can't realistically implement hardware interlocks because you don't have many clear-set criteria to go on (eg. a control surface deflection might be OK at one airspeed, but cause serious aircraft damage at another).