3 ms·

Will you please elaborate?

WJW 3y ago
Aviation in particular has a very strong culture around (government mandated) checklists and post-crash investigations. This has both pros and cons. The pros is that every airline learns from the mistakes made by every other airline and over time the system becomes really quite safe indeed. The cons are that it is quite expensive and time consuming. Imagine if every software company was obliged by law to: - Every single release has to have been signed off by someone who got their "software release engineer" certification at the software equivalent of the FAA. - This engineer is required by law to not sign off unless every box on a 534 item checklist has been manually verified. - Any time an unplanned downtime happens at any company, a government team comes in to investigate the root cause and add points nr 535 through 567 to the checklist to make sure it never happens again. If such a system was mandated for software companies, most of the common bugs would very rapidly become a thing of the past. Development velocity would also fall through the floor though, and most startups would probably die overnight. Only companies that could support the overhead of such a heavyweight process would be viable, and the barrier to entry would massively increase.
- bluGill 3y ago
  I wish someone would create that 500 line checklist. I've seem attempts, but they tend to be either not actionable (is the software high quality - meaningless), or of metrics that are just gamed (is test code coverage > 80%?)
  - MaxBarraclough 3y ago
    Not quite what you're asking for, but the Joint Strike Fighter C++ Coding Standards document is freely available. [0] It's 141 pages. It's specific to the complex and unsafe C++ language though, rather than addressing broader software development methodology. [0] [PDF] https://www.stroustrup.com/JSF-AV-rules.pdf https://www.stroustrup.com/JSF-AV-rules.pdf
  - mdaniel 3y ago
    > or of metrics that are just gamed (is test code coverage > 80%?) The rebuttal to your implied Goodhart's Law <https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law> that was offered by my manager was "tension metrics" <https://en.wikiversity.org/wiki/IT_Service_Management/Continual_Service_Improvement#:~:text=tension%20metrics https://en.wikiversity.org/wiki/IT_Service_Management/Contin...> If I understand his theory correctly, in your case there would be a competing metric to the "test coverage" one that said for any changeset, a test cannot itself change by more than 20% in the same changeset as non-test code. So you can change the code such that it still passes the existing tests, or you can change the test to adapt to new requirements, but you cannot rewrite the tests to match your newly changed code I'm acutely aware this is a terrible example, the devil's in the details, and (in my experience) each company's metrics are designed to drive down their own organizational risk <https://en.wikipedia.org/wiki/Conway%27s_law https://en.wikipedia.org/wiki/Conway%27s_law>, combined with "you're always fighting the last war" :-D
    - marcosdumay 3y ago
      Hum, now you are proposing a process checklist that can't ever be completely checked out, by design. The entire thing is terrible from principle. You won't find a good example, because that's not how you use a process checklist.
- JumpinJack_Cash 3y ago
  > > Aviation in particular has a very strong culture around (government mandated) checklists and post-crash investigations That's the reason why aviation can only shine when it becomes a private means of transportation, and I don't mean 70mm private jets but, 150k light helicopters. When a critical mass is hit then accidents will become no more traumatic to the collective psyche than car accidents, the lighter the aircraft the better because it would seem exactly like a car crash as opposed to leaving a huge burning hole into the ground

WJW 3y ago

Aviation in particular has a very strong culture around (government mandated) checklists and post-crash investigations. This has both pros and cons. The pros is that every airline learns from the mistakes made by every other airline and over time the system becomes really quite safe indeed. The cons are that it is quite expensive and time consuming. Imagine if every software company was obliged by law to: - Every single release has to have been signed off by someone who got their "software release engineer" certification at the software equivalent of the FAA. - This engineer is required by law to not sign off unless every box on a 534 item checklist has been manually verified. - Any time an unplanned downtime happens at any company, a government team comes in to investigate the root cause and add points nr 535 through 567 to the checklist to make sure it never happens again. If such a system was mandated for software companies, most of the common bugs would very rapidly become a thing of the past. Development velocity would also fall through the floor though, and most startups would probably die overnight. Only companies that could support the overhead of such a heavyweight process would be viable, and the barrier to entry would massively increase.

bluGill 3y ago
I wish someone would create that 500 line checklist. I've seem attempts, but they tend to be either not actionable (is the software high quality - meaningless), or of metrics that are just gamed (is test code coverage > 80%?)
- MaxBarraclough 3y ago
  Not quite what you're asking for, but the Joint Strike Fighter C++ Coding Standards document is freely available. [0] It's 141 pages. It's specific to the complex and unsafe C++ language though, rather than addressing broader software development methodology. [0] [PDF] https://www.stroustrup.com/JSF-AV-rules.pdf https://www.stroustrup.com/JSF-AV-rules.pdf
- mdaniel 3y ago
  > or of metrics that are just gamed (is test code coverage > 80%?) The rebuttal to your implied Goodhart's Law <https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law> that was offered by my manager was "tension metrics" <https://en.wikiversity.org/wiki/IT_Service_Management/Continual_Service_Improvement#:~:text=tension%20metrics https://en.wikiversity.org/wiki/IT_Service_Management/Contin...> If I understand his theory correctly, in your case there would be a competing metric to the "test coverage" one that said for any changeset, a test cannot itself change by more than 20% in the same changeset as non-test code. So you can change the code such that it still passes the existing tests, or you can change the test to adapt to new requirements, but you cannot rewrite the tests to match your newly changed code I'm acutely aware this is a terrible example, the devil's in the details, and (in my experience) each company's metrics are designed to drive down their own organizational risk <https://en.wikipedia.org/wiki/Conway%27s_law https://en.wikipedia.org/wiki/Conway%27s_law>, combined with "you're always fighting the last war" :-D
  - marcosdumay 3y ago
    Hum, now you are proposing a process checklist that can't ever be completely checked out, by design. The entire thing is terrible from principle. You won't find a good example, because that's not how you use a process checklist.
JumpinJack_Cash 3y ago
> > Aviation in particular has a very strong culture around (government mandated) checklists and post-crash investigations That's the reason why aviation can only shine when it becomes a private means of transportation, and I don't mean 70mm private jets but, 150k light helicopters. When a critical mass is hit then accidents will become no more traumatic to the collective psyche than car accidents, the lighter the aircraft the better because it would seem exactly like a car crash as opposed to leaving a huge burning hole into the ground