4 ms·

There are in fact more levels - the definition of "level 3" is hand-wavy enough that it may capture everything, but there surely is at least a level 4, illustra

There are in fact more levels - the definition of "level 3" is hand-wavy enough that it may capture everything, but there surely is at least a level 4, illustrated by the recent Hawaii missile alert: the software worked "as designed", but if it's easy to misuse it, can you actually argue that it was "correct"? Somewhat different (and ironically, illustrated in the article itself) are the very-reasonable-but-wrong assumptions about the environment: Take the case of Microsoft Windows & SimCity - who fixed the bug? Microsoft did... for all intents and purposes, even if in a technical/theoretical sense the Windows memory manager was "correct" - in a very practical sense, it was buggy and needed to be fixed. This occurs in practice far more frequently than we are willing to admit.

crdoconnor 9y ago
These are actually the bugs I run up against the most often once #1 and #2 are dealt with: program functions as intended, but what was intended was wrong. Well, that and "bug in dependency". Formal methods are nice and everything but it's a massive waste of time if those are your main problems because you're doubling down on solving the wrong problem.
- virgilp 9y ago
  I wouldn't say "a massive waste of time" - except if done at the wrong level. I like Chris Newcorde's view [1] that TLA+ is "a pseudocode for writing design documentation". At that level, it's completely appropriate to use formal methods to check that you specified/investigated/considered all the edge cases. However, I find that using it to "prove program correctness" is indeed a waste of time, since what is most often wrong is your assumptions, not necessarily your implementation. [1]http://hpts.ws/papers/2011/sessions_2011/Debugging.pdf http://hpts.ws/papers/2011/sessions_2011/Debugging.pdf
neo2006 9y ago
UI design is a different disciplines than software design, the Hawaii missile alert issue was a UI issue the software presenting the UI worked as intended. The issue is that we consider UI design as software and we perform UI testing (which is not even testing the UI but the software underneath) and we say everything is good we are ready for prod.
- virgilp 9y ago
  It's really just an easy-to-understand symptom of the general problem: if you write/design software that is easy to misuse, that's really a faulty software. If you want a different example - I think Tony Hoare's "billion-dolar mistake" statement is an admission of the fact that 'null' can really be interpreted as a language design bug. Technically speaking, a language design can't be buggy if it's coherent - it just "is". But in practical, everyday terms - the fact that null leads to lots of problems, very often, could/should be considered a "language design bug". That was kinda' my point. The article claims that "bug" is not just "coding errors", and I agree - and furthermore, I say that "design considerations are still bugs by that definition"
  - neo2006 9y ago
    I don't agree with the author, IMO bugs are coding error, the crash or the bad behaviour coming out of it is just a symptom. Not having any symptom does not mean that I do not have any defect/disease, with the appropriate set of tests the doctor could find that I have a disease but with no symptoms. To come back to the auth example on simcity, the software had a coding error and the proof of it is that when the symptoms were visible (with the windows 3.1 malloc version) we needed a change in the code to fix the error. The other only category that do not fell under coding error for me is design/architecture error and Hawaii issue fell into that category and it's not bug because it do not prevent us from use the software as intendant it just make the user more error prone or not able to add feature easily or wait too long or worst, no symptom at all until you want to modify expand the software.
    - kulu2002 9y ago
      NFRs or Non Functional Requirements are root cause of bugs in most of the cases. Functional requirements are easy to meet. It is NFR where our assumptions come into picture. As an example consider selecting improper data type to represent something. When this datatype fails to accommodate given quantity/ overflows creates the bug. But this will not happen immediately. Even such issues can easily escape rigorous testing. Check 911 outage case for that matter [1]. I think it was more than just a coding error - A bad design decision. [1] https://www.theverge.com/2014/10/20/7014705/coding-error-911-fcc-washington https://www.theverge.com/2014/10/20/7014705/coding-error-911...

crdoconnor 9y ago

These are actually the bugs I run up against the most often once #1 and #2 are dealt with: program functions as intended, but what was intended was wrong. Well, that and "bug in dependency". Formal methods are nice and everything but it's a massive waste of time if those are your main problems because you're doubling down on solving the wrong problem.

virgilp 9y ago
I wouldn't say "a massive waste of time" - except if done at the wrong level. I like Chris Newcorde's view [1] that TLA+ is "a pseudocode for writing design documentation". At that level, it's completely appropriate to use formal methods to check that you specified/investigated/considered all the edge cases. However, I find that using it to "prove program correctness" is indeed a waste of time, since what is most often wrong is your assumptions, not necessarily your implementation. [1]http://hpts.ws/papers/2011/sessions_2011/Debugging.pdf http://hpts.ws/papers/2011/sessions_2011/Debugging.pdf

neo2006 9y ago

UI design is a different disciplines than software design, the Hawaii missile alert issue was a UI issue the software presenting the UI worked as intended. The issue is that we consider UI design as software and we perform UI testing (which is not even testing the UI but the software underneath) and we say everything is good we are ready for prod.

virgilp 9y ago
It's really just an easy-to-understand symptom of the general problem: if you write/design software that is easy to misuse, that's really a faulty software. If you want a different example - I think Tony Hoare's "billion-dolar mistake" statement is an admission of the fact that 'null' can really be interpreted as a language design bug. Technically speaking, a language design can't be buggy if it's coherent - it just "is". But in practical, everyday terms - the fact that null leads to lots of problems, very often, could/should be considered a "language design bug". That was kinda' my point. The article claims that "bug" is not just "coding errors", and I agree - and furthermore, I say that "design considerations are still bugs by that definition"
- neo2006 9y ago
  I don't agree with the author, IMO bugs are coding error, the crash or the bad behaviour coming out of it is just a symptom. Not having any symptom does not mean that I do not have any defect/disease, with the appropriate set of tests the doctor could find that I have a disease but with no symptoms. To come back to the auth example on simcity, the software had a coding error and the proof of it is that when the symptoms were visible (with the windows 3.1 malloc version) we needed a change in the code to fix the error. The other only category that do not fell under coding error for me is design/architecture error and Hawaii issue fell into that category and it's not bug because it do not prevent us from use the software as intendant it just make the user more error prone or not able to add feature easily or wait too long or worst, no symptom at all until you want to modify expand the software.
  - kulu2002 9y ago
    NFRs or Non Functional Requirements are root cause of bugs in most of the cases. Functional requirements are easy to meet. It is NFR where our assumptions come into picture. As an example consider selecting improper data type to represent something. When this datatype fails to accommodate given quantity/ overflows creates the bug. But this will not happen immediately. Even such issues can easily escape rigorous testing. Check 911 outage case for that matter [1]. I think it was more than just a coding error - A bad design decision. [1] https://www.theverge.com/2014/10/20/7014705/coding-error-911-fcc-washington https://www.theverge.com/2014/10/20/7014705/coding-error-911...