7 ms·

Not sure if I agree that no one really knows how to do it, but yeah, software is not as disciplined as something like civil engineering. And for most things, it

Not sure if I agree that no one really knows how to do it, but yeah, software is not as disciplined as something like civil engineering. And for most things, it doesn’t need to be. For others, their teams operate much more slowly and with more testing and QA - think of a basic web app developer versus someone writing software for Boeing aircraft. The jobs are completely different. For most web app developers, screwing up doesn’t endanger anyone’s life.

mruts 8y ago
I mean we can kind of make software, but it always has bugs and even the biggest companies who have (supposedly) the best developers consistently make products that don’t work very well at all. Software qualiry looks really bad compared to, say, bridges or cars or airplanes.
- ethbro 8y ago
  Respectfully, I call bullshit on this. We know some very good ways to build reliable software. It's just that most industries can't / don't want to pay the costs associated with doing so. The don't-know / don't-want distinction is functionally irrelevant, but I have a minor tick about it because asserting the former is often used to avoid admitting to the later. Legacy IBM and ATT, for all their cluster%_$^!s, were amazing at engineering reliable systems (software and hardware). Maybe that's not practical nowadays, outside of heavily regulated industries like military / aerospace, because markets have greater numbers of competitors. But we do know how. An accurate and modern truth would probably be "We don't know how to quickly build reliable software at low cost."
  - mruts 8y ago
    I mean, clearly, if we really wanted to, we could build all of our software with Coq and have it be formally verified. But we don’t. The Curry Howard isomorphism proves that code and math are equivalent and since we can prove a bunch of stuff with math, we should also be able to prove that our software systems are big free. And so you’re right, maybe my statement is a little bit of a simplification, but not by much. The amount of man hours it would take to prove that the Linux kernel had no bugs according to some specification would require an absolutely prohibitive amount of time. As such formal verification is so prohibitive that only the most trivial systems could ever be proved. This is pretty similar to saying that we don’t know how to build bug-free code. We need a paradigm change for things to get better, and I’m not sure it will happen or if it’s even possible.
    - henrikeh 8y ago
      It is not about tools. It is about culture and process. Look at something like how SQLite for a real example of living up to the requirements of aviation software: 100 % branch coverage, extensive testing, multiple independent platforms in the test suite etc etc. You don’t need anything fancy – but you do need to understand your requirements and thoroughly simulate and verify the intended behavior.
    - AnimalMuppet 8y ago
      I think of it in slightly different terms. Yes, we could build software that reliably. (Perhaps not totally bug free, but, say, with 1/100th of the bugs it currently has.) But that takes a lot more time and effort (say, 100 times as much, as a rough number). That means, because it would take 100 times as much effort to produce each piece of software, we'd only have 1/100th as much software. Would that be a better world? Sure, I hate it when my word processor crashes. On the other hand, I really like having a word processor at all. It has value for me, even in a somewhat buggy state, more value than a typewriter does. Would I give up having a word processor in order to have, say, a much less buggy OS? I don't think I would. I hate the bugs. But the perfect may be the enemy of the "good enough to get real work done".
      - ethbro 8y ago
        I look at that as the efficiency distinction. It goes without saying that if we had more time, there would be fewer bugs. But there are also things (tooling, automation, code development processes) that can decrease the amount of bugs without ballooning development time. Things that decrease bugs_created per unit_of_development_time. Automated test suites with good coverage, or KASAN [1, mentioned on here recently]. Code scanners, languages that prohibit unsafe development operations or guard them more closely, and automated fuzzing too. [1] https://www.kernel.org/doc/html/v4.14/dev-tools/kasan.html https://www.kernel.org/doc/html/v4.14/dev-tools/kasan.html
- Klathmon 8y ago
  Do you know that first hand, or is this a case that you can see the cracks in the software industry because you are part of it? I worked as a mechanic years ago, and I feel the same way about cars as you (and I) do about software. Everything is duct taped together and it's a miracle any of it works as well as it does. Safety regulations and tests are gamed to all hell, reliability almost always seems to come from an evolutionary method of "just keep using what works and change what doesn't" with hastily tacked on explanations as to why it works, and marketing is so divorced from reality that people assume significantly more is happening in the car than really is. Now I don't know anything about civil engineering or aricraft, but I have a hunch it's not as well put together as it may seem at first glance. I still think that software is behind those other fields, and I'm honestly still on the fence if regulations would significantly help or would just slow things down, but it's absolutely the "wild west" in software.
  - mruts 8y ago
    I'm not a "real" engineer, but many of my cousins are (civil engineers) and everything about a bridge, say, is modeled and proven ahead of time. I work as a developer and everyday I feel like this isn't the right the way to do it. I'm not sure what the right way is, but being a software developer I probably have less faith in it then the general population. Something just feels wrong about it, how we do things..
    - User23 8y ago
      Imagine building a bridge without a specification. It would be criminally irresponsible. It has been known for decades now that proper specification is absolutely necessary to produce correct software. This is practically a tautology, because there isn't a formally meaningful way to define "correct" besides "satisfies some specified behaviors." Good unit testing does this in an ad hoc way, because the tests serve as a (usually incomplete) specification of the desired behavior. That said we must remember that testing can never prove the absence of bugs, only their presence. The only practicable ways to prove the absence of bugs in nontrivial programs are automatic formal verification or constructive proof using formally defined semantics. Of course for most commercial software correctness is so irrelevant that it's left undefined. All that matters is pleasantness, that is to say does it generally please the user, perhaps by attracting customers or investors.
- fma 8y ago
  Look at the Equifax hack. Vulnerability was known but no one cared to apply the patch. In engineering, you chose 2 amongst being better, faster or cheaper. It's clear in IT we always choose faster and cheaper. No one went to jail for the Equifax hack except that one low level manager for insider trading.
- ummonk 8y ago
  Airplanes have software. It works very well without bugs. Because that is what is demanded in the aviation industry.

mruts 8y ago

I mean we can kind of make software, but it always has bugs and even the biggest companies who have (supposedly) the best developers consistently make products that don’t work very well at all. Software qualiry looks really bad compared to, say, bridges or cars or airplanes.

ethbro 8y ago
Respectfully, I call bullshit on this. We know some very good ways to build reliable software. It's just that most industries can't / don't want to pay the costs associated with doing so. The don't-know / don't-want distinction is functionally irrelevant, but I have a minor tick about it because asserting the former is often used to avoid admitting to the later. Legacy IBM and ATT, for all their cluster%_$^!s, were amazing at engineering reliable systems (software and hardware). Maybe that's not practical nowadays, outside of heavily regulated industries like military / aerospace, because markets have greater numbers of competitors. But we do know how. An accurate and modern truth would probably be "We don't know how to quickly build reliable software at low cost."
- mruts 8y ago
  I mean, clearly, if we really wanted to, we could build all of our software with Coq and have it be formally verified. But we don’t. The Curry Howard isomorphism proves that code and math are equivalent and since we can prove a bunch of stuff with math, we should also be able to prove that our software systems are big free. And so you’re right, maybe my statement is a little bit of a simplification, but not by much. The amount of man hours it would take to prove that the Linux kernel had no bugs according to some specification would require an absolutely prohibitive amount of time. As such formal verification is so prohibitive that only the most trivial systems could ever be proved. This is pretty similar to saying that we don’t know how to build bug-free code. We need a paradigm change for things to get better, and I’m not sure it will happen or if it’s even possible.
  - henrikeh 8y ago
    It is not about tools. It is about culture and process. Look at something like how SQLite for a real example of living up to the requirements of aviation software: 100 % branch coverage, extensive testing, multiple independent platforms in the test suite etc etc. You don’t need anything fancy – but you do need to understand your requirements and thoroughly simulate and verify the intended behavior.
  - AnimalMuppet 8y ago
    I think of it in slightly different terms. Yes, we could build software that reliably. (Perhaps not totally bug free, but, say, with 1/100th of the bugs it currently has.) But that takes a lot more time and effort (say, 100 times as much, as a rough number). That means, because it would take 100 times as much effort to produce each piece of software, we'd only have 1/100th as much software. Would that be a better world? Sure, I hate it when my word processor crashes. On the other hand, I really like having a word processor at all. It has value for me, even in a somewhat buggy state, more value than a typewriter does. Would I give up having a word processor in order to have, say, a much less buggy OS? I don't think I would. I hate the bugs. But the perfect may be the enemy of the "good enough to get real work done".
    - ethbro 8y ago
      I look at that as the efficiency distinction. It goes without saying that if we had more time, there would be fewer bugs. But there are also things (tooling, automation, code development processes) that can decrease the amount of bugs without ballooning development time. Things that decrease bugs_created per unit_of_development_time. Automated test suites with good coverage, or KASAN [1, mentioned on here recently]. Code scanners, languages that prohibit unsafe development operations or guard them more closely, and automated fuzzing too. [1] https://www.kernel.org/doc/html/v4.14/dev-tools/kasan.html https://www.kernel.org/doc/html/v4.14/dev-tools/kasan.html
Klathmon 8y ago
Do you know that first hand, or is this a case that you can see the cracks in the software industry because you are part of it? I worked as a mechanic years ago, and I feel the same way about cars as you (and I) do about software. Everything is duct taped together and it's a miracle any of it works as well as it does. Safety regulations and tests are gamed to all hell, reliability almost always seems to come from an evolutionary method of "just keep using what works and change what doesn't" with hastily tacked on explanations as to why it works, and marketing is so divorced from reality that people assume significantly more is happening in the car than really is. Now I don't know anything about civil engineering or aricraft, but I have a hunch it's not as well put together as it may seem at first glance. I still think that software is behind those other fields, and I'm honestly still on the fence if regulations would significantly help or would just slow things down, but it's absolutely the "wild west" in software.
- mruts 8y ago
  I'm not a "real" engineer, but many of my cousins are (civil engineers) and everything about a bridge, say, is modeled and proven ahead of time. I work as a developer and everyday I feel like this isn't the right the way to do it. I'm not sure what the right way is, but being a software developer I probably have less faith in it then the general population. Something just feels wrong about it, how we do things..
  - User23 8y ago
    Imagine building a bridge without a specification. It would be criminally irresponsible. It has been known for decades now that proper specification is absolutely necessary to produce correct software. This is practically a tautology, because there isn't a formally meaningful way to define "correct" besides "satisfies some specified behaviors." Good unit testing does this in an ad hoc way, because the tests serve as a (usually incomplete) specification of the desired behavior. That said we must remember that testing can never prove the absence of bugs, only their presence. The only practicable ways to prove the absence of bugs in nontrivial programs are automatic formal verification or constructive proof using formally defined semantics. Of course for most commercial software correctness is so irrelevant that it's left undefined. All that matters is pleasantness, that is to say does it generally please the user, perhaps by attracting customers or investors.
fma 8y ago
Look at the Equifax hack. Vulnerability was known but no one cared to apply the patch. In engineering, you chose 2 amongst being better, faster or cheaper. It's clear in IT we always choose faster and cheaper. No one went to jail for the Equifax hack except that one low level manager for insider trading.
ummonk 8y ago
Airplanes have software. It works very well without bugs. Because that is what is demanded in the aviation industry.