8 ms·

> How did software get so reliable without proof? Simple, software started shipping with more bugs in backlogs :-P — More seriously, software today is endemic

> How did software get so reliable without proof? Simple, software started shipping with more bugs in backlogs :-P — More seriously, software today is endemically crappy, and often poorly designed. I dunno why Hoare thought it was any better. The only saving grace is that (modern) software largely stays away from serious stuff. Eg: The airline industry is reluctant to upgrade software from decades ago, but happy to incrementally upgrade other pieces of the system on shorter cycles. Then of course, we have systems like the Boeing 737Max MCAS where the software did what it was supposed to (taken literally) but the software system was poorly designed. EDIT: Just remembered this fantastic talk by Joe Armstrong https://youtu.be/lKXe3HUG2l4 https://youtu.be/lKXe3HUG2l4 The smooth running of every abstract system depends crucially on the (human) operator handling the point of contact with reality, in practice, often bending reality to make it tractable for the system. Any bureaucracy would grind to a halt if it wasn’t intelligent humans carrying out the processes! Software is no different. Just like it’s hard to take a technology from “zero to one”, it is hard to take the amount of necessary human oversight from one to zero. For this reason, I would much rather think of software as amplifying the capabilities of that human, rather than automating away other humans. In practice, these systems will end up needing a highly skilled and trustworthy human operator to shepherd them — might as well design the system to make it maximally easy for those operators to understand/debug/tweak.

WalterBright 7y ago
> software today is endemically crappy, and often poorly designed So is every other engineering product that hasn't gone through years (decades) of evolutionary refinement.
- averros 7y ago
  Not really. When programming was done by real engineers (rather than "coders", "developers", or horribly misnamed "software engineers") - with proper engineering discipline which involved deliberate design and documentation rather than "agile" hacking, it was more reliable. By far. My personal recent experience of actually doing software the old-fashioned way involved writing correctness critical high-performance code for a major data warehouse vendor, which mostly stayed with zero known bugs in production - greatly contributing to the vendor's reputation as reliable and dependable place to keep your data in. And how do I know what the old-fashioned way to write code is? Well, I've been writing code professionally for nearly 40 years.
  - ColanR 7y ago
    The more I learn about modern software practices, the more I come to think that software is worse today because programmers are poorly disciplined and badly trained. Of course, that's likely because the barrier to entry is so much lower today than it used to be.
    - WalterBright 7y ago
      The barrier of entry to software has been zero ever since the 8080 was introduced.
      - fxtentacle 7y ago
        Back then there were a lot less stackoverflow copy & paste mistakes.
        akiselev 7y ago
        There were also far fewer users to really discover the nasty edge case bugs.
        speedplane 7y ago
        > There were also far fewer users to really discover the nasty edge case bugs. If you consider why a company makes something reliable or not, it's a relatively simple formula: = Number of Users x (Benefit of Getting it Right x Probability of Getting it Right) - (Cost of Getting it Wrong x (1 - Probability of Getting it Right))) As the number of users in any system increases, the cost overall cost of getting it wrong also increases. You can then devote more fixed cost resources to improving probability of getting it right.
  - WalterBright 7y ago
    > it was more reliable. By far [talking about non-software engineering] There's a huge difference between reading books about proper engineering and how it is actually practiced. Much of the sloppiness is covered by simply over-engineering it. Designs are constantly improved over time based on service experience. Heck, lots of times the first machine off the line cannot even be assembled because the design dimensions on the drawings are wrong. The idea that non-software engineering is done by careful professionals following best practice and not making lots of mistakes is painfully wrong.
- machawinka 7y ago
  I doubt software engineering is engineering discipline. At least not the way we are doing it now. We are not engineers, we are craftsmen. It is a miracle that the software that drives the world more or less works. So far there hasn't been any major catastrophe due to software but it will happen. Software sucks.
  - ci5er 7y ago
    Over time, I think I am coming to agree with you. I went to an engineering school. People who designed bridges had a wide, but constrained parameter space, and well-accepted design patterns. I started out in semiconductor (MCU) systems and sometimes circuit design, and we had a broader (but not yuuuge!) parameter space, but it was growing as transistors got cheaper. Less well-accepted design patterns, because what you do with 50K transistors and 500K transistors and 5M transistors and 50M transistors - to use effectively - you need different patterns - and that changed so fast! I did software-ish things with my h/w teams, and they would mock me because "software is easy". "You can do anything". And to do a rev, you didn't need to burn a $50K and 6 weeks (or whatever it is now) at the fab for each turn. The problem with software is that it is SO unconstrained. You truly CAN do anything - except engineer in an unconstrained environment. I guess this realization (and Python blowing up on me in run-time environments) have taught me that: Constraints suck. But they are good. Software could use more of them, because they force discipline at dev time, at compile time, which reduces blow-ups at run-time.
    - agumonkey 7y ago
      I can recall a few articles where software felt like engineering. bits derived data needs, complexity based hardware sizing. Real solid constraints to use as foundations. rare
    - WalterBright 7y ago
      > People who designed bridges had a wide, but constrained parameter space, and well-accepted design patterns. And yet they still fall down due to stupid mistakes that nobody noticed. https://www.usatoday.com/story/news/nation/2019/10/22/design-error-blamed-florida-international-university-pedestrian-bridge-collapse/2449316001/ https://www.usatoday.com/story/news/nation/2019/10/22/design...
- CivBase 7y ago
  I don't buy it. I think the real problem is that software became one of the worlds biggest commodities practically overnight and developers are usually more incentivized to get stuff done faster rather than produce higher quality. There's practically no material cost to software development, the proliferation of the internet makes patching trivial, and the overwhelming majority of software amounts to an nothing more than an inconvenience if it fails. Considering the high value of software and the relatively low cost of bugs, it's only natural that speed is valued so much more than quality.
  - EnderMB 7y ago
    I'd go even further with this. I'd say that most software engineers are ultimately powerless to improve software beyond a certain point because doing so would cost more than their employer is willing to spend. Given enough time, any software engineer could make a reasonably solid product/service that would stand the test of time. Most projects are run on a deficit of either time or resource, and as a result we end up with cut corners.
  - lloeki 7y ago
    Ironically it seems people as well as society as a whole fails to account that a zillion times inconvenience is death by a thousand cuts. This includes everyone writing code. Goddammit when you take a long, hard look, objectively (like really) stepping back for a moment, every piece of software (and hardware to some extent as the line between both is getting blurry) around us is just a huge pile of Rube Goldberg machinery that barely happens to work on the happy path. This process is not entirely unlike global warming and similar large scale psychological risk assessment failures.
    - sitkack 7y ago
      And each individual piece might not be mission critical, but some mission critical component of the system relies on some non-mission critical piece of software or service. The more I look at how fragile all human created systems are, I realize that humanity will perish with the equivalent of tripping over its shoelaces. Own goal!
  - 7y ago
- collyw 7y ago
  The problem I see is that we keep throwing away things that work well and are understood. Relational databases will work well for most products, its understood how to design them well, optimize them, tune the servers. But nowadays we need to stick everything into microservices on a Kubernetes cluster with Mongo instead and we are told that this is progress.

WalterBright 7y ago

> software today is endemically crappy, and often poorly designed So is every other engineering product that hasn't gone through years (decades) of evolutionary refinement.

averros 7y ago
Not really. When programming was done by real engineers (rather than "coders", "developers", or horribly misnamed "software engineers") - with proper engineering discipline which involved deliberate design and documentation rather than "agile" hacking, it was more reliable. By far. My personal recent experience of actually doing software the old-fashioned way involved writing correctness critical high-performance code for a major data warehouse vendor, which mostly stayed with zero known bugs in production - greatly contributing to the vendor's reputation as reliable and dependable place to keep your data in. And how do I know what the old-fashioned way to write code is? Well, I've been writing code professionally for nearly 40 years.
- ColanR 7y ago
  The more I learn about modern software practices, the more I come to think that software is worse today because programmers are poorly disciplined and badly trained. Of course, that's likely because the barrier to entry is so much lower today than it used to be.
  - WalterBright 7y ago
    The barrier of entry to software has been zero ever since the 8080 was introduced.
    - fxtentacle 7y ago
      Back then there were a lot less stackoverflow copy & paste mistakes.
      - akiselev 7y ago
        There were also far fewer users to really discover the nasty edge case bugs.
        speedplane 7y ago
        > There were also far fewer users to really discover the nasty edge case bugs. If you consider why a company makes something reliable or not, it's a relatively simple formula: = Number of Users x (Benefit of Getting it Right x Probability of Getting it Right) - (Cost of Getting it Wrong x (1 - Probability of Getting it Right))) As the number of users in any system increases, the cost overall cost of getting it wrong also increases. You can then devote more fixed cost resources to improving probability of getting it right.
- WalterBright 7y ago
  > it was more reliable. By far [talking about non-software engineering] There's a huge difference between reading books about proper engineering and how it is actually practiced. Much of the sloppiness is covered by simply over-engineering it. Designs are constantly improved over time based on service experience. Heck, lots of times the first machine off the line cannot even be assembled because the design dimensions on the drawings are wrong. The idea that non-software engineering is done by careful professionals following best practice and not making lots of mistakes is painfully wrong.
machawinka 7y ago
I doubt software engineering is engineering discipline. At least not the way we are doing it now. We are not engineers, we are craftsmen. It is a miracle that the software that drives the world more or less works. So far there hasn't been any major catastrophe due to software but it will happen. Software sucks.
- ci5er 7y ago
  Over time, I think I am coming to agree with you. I went to an engineering school. People who designed bridges had a wide, but constrained parameter space, and well-accepted design patterns. I started out in semiconductor (MCU) systems and sometimes circuit design, and we had a broader (but not yuuuge!) parameter space, but it was growing as transistors got cheaper. Less well-accepted design patterns, because what you do with 50K transistors and 500K transistors and 5M transistors and 50M transistors - to use effectively - you need different patterns - and that changed so fast! I did software-ish things with my h/w teams, and they would mock me because "software is easy". "You can do anything". And to do a rev, you didn't need to burn a $50K and 6 weeks (or whatever it is now) at the fab for each turn. The problem with software is that it is SO unconstrained. You truly CAN do anything - except engineer in an unconstrained environment. I guess this realization (and Python blowing up on me in run-time environments) have taught me that: Constraints suck. But they are good. Software could use more of them, because they force discipline at dev time, at compile time, which reduces blow-ups at run-time.
  - agumonkey 7y ago
    I can recall a few articles where software felt like engineering. bits derived data needs, complexity based hardware sizing. Real solid constraints to use as foundations. rare
  - WalterBright 7y ago
    > People who designed bridges had a wide, but constrained parameter space, and well-accepted design patterns. And yet they still fall down due to stupid mistakes that nobody noticed. https://www.usatoday.com/story/news/nation/2019/10/22/design-error-blamed-florida-international-university-pedestrian-bridge-collapse/2449316001/ https://www.usatoday.com/story/news/nation/2019/10/22/design...
CivBase 7y ago
I don't buy it. I think the real problem is that software became one of the worlds biggest commodities practically overnight and developers are usually more incentivized to get stuff done faster rather than produce higher quality. There's practically no material cost to software development, the proliferation of the internet makes patching trivial, and the overwhelming majority of software amounts to an nothing more than an inconvenience if it fails. Considering the high value of software and the relatively low cost of bugs, it's only natural that speed is valued so much more than quality.
- EnderMB 7y ago
  I'd go even further with this. I'd say that most software engineers are ultimately powerless to improve software beyond a certain point because doing so would cost more than their employer is willing to spend. Given enough time, any software engineer could make a reasonably solid product/service that would stand the test of time. Most projects are run on a deficit of either time or resource, and as a result we end up with cut corners.
- lloeki 7y ago
  Ironically it seems people as well as society as a whole fails to account that a zillion times inconvenience is death by a thousand cuts. This includes everyone writing code. Goddammit when you take a long, hard look, objectively (like really) stepping back for a moment, every piece of software (and hardware to some extent as the line between both is getting blurry) around us is just a huge pile of Rube Goldberg machinery that barely happens to work on the happy path. This process is not entirely unlike global warming and similar large scale psychological risk assessment failures.
  - sitkack 7y ago
    And each individual piece might not be mission critical, but some mission critical component of the system relies on some non-mission critical piece of software or service. The more I look at how fragile all human created systems are, I realize that humanity will perish with the equivalent of tripping over its shoelaces. Own goal!
- 7y ago
collyw 7y ago
The problem I see is that we keep throwing away things that work well and are understood. Relational databases will work well for most products, its understood how to design them well, optimize them, tune the servers. But nowadays we need to stick everything into microservices on a Kubernetes cluster with Mongo instead and we are told that this is progress.