10 ms·

Controversial—I know—(and I'm happy to hear counterarguments) but I think there's a much simpler way of accounting for the present state of software quality. I

Controversial—I know—(and I'm happy to hear counterarguments) but I think there's a much simpler way of accounting for the present state of software quality. It amounts to 'tolerance'. Commercial software is optimized with respect to a set of business goals, and one dimension of the space that process operates on is software quality. Beyond a certain threshold, increasing quality has diminishing utility. (I.e. there's effectively a tolerance for quality; it can be imperfect and not change the operation of the machine—i.e. business—which it's a component of.) The present state of software is precisely in the neighborhood of where that added utility begins rapidly diminishing. (This assumes that the business in question is competent, which we can approximate by observing that they survive. Outside the possibility of that being incorrect, this implies that the level of software quality hovers right around where it it's good enough that improving it more wouldn't significantly impact customer purchasing decisions. Of course this is not an ideal state of affairs—but how could you expect otherwise? —and interpreting it as a sign of societal collapse... seems a bit absurd to me.)

swivelmaster 4y ago
I would argue that the decline in software quality is related to the rise of SaaS coupled with Product Manager-driven development. If a product is generating consistent revenue, resources are devoted to expanding that revenue. It's extraordinarily difficult to justify bugfixes and incremental improvements using exclusively analytics data, so organizations are generally not incentivized to reward that kind of activity.
- westoncb 4y ago
  > It's extraordinarily difficult to justify bugfixes and incremental improvements using exclusively analytics data Why would analytics be any less capable of conveying how these attributes of a software product affect customer purchasing decisions? If the metrics relate to e.g. retention, new users etc., these are independent of which aspects of the software cause them. A correlation is a correlation. Unless there is some reason to think product managers would be especially unaware of quality as a variable to observe.
- majormajor 4y ago
  Do you think there's no such thing as a number of bugs that would cause customers to leave more rapidly? I don't see a big difference between what you're saying and the person you're responding to: until the level of bugs reaches the point where it hits revenue, it doesn't make sense to invest in fixing them. There's a tolerance for some level of imperfection - in fact, expecting perfection is itself somewhat irrational here since it's not like other industries routinely deeliver perfection. I'm also not convinced we're seeing an overall decline in software quality. There was a LOT of shit software from the 80s and 90s I don't want to go back to. EDIT: It's a bit of a Yogi Berra - "modern software is terrible, people just shovel out crap since the customers don't want to wait!"
routerl 4y ago
This is a fascinating point and it really makes me think of the mechanical analogy: different mechanical engineers can provide different levels of tolerance (i.e. precision), depending on their training and experience. You wouldn't hire an F1 engineer to design retail automobile machining, because both the hire and the machining process would be overkill. The software industry is stuck in this rut where everyone acts as if they're designing F1 cars, but few teams outside of FAANG actually are. In other words, for the majority of software jobs, a bootcamp and CRUD experience is likely enough.
- majormajor 4y ago
  I think most software development is already internal, line-of-business stuff at companies HN largely ignores. The level of tech used is quite boring. Ask someone with LinkedIn Recruiter to run you through a broad search of developers mentioning stuff in-fashion for bigcorp internal teams like Java or .Net - you'll see so a TON of profiles for stuff that has nothing to do with FAANG or FAANG-like tech stacks.
- mysterydip 4y ago
  > You wouldn't hire an F1 engineer to design retail automobile machining, because both the hire and the machining process would be overkill. Not only overkill, but detrimental to the goals of the retail market. The tolerances in an F1 car are so tight that the engine needs warm oil through it before it can even be started. Great for maximum performance, not great for the driver that wants to just hop in the car, turn the key, and go.
- WalterBright 4y ago
  I'm sure the F1 engineer is capable enough to adapt to different requirements.
  - tstrimple 4y ago
    Why are you so confident? Jonathan Blow can’t seem to adapt.
    - 4y ago
tgbugs 4y ago
If this is the case then it means that there is no competition in the ecosystem and that the regulators need to start busting monopolies and other anticompetitive behavior. In a competitive market where there is actually free entry and free exit and consumers are not locked in there should be no such thing as diminishing returns at the level of brokenness we see here, unless we somehow think that writing functioning software is somehow impossibly difficult.
- majormajor 4y ago
  Do you have any fields in mind where people are picking "perfect at higher cost" over "cheap and crappy but adequate" en masse? I have trouble thinking of any, and "lack of competition" hardly seems to be the reason. "Race to the bottom" isn't something that happens without competition, after all.
  - tgbugs 4y ago
    I'm oversimplifying a bit here, but I think the strategy that most major software firms are following right now would match the "adaptive radiation" phase of ecosystem evolution. There are so many empty niches as a result of the high dimensional space of user requirements (where say, 2 additional features are sufficient to differentiate a product, see e.g. discussions about slack vs discord vs teams from today) that companies hardly need directly compete with each other. This would be a virtuous explanation for lack of competition. Another candidate explanation for what is going on in your "race to the bottom" likely has to do more with signal to noise issues and other types of adverse selection where e.g. a company needs to spend only just enough money for a product to make it past the average review date, at which point the device/whatever will fail. These literally marginal devices/software/whatever are then sold by literally copying and pasting the specs from some other product that is selling well and now consumers are toast. In the software world this looks like a billion clones of the same game, etc. This however is not what I would call good faith competition, and in biological systems species that do this eventually get wiped away completely when they finish degrading the ecosystem they are in. This will look like a mobile gaming crash (about which there seem to be some google hits for back at the start of August). The kinds of things I'm thinking about are e.g. the fact that text selection in HTML documents is SO BAD that people have started to add copy buttons. Want to copy a url out of some calendaring software? OOPS NO you actually wanted to select literally all the other text on the page right? There are what 2.5ish major browser vendors (supposedly) now? That is by design and a sign of anticompetitive behavior. Another example would be the GTK+ file picker. Though in this case it is far more likely that the desktop linux environment is extremely marginal in terms of capital than anticompetitive behavior on the part of redhat.
johnfn 4y ago
As soon as Blow and others start pontificating about how modern software is terrible, I get frustrated; my viewpoint differs with theirs so drastically that I don't even know where to begin. I might start somewhere around here: I think the fact that modern software is "terrible" is actually a sign of the dramatic success, not failure, of programming. The reasons that Blow et all normally cite as why software is terrible are usually that, 20 and 30 years ago, engineers were able to pull off superhuman feats compared to the engineers of today. I accept this unquestionably. They wrote software that ran quicker and was more space-efficient than modern software. But running software that fit in kilobytes of RAM was never the goal of software engineering. The goal was providing value to your users. Blow et. al. seem to think that efficient software is the goal. I couldn't disagree more. The goal is always to produce things that users can actually use. And if that means doing hideously inefficient things, like running your app inside a chrome shell, so be it, if that means you delivered value faster. And the amount of engineering effort per unit of value created has gone down so much today compared to 30 years ago. In the past, engineers could squander days or weeks trying to make code more memory efficient, or finding the exact right sequence of ASM instructions to waste less CPU cycles. These are issues I don't even have to think about. Heck, I can do extremely stupid and inefficient things, like use a garbage collector, or electron, and for the most part, users don't even care[1]. What an incredible success! Essentially, I think what Blow is seeing when he cites the demise of modern software engineering is in fact that the bar to writing usable software has dropped immensely. I can understand why this is a frustrating thing for him, because it effectively means that his skillset is not as valuable as it used to be. But I just can't agree at all. How amazing that we can be so wasteful and yet still produce such great software, and furthermore what an incredible success that engineers like me don't need to even think about ASM and RAM efficiency. [1] Yes, some users do care, and I think that a disproportionate majority of them hang out on Hacker News. :P
- agentultra 4y ago
  I too find his presentation pretentious and his claims suspect. He has released two games in his career so far and an as-yet-to-be-released compiler that he’s been toying with for years: and he calls modern software developers unproductive! However I think efficiency still matters depending on the use case and to his credit I don’t think he says that everyone has to care about efficiency. I think the point he makes is that we might be losing the ability to produce efficient code because we’re not teaching these techniques. And that is perhaps why we buy $3000 Facebook browsing machines that don’t feel any faster than computers 20 years ago. Often they are slower. If we only teach folks how to script Unity and Unreal how are we supposed to get the next generation of engine developers to build the foundations of the technology?
bigbillheck 4y ago
As far as I can tell, 'software quality' these days is pretty good, all things considered, as compared to twenty or thirty years ago.
annoyingnoob 4y ago
> The present state of software is precisely in the neighborhood of where that added utility begins rapidly diminishing. I can think of several commercial software offerings where quality is and always has been lacking. Quickbooks is one example, it has been extremely buggy the entire time it has been a product. Yet, lots of people still buy it and use it.
bmitc 4y ago
> Commercial software is optimized with respect to a set of business goals I'm not sure that's entirely true. It ostensibly is, but the way it is written is not optimized for long-term business goals. Because the way software tools work and the way software is written is that it is all completely optimized for short-term business goals and results. I'm trying to pen an article that addresses this, because software is often if not solely viewed as just doing something, when that's only a component of what it does. If we additionally view it as a way to communicate amongst humans and a way to think about and encode a domain, then we begin to see the failings of software in today's world. In this holistic viewpoint, software is best viewed as stored and interactive knowledge. If software is simply viewed as a way to do something, then the stored knowledge becomes highly implicit and incomplete, and a communication breakdown happens. Viewing things with this lens, it is no surprise why today's software is primarily towers of shit. There is no handling of variety, that is complexity, because the way to handle it is to properly communicate within the medium of software systems and encode the domain. Otherwise, software just doing something often creates more complexity than it reduces.
- westoncb 4y ago
  I think it's an interesting thesis, and I like the point about using it to encode domains represented through it (I agree if you want to build non-overcomplicated systems, the first major factor is to minimize impedance mismatch with the domain it represents in the base structures you build on, i.e. in the 'language' you develop within your codebase to eventually express domain logic through.) So while I have some issues with this as a generalization (because I think the reality is more diverse): > Because the way software tools work and the way software is written is that it is all completely optimized for short-term business goals and results. —let's assume it now for the sake of argument. What's still weak in the thesis imo is the connection from your proposed better way of approaching software development to longer-term business goals. For instance, I think one possibility is that the typical state of affairs, over a longer time-span, is that what the software itself should even be, what it needs to accomplish, which systems it interacts with etc., is volatile enough that in most cases (there are very important exceptions of course), businesses would do better long term by optimizing for flexibility in changing even the spec for the software, vs devoting more resources to ensuring any individual representation of a software concept is particularly durable. What I'd want to know next on the matter is what evidence exists for the likelihood of one stance over the other being correct. Are there examples of extraordinary long-term success from companies whose approach to development went in the direction of durability (I'll say as an approximation)? Is it a common occurrence for companies to succeed short-term by making sacrifices in quality, and then to fail longer-term specifically because of their over-complex / incoherent, buggy software (or is that more of an exceptional case)?

swivelmaster 4y ago

I would argue that the decline in software quality is related to the rise of SaaS coupled with Product Manager-driven development. If a product is generating consistent revenue, resources are devoted to expanding that revenue. It's extraordinarily difficult to justify bugfixes and incremental improvements using exclusively analytics data, so organizations are generally not incentivized to reward that kind of activity.

westoncb 4y ago
> It's extraordinarily difficult to justify bugfixes and incremental improvements using exclusively analytics data Why would analytics be any less capable of conveying how these attributes of a software product affect customer purchasing decisions? If the metrics relate to e.g. retention, new users etc., these are independent of which aspects of the software cause them. A correlation is a correlation. Unless there is some reason to think product managers would be especially unaware of quality as a variable to observe.
majormajor 4y ago
Do you think there's no such thing as a number of bugs that would cause customers to leave more rapidly? I don't see a big difference between what you're saying and the person you're responding to: until the level of bugs reaches the point where it hits revenue, it doesn't make sense to invest in fixing them. There's a tolerance for some level of imperfection - in fact, expecting perfection is itself somewhat irrational here since it's not like other industries routinely deeliver perfection. I'm also not convinced we're seeing an overall decline in software quality. There was a LOT of shit software from the 80s and 90s I don't want to go back to. EDIT: It's a bit of a Yogi Berra - "modern software is terrible, people just shovel out crap since the customers don't want to wait!"

routerl 4y ago

This is a fascinating point and it really makes me think of the mechanical analogy: different mechanical engineers can provide different levels of tolerance (i.e. precision), depending on their training and experience. You wouldn't hire an F1 engineer to design retail automobile machining, because both the hire and the machining process would be overkill. The software industry is stuck in this rut where everyone acts as if they're designing F1 cars, but few teams outside of FAANG actually are. In other words, for the majority of software jobs, a bootcamp and CRUD experience is likely enough.

majormajor 4y ago
I think most software development is already internal, line-of-business stuff at companies HN largely ignores. The level of tech used is quite boring. Ask someone with LinkedIn Recruiter to run you through a broad search of developers mentioning stuff in-fashion for bigcorp internal teams like Java or .Net - you'll see so a TON of profiles for stuff that has nothing to do with FAANG or FAANG-like tech stacks.
mysterydip 4y ago
> You wouldn't hire an F1 engineer to design retail automobile machining, because both the hire and the machining process would be overkill. Not only overkill, but detrimental to the goals of the retail market. The tolerances in an F1 car are so tight that the engine needs warm oil through it before it can even be started. Great for maximum performance, not great for the driver that wants to just hop in the car, turn the key, and go.
WalterBright 4y ago
I'm sure the F1 engineer is capable enough to adapt to different requirements.
- tstrimple 4y ago
  Why are you so confident? Jonathan Blow can’t seem to adapt.
  - 4y ago

tgbugs 4y ago

If this is the case then it means that there is no competition in the ecosystem and that the regulators need to start busting monopolies and other anticompetitive behavior. In a competitive market where there is actually free entry and free exit and consumers are not locked in there should be no such thing as diminishing returns at the level of brokenness we see here, unless we somehow think that writing functioning software is somehow impossibly difficult.

majormajor 4y ago
Do you have any fields in mind where people are picking "perfect at higher cost" over "cheap and crappy but adequate" en masse? I have trouble thinking of any, and "lack of competition" hardly seems to be the reason. "Race to the bottom" isn't something that happens without competition, after all.
- tgbugs 4y ago
  I'm oversimplifying a bit here, but I think the strategy that most major software firms are following right now would match the "adaptive radiation" phase of ecosystem evolution. There are so many empty niches as a result of the high dimensional space of user requirements (where say, 2 additional features are sufficient to differentiate a product, see e.g. discussions about slack vs discord vs teams from today) that companies hardly need directly compete with each other. This would be a virtuous explanation for lack of competition. Another candidate explanation for what is going on in your "race to the bottom" likely has to do more with signal to noise issues and other types of adverse selection where e.g. a company needs to spend only just enough money for a product to make it past the average review date, at which point the device/whatever will fail. These literally marginal devices/software/whatever are then sold by literally copying and pasting the specs from some other product that is selling well and now consumers are toast. In the software world this looks like a billion clones of the same game, etc. This however is not what I would call good faith competition, and in biological systems species that do this eventually get wiped away completely when they finish degrading the ecosystem they are in. This will look like a mobile gaming crash (about which there seem to be some google hits for back at the start of August). The kinds of things I'm thinking about are e.g. the fact that text selection in HTML documents is SO BAD that people have started to add copy buttons. Want to copy a url out of some calendaring software? OOPS NO you actually wanted to select literally all the other text on the page right? There are what 2.5ish major browser vendors (supposedly) now? That is by design and a sign of anticompetitive behavior. Another example would be the GTK+ file picker. Though in this case it is far more likely that the desktop linux environment is extremely marginal in terms of capital than anticompetitive behavior on the part of redhat.

johnfn 4y ago

As soon as Blow and others start pontificating about how modern software is terrible, I get frustrated; my viewpoint differs with theirs so drastically that I don't even know where to begin. I might start somewhere around here: I think the fact that modern software is "terrible" is actually a sign of the dramatic success, not failure, of programming. The reasons that Blow et all normally cite as why software is terrible are usually that, 20 and 30 years ago, engineers were able to pull off superhuman feats compared to the engineers of today. I accept this unquestionably. They wrote software that ran quicker and was more space-efficient than modern software. But running software that fit in kilobytes of RAM was never the goal of software engineering. The goal was providing value to your users. Blow et. al. seem to think that efficient software is the goal. I couldn't disagree more. The goal is always to produce things that users can actually use. And if that means doing hideously inefficient things, like running your app inside a chrome shell, so be it, if that means you delivered value faster. And the amount of engineering effort per unit of value created has gone down so much today compared to 30 years ago. In the past, engineers could squander days or weeks trying to make code more memory efficient, or finding the exact right sequence of ASM instructions to waste less CPU cycles. These are issues I don't even have to think about. Heck, I can do extremely stupid and inefficient things, like use a garbage collector, or electron, and for the most part, users don't even care[1]. What an incredible success! Essentially, I think what Blow is seeing when he cites the demise of modern software engineering is in fact that the bar to writing usable software has dropped immensely. I can understand why this is a frustrating thing for him, because it effectively means that his skillset is not as valuable as it used to be. But I just can't agree at all. How amazing that we can be so wasteful and yet still produce such great software, and furthermore what an incredible success that engineers like me don't need to even think about ASM and RAM efficiency. [1] Yes, some users do care, and I think that a disproportionate majority of them hang out on Hacker News. :P

agentultra 4y ago
I too find his presentation pretentious and his claims suspect. He has released two games in his career so far and an as-yet-to-be-released compiler that he’s been toying with for years: and he calls modern software developers unproductive! However I think efficiency still matters depending on the use case and to his credit I don’t think he says that everyone has to care about efficiency. I think the point he makes is that we might be losing the ability to produce efficient code because we’re not teaching these techniques. And that is perhaps why we buy $3000 Facebook browsing machines that don’t feel any faster than computers 20 years ago. Often they are slower. If we only teach folks how to script Unity and Unreal how are we supposed to get the next generation of engine developers to build the foundations of the technology?

bigbillheck 4y ago

As far as I can tell, 'software quality' these days is pretty good, all things considered, as compared to twenty or thirty years ago.

annoyingnoob 4y ago

> The present state of software is precisely in the neighborhood of where that added utility begins rapidly diminishing. I can think of several commercial software offerings where quality is and always has been lacking. Quickbooks is one example, it has been extremely buggy the entire time it has been a product. Yet, lots of people still buy it and use it.

bmitc 4y ago

> Commercial software is optimized with respect to a set of business goals I'm not sure that's entirely true. It ostensibly is, but the way it is written is not optimized for long-term business goals. Because the way software tools work and the way software is written is that it is all completely optimized for short-term business goals and results. I'm trying to pen an article that addresses this, because software is often if not solely viewed as just doing something, when that's only a component of what it does. If we additionally view it as a way to communicate amongst humans and a way to think about and encode a domain, then we begin to see the failings of software in today's world. In this holistic viewpoint, software is best viewed as stored and interactive knowledge. If software is simply viewed as a way to do something, then the stored knowledge becomes highly implicit and incomplete, and a communication breakdown happens. Viewing things with this lens, it is no surprise why today's software is primarily towers of shit. There is no handling of variety, that is complexity, because the way to handle it is to properly communicate within the medium of software systems and encode the domain. Otherwise, software just doing something often creates more complexity than it reduces.

westoncb 4y ago
I think it's an interesting thesis, and I like the point about using it to encode domains represented through it (I agree if you want to build non-overcomplicated systems, the first major factor is to minimize impedance mismatch with the domain it represents in the base structures you build on, i.e. in the 'language' you develop within your codebase to eventually express domain logic through.) So while I have some issues with this as a generalization (because I think the reality is more diverse): > Because the way software tools work and the way software is written is that it is all completely optimized for short-term business goals and results. —let's assume it now for the sake of argument. What's still weak in the thesis imo is the connection from your proposed better way of approaching software development to longer-term business goals. For instance, I think one possibility is that the typical state of affairs, over a longer time-span, is that what the software itself should even be, what it needs to accomplish, which systems it interacts with etc., is volatile enough that in most cases (there are very important exceptions of course), businesses would do better long term by optimizing for flexibility in changing even the spec for the software, vs devoting more resources to ensuring any individual representation of a software concept is particularly durable. What I'd want to know next on the matter is what evidence exists for the likelihood of one stance over the other being correct. Are there examples of extraordinary long-term success from companies whose approach to development went in the direction of durability (I'll say as an approximation)? Is it a common occurrence for companies to succeed short-term by making sacrifices in quality, and then to fail longer-term specifically because of their over-complex / incoherent, buggy software (or is that more of an exceptional case)?