7 ms·

How to think about software quality (2022)

154 points by adityaathalye 3y ago

pavel_lishin 3y ago
So much software suffers from a large class of users having no choice in its use. For example, our local school district uses Bloomz, which is pretty terrible - both on the web, and in its native app. I can report all the bugs I want, but since I'm never the person making a purchasing decision, how highly do I think those reports are prioritized? I wonder if the teacher/administrator side of things is better.
- deleted 3y ago
  [deleted]
- dboreham 3y ago
  I found the best approach is to search LinkedIn for someone at the vendor that's closely connected to me, then compose a cheery message to that person introducing myself as a fellow developer and parent of a user, ask the best way to inject a bug report etc. This actually worked. Ymmv of course.
  - kqr 3y ago
    Ohh, this is actually clever. Subversive and I'm not sure I like it, but it sounds very effective.
    - zx8080 3y ago
      Sounds like a hack to make a person to work overtime.
      - amelius 3y ago
        ... while having fun while doing that work.
      - the_fury 3y ago
        If I get a message on linkedin that pertains to a work issue, you had better believe that I'm going to be dealing with it on work time.
  - computerfriend 3y ago
    One of the reasons I never put my employer on my LinkedIn profile.
    - datavirtue 3y ago
      Most of life's fulfilling opportunities come from just showing up. Showing up on searches, at work, at the pub.
    - deprecative 3y ago
      If I got a message like this I'd just block the sender. As a developer I have no say in what my owner tells me to work on. I type what they want and nothing more.
      - D-Coder 3y ago
        They're not asking you to fix anything. They are asking where to submit a bug report that someone (someone else) will look at.
        deprecative 3y ago
        My answer doesn't change. I'm not on LinkedIn during work hours.
  - binarysneaker 3y ago
    As someone this happened to, I was actually impressed they looked me up and reached out. Knowing they cared that much meant that I cared too and I was able to prioritise the request with the product team. When we completed the request, we celebrated, and even implemented uservoice on some sites to improve the feedback loop.
- hitchstory 3y ago
  This software also tends to be waterfalled to hell and back. Theyve already planned out their deliverables for the next 12 months and will be held accountable for meeting them. Your bugfix doesnt fit into that picture unless an exec treats it as a priority. ...in which case it'll be jammed into the devs' task queue with no expectation that the pre-agreed delivery dates will shift.
  - datavirtue 3y ago
    The issues you describe have nothing to do with "waterfall."
    - hitchstory 3y ago
      >Theyve already planned out their deliverables for the next 12 months and will be held accountable for meeting them.
      - unclebucknasty 3y ago
        Not sure I understand your point. Are you saying product roadmaps are incompatible with agile?
        hitchstory 3y ago
        If they include detailed plans and timelines for feature deliveries then yes.
        unclebucknasty 3y ago
        Agile doesn't mean "not planning" or even "vague planning".
- adityaathalye 3y ago
  Post author here. I agree; so often constructive feedback loops from end users to builders are woefully absent from the equation. I feel the greater the distance between the purchase decision maker and the end user, the weaker the feedback gets. Riffing off the six degrees of separation idea and exponential drop off of signals though lossy media, it's like by the time someone gets to a third degree of separation, any feedback signal barely registers. Only mass action has any chance of being heard, much less heeded.
  - AnimalMuppet 3y ago
    Ah. So that's (part of) why enterprise software is so universally bad. The users are further from the purchasers than they are in, say, developer tools.
    - adityaathalye 3y ago
      I believe so. I'll pick on Government websites and the systems that back them as an all-too-familiar, internationally reviled category of software, as an example. These tend to be incredibly painful to use. Some tendering process somewhere awards a contract to the lowest bidder who then has the unenviable job of wrangling the delightful intricacies of government machinery to put something out that maybe does what it was supposed to do for the common citizen. In such a world, I feel gov.uk stands out as a remarkable counter-example, worthy of emulation. e.g. https://www.gov.uk/guidance/government-design-principles https://www.gov.uk/guidance/government-design-principles
- jSully24 3y ago
  You mention not being the person making a purchasing decision, who is? Can you connect with them? I would encourage you to put together a story that illustrates the pain being caused by this software and get with the purchasers. Not only might it help your organization figure out they need to do something different it could also show off your skills to the orgainization. Could you collect the time you and other people are spending working on these problems? How many issues are reported to you? How many bugs you've filed and have not had fixed by the vendor? Rolling this up into a real dollar coast it's having on you and the school district users.
  - deleted 3y ago
    [deleted]
  - cratermoon 3y ago
    > who is? Can you connect with them? For something like this, usually at the senior executive level, and organizationally. There's not really one person making the decision, it's done through the organization's decision-making system. Nominally there's a responsible person, but practically speaking that individual isn't completely autonomous. Large organizations have been making and dealing with decisions like this for ages, and rarely does dollar cost of usability ever influence. Products SAP and MS Teams get and retain market share by making the cost of moving off that software outweigh the gains.
  - pavel_lishin 3y ago
    The person making the decision is very like the superintendent of the school system, likely informed by someone from IT and maybe someone representing the teachers somehow. They might listen to me if I bring in a coalition of other parents, but there's two reasons why I'm probably not going to do this: 1. They're likely locked into at least a year-long contract, if not more 2. Doing this is a huge amount of work for me. I'd have to find a significant number of parents who are _also_ as peeved as me, and then get them to take action - even if it's signing something - and then I'd have to bring the petition up to the school board, etc., etc., etc. It's much easier to just mute the notifications on the app, and go through them once a week. 99% of them are just wildly inactionable anyway - things like reminding me that school is out, or that a student interest group that's irrelevant to me is having a meeting two Thursdays from now. If I was annoyed enough to take action, I'd rather just offer to build them a replacement app - shorten that feedback loop, ya know?
- germinalphrase 3y ago
  As a former teacher, my experience of education specific software was almost universally negative. We may even get an in-person meeting with the software vendor only for the reported issues to remain unpatched indefinitely.
  - dahart 3y ago
    As a developer, I’m not at all surprised, I suspect it’s negative from almost all points of view except at the district or city or state level where they pat themselves on the back for bringing technology into classrooms via spending some money, what little money they have, on tech that they don’t use and don’t understand. I don’t even have that much experience with it, but I’ve tried making educational software, in the form of multi-user whiteboards teachers could use for their classes. The people approving purchasing decisions don’t use the software, and don’t ask teachers or students about the experience or value of the software. The insane sales cycles are a never-ending nightmare of requirements and delays, far beyond the reasonable concerns about kids & privacy. After years of run-arounds, we gave up trying to use any official channels and just let teachers use the software free. As a parent, the experience was also very negative. My kids hated almost everything the school forced them to use, because the software was terrible compared to non-educational software. Teachers didn’t get any training on the software, and didn’t have the resources or time to use it effectively. Hardware was always outdated or supremely cheap and crappy. Possible the single worst software experience was the automated essay grading software. It was beyond broken, actively harmful, and it literally didn’t work. I came home from work one day and my oldest kid was in tears and my wife at her wits end trying to finish a homework assignment to write a short concise 2 paragraph essay on a specific topic and achieve an arbitrary 90% score or whatever within 10 tries, and they’d used up 8 of them and scores were going down each time. This software isn’t even supposed to be used unsupervised for grading, but the teachers didn’t seem to understand, and weren’t following recommendations. I figured out quickly the software doesn’t understand english and would score higher if you fed it more words, regardless of their quality or relevance - the polar opposite of the assignment’s goal. So I gave it a wikipedia article on why people shouldn’t use software grading robots, which scored 100%, and then scheduled a meeting with the teacher, who proceeded to complain about the software and told me she didn’t have a choice because it was district rules and a state software contract.
- cratermoon 3y ago
  There's a whole thing around the nature of who makes the purchasing decisions vs who are the real users. For key organizational systems like Bloomz for schools, the decision is made at the senior executive level, and input from the people who will use the software on a day-to-day basis is rarely, if ever, taken into consideration. Sometimes the decision is made on the golf course between business buddies. In these situations, the vendor doesn't have to care much about user experience or feedback, all they need to do is be able to check all the boxes on the whatever list of features management cares about. It comes up on HN off and on, here's an example: https://news.ycombinator.com/item?id=21224209 https://news.ycombinator.com/item?id=21224209. Thread unrolled here: https://threadreaderapp.com/thread/1182635589604171776.html https://threadreaderapp.com/thread/1182635589604171776.html
- rqtwteye 3y ago
  This plagues most enterprise software. Decisions are made by people who will most likely never actually use the software. So the decisions are made by cost and ease of dealing with the vendor. Usability is way down the list of priorities.
makerdiety 3y ago
What about free software? That is, how can human society achieve good quality free software if it is physically impossible for free software to develop? Because personal income for free software developers is important. I guess since it's personal income, then it's a personal problem for free software developers then? That's fine, but income level coincidentally correlates with high quality software. I guess free software just doesn't matter then.
- mantas 3y ago
  Correlation between in one level and quality software? Heh, lol.
- mch82 3y ago
  Options include… Volunteer. Donate. Ask the elected officials that represent you to choose open source software for government use cases. Encourage grants to fund open source. If you’re at a company, contribute back to open source.
  - makerdiety 3y ago
    Government officials for republics function to serve a greatest common denominator that doesn't even exist in reality. So there's no political representation at all for free software. Well, the greatest common denominator for political representation does exist. But it by its nature excludes free software as even a talking point. Namely that capitalism is mutually exclusive from free software. Capitalism hates free software to the point that free software cannot even literally get a word, a voice, or a conceptualization that can help refine it. That leaves us with the only realistic options for supporting quality efforts in free software being volunteer work that isn't empowered. Everything goes downhill from there. Grants and company lip service are only superficial reactions to the free software problem.
    - bdw5204 3y ago
      Capitalism is perfectly fine with FOSS software especially if it can use the idealism behind it to save on labor costs. Especially since the definitions of both "free software" and "open source" explicitly reject non-commercial licenses as heretical.
hiAndrewQuinn 3y ago
Most software exists to save someone somewhere time. I think this is the One True Metric by which we measure Software Quality. Software which sells well historically has done so by saving huge numbers of people huge amounts of time, for whatever other complicated business they're already involved in. Excel, ERP systems, and GPT-4 all do this to great commercial success. You only need to step back and wonder how many more man hours it would take to do what they currently do by hand to realise this. Software can also save the time of other developers by being simple, well-architected, reliably documented, etc. This too is an aspect of software quality. Its recursive nature can often become a competitive advantage in its own right: High quality code, can lead to faster change implementations (fixes and features alike), which can filter downward to a higher quality product for the end user.
- yarekt 3y ago
  I like the sentiment, but you can have quality, well architected software that’s pleasure to work in, and it being completely useless/obsolete to customers. Value != quality. I think software quality (and all the symptoms discussed in the post) are to do with how quickly the software can be changed: Lower tech debt results in software that’s quick to change to new requirements, or change in business domain. It’s also important to realise that just like with performance, you get diminishing returns at very high levels
  - detourdog 3y ago
    I think software quality is more related fo how little it changes while remaining useful. I see the current software environment has different stake holders all defining quality from their own perspective.
    - yarekt 3y ago
      Ah I see, quality can also mean ingenuity, but in my case as a software developer ingenious solutions don’t always present themselves. I’d settle for old boring plain solution that works well enough, but is of enough quality that it doesn’t cause fires, and is quick to change with the business
doubloon 3y ago
when you make a post about software quality but did not test it on enough browsers to realize your constant width font ascii art is not rendering correctly in firefox or chrome, so the user has to grab an invisible scrollbar to see the picture... zooming out of course doesn't work.
- adityaathalye 3y ago
  One of the little ironies of life. I know about this (and some other annoying problems), and have been meaning to fix the typography on my site for a long while. But that would draw me into a rabbit hole of some other improvements I want to make to my site maker. And that will be, well, there's an xkcd for it. Two actually. Automation: https://xkcd.com/1319/ https://xkcd.com/1319/ Efficiency: https://xkcd.com/1445/ https://xkcd.com/1445/
  - adityaathalye 3y ago
    P.S. Also, I did not tell them to make invisible scroll bars (nor have my CSS rules, as far as I can tell)!
    - mdaniel 3y ago
      I think some of the bad outcome may be due to macOS being sneaky with the scrollbar display <https://support.apple.com/guide/mac-help/change-appearance-settings-mchlp1225/mac#:~:text=Show%20scroll%20bars https://support.apple.com/guide/mac-help/change-appearance-s...>, but your style.css for sure isn't helping matters with its overflow: specifications. The tradeoff, as best I can tell, is whether macOS folks get hidden scrollbars (I don't know anything about the zoom complaint, since I'm on desktop) versus omitting the overflow: and having the asciiart clip outside of its bounding box due to some other CSS interaction that I didn't bother to track down https://www.evalapply.org/static/css/style.css#:~:text=overflow-x%3a%20hidden https://www.evalapply.org/static/css/style.css#:~:text=overf... and https://www.evalapply.org/static/css/style.css#:~:text=overflow-x%3a%20auto https://www.evalapply.org/static/css/style.css#:~:text=overf...
mojomark 3y ago
Good read (reminder) and love the reference list. Question for the OP. I suspect we're of similar tenure, and I'd love to det your perspective... I'm a mechanical systems engineer in an industry that is growing from industrial automation to true autonomy (robotics). I've always liked coding, but don't do it enough to be a serious contributer to "production quality" code. Nevertheless, since last year I've transitioned from a more mechanical/hardware design firm to a much more mathy/software engineering firm. I find myself in a role, bridging robotics hardware and software teams working on a ~10yr old codebase. We have some great engineers! I've tended to stay on the customer requirements and algorithm development/analysis side of things, but I struggle with the lack of benchmarking and testing our group does, and I'm continuously surprised at the simple bugs we find that cause application crashes. Moreover, when a developer shows me code, I'm stricken by the number of undocumented "magic numbers" that seem like forgotten patches popped in there at one time to make things work for a test or demo, and the number if/then statements I see that are holding things together. I'm also faced with a growing cry for "how can we best make use of these LLM things" (for which there are a thousand and one answers). I see/feel the suffering, and what looks to me (from my 1000ft perch) like thousands of cuts. Bottom line, I'm increasingly thinking that I really need to take a month and try tracing through the thousands of lines of C++ (which I haven't really written since grad school) in order to truly understand the state/configuration of pur code baseline so that I can provide some more explicit leadership in terms of - do we refactor everything that looks messy or the we don't 100% understand? Do we take a T.O. to build up our automated unit and system level testing? etc. As a manager of a SW team, from your experience is it a good idea for me to dig deep into the code to gain an intimate understanding? That feels an awful lot like micromanagement to me, but I will do it if that's what 'wisdom of experience' in this domain proves is the most value-add.
- kushie 3y ago
  i find myself in a similar sounding role, i don't have answers for you though. i'm wondering, do you use git for version control and repositories?
  - deleted 3y ago
    [deleted]
teucris 3y ago
Minor nit regarding this thing that will destroy quality: “We'll fix it if customers complain." You can’t fix everything, and perfect is the enemy of good. While punting testing to users is a no-no, recognizing that edge case issues can be deprioritized is crucial to getting things out the door. This means shipping stuff with known defects that you can respond to if necessary.
- marcosdumay 3y ago
  > You can’t fix everything Oh, you certainly can't. But you can choose how many things needing fixing you will make. Doing some shit without thinking and waiting for people to complain about symptoms of deep issues is a very effective way to create lots and lots of need for fixing things.
  - mdaniel 3y ago
    And my experience has been that a quality culture like that can "resource starve" annoyances for literally years due to while True: ship_feature() if bugs: to_fix = highest_priority(bugs) fix = make_patch(to_fix) fix = introduce_new_bug(fix) # defer bugs[1:] until they meet the complaint criteria and that's how meetings end up with "but, how hard can that possibly be to fix?" statements
    - NomDePlum 3y ago
      If you accept some bugs as "features" that you will accept, not fix, you can have a bug free policy. Doing this means you make decisions early and don't carry a ton of historical baggage that gets endlessly discussed.
    - hinkley 3y ago
      I knew a guy who said on a previous project they had a system that raised the priority of old bugs on some interval. I guess the idea was to either get them fixed or agree that they never would. There are morale and duplicate bug issues with keeping a backlog that grows faster than you can close them, so I guess that makes a bit of sense. And better than GitHub auto-shooting-the-messenger by clocking bugs after a month or two of inactivity.
sriharis 3y ago
This is well written, adi. I particularly like the book references.
- adityaathalye 3y ago
  Thank you :) If there is just one I would pick from the lot, it would be the Meadows book.
jiggawatts 3y ago
One thing I like about this is that the author doesn’t talk in terms of absolutes. He assumes (correctly) that advice is not universally applicable and that each process needs to be traced back to fundamental axioms or requirements. As a random example I’m right now trying to explain this to a large government enterprise customer. They write “cold blooded” software, the type where they have a development project with a fixed budget and a fixed scope and then the code goes into hibernation for five years until they can “find the budget” again. So of course, they picked the Angular framework, one of the fastest moving, most “hot blooded” frameworks in the world. Angular might be the right choice for a startup writing highly interactive software. It is definitely the wrong choice for an org that thinks of maintenance as something that occurs twice a decade. It’s these things that most fundamentally kill quality. Picking a framework, language, or even operating system because it’s “hip” or “cool” or the “buzzword of the year”. Not because it’s suitable for the project. The thousands of cuts come later, but they barely rate in comparison to having your legs chopped off at the knees right from the beginning.

154 points by adityaathalye 3y ago

So much software suffers from a large class of users having no choice in its use. For example, our local school district uses Bloomz, which is pretty terrible - both on the web, and in its native app. I can report all the bugs I want, but since I'm never the person making a purchasing decision, how highly do I think those reports are prioritized? I wonder if the teacher/administrator side of things is better.

deleted 3y ago
[deleted]
dboreham 3y ago
I found the best approach is to search LinkedIn for someone at the vendor that's closely connected to me, then compose a cheery message to that person introducing myself as a fellow developer and parent of a user, ask the best way to inject a bug report etc. This actually worked. Ymmv of course.
- kqr 3y ago
  Ohh, this is actually clever. Subversive and I'm not sure I like it, but it sounds very effective.
  - zx8080 3y ago
    Sounds like a hack to make a person to work overtime.
    - amelius 3y ago
      ... while having fun while doing that work.
    - the_fury 3y ago
      If I get a message on linkedin that pertains to a work issue, you had better believe that I'm going to be dealing with it on work time.
- computerfriend 3y ago
  One of the reasons I never put my employer on my LinkedIn profile.
  - datavirtue 3y ago
    Most of life's fulfilling opportunities come from just showing up. Showing up on searches, at work, at the pub.
  - deprecative 3y ago
    If I got a message like this I'd just block the sender. As a developer I have no say in what my owner tells me to work on. I type what they want and nothing more.
    - D-Coder 3y ago
      They're not asking you to fix anything. They are asking where to submit a bug report that someone (someone else) will look at.
      - deprecative 3y ago
        My answer doesn't change. I'm not on LinkedIn during work hours.
- binarysneaker 3y ago
  As someone this happened to, I was actually impressed they looked me up and reached out. Knowing they cared that much meant that I cared too and I was able to prioritise the request with the product team. When we completed the request, we celebrated, and even implemented uservoice on some sites to improve the feedback loop.
hitchstory 3y ago
This software also tends to be waterfalled to hell and back. Theyve already planned out their deliverables for the next 12 months and will be held accountable for meeting them. Your bugfix doesnt fit into that picture unless an exec treats it as a priority. ...in which case it'll be jammed into the devs' task queue with no expectation that the pre-agreed delivery dates will shift.
- datavirtue 3y ago
  The issues you describe have nothing to do with "waterfall."
  - hitchstory 3y ago
    >Theyve already planned out their deliverables for the next 12 months and will be held accountable for meeting them.
    - unclebucknasty 3y ago
      Not sure I understand your point. Are you saying product roadmaps are incompatible with agile?
      - hitchstory 3y ago
        If they include detailed plans and timelines for feature deliveries then yes.
        unclebucknasty 3y ago
        Agile doesn't mean "not planning" or even "vague planning".
adityaathalye 3y ago
Post author here. I agree; so often constructive feedback loops from end users to builders are woefully absent from the equation. I feel the greater the distance between the purchase decision maker and the end user, the weaker the feedback gets. Riffing off the six degrees of separation idea and exponential drop off of signals though lossy media, it's like by the time someone gets to a third degree of separation, any feedback signal barely registers. Only mass action has any chance of being heard, much less heeded.
- AnimalMuppet 3y ago
  Ah. So that's (part of) why enterprise software is so universally bad. The users are further from the purchasers than they are in, say, developer tools.
  - adityaathalye 3y ago
    I believe so. I'll pick on Government websites and the systems that back them as an all-too-familiar, internationally reviled category of software, as an example. These tend to be incredibly painful to use. Some tendering process somewhere awards a contract to the lowest bidder who then has the unenviable job of wrangling the delightful intricacies of government machinery to put something out that maybe does what it was supposed to do for the common citizen. In such a world, I feel gov.uk stands out as a remarkable counter-example, worthy of emulation. e.g. https://www.gov.uk/guidance/government-design-principles https://www.gov.uk/guidance/government-design-principles
jSully24 3y ago
You mention not being the person making a purchasing decision, who is? Can you connect with them? I would encourage you to put together a story that illustrates the pain being caused by this software and get with the purchasers. Not only might it help your organization figure out they need to do something different it could also show off your skills to the orgainization. Could you collect the time you and other people are spending working on these problems? How many issues are reported to you? How many bugs you've filed and have not had fixed by the vendor? Rolling this up into a real dollar coast it's having on you and the school district users.
- deleted 3y ago
  [deleted]
- cratermoon 3y ago
  > who is? Can you connect with them? For something like this, usually at the senior executive level, and organizationally. There's not really one person making the decision, it's done through the organization's decision-making system. Nominally there's a responsible person, but practically speaking that individual isn't completely autonomous. Large organizations have been making and dealing with decisions like this for ages, and rarely does dollar cost of usability ever influence. Products SAP and MS Teams get and retain market share by making the cost of moving off that software outweigh the gains.
- pavel_lishin 3y ago
  The person making the decision is very like the superintendent of the school system, likely informed by someone from IT and maybe someone representing the teachers somehow. They might listen to me if I bring in a coalition of other parents, but there's two reasons why I'm probably not going to do this: 1. They're likely locked into at least a year-long contract, if not more 2. Doing this is a huge amount of work for me. I'd have to find a significant number of parents who are _also_ as peeved as me, and then get them to take action - even if it's signing something - and then I'd have to bring the petition up to the school board, etc., etc., etc. It's much easier to just mute the notifications on the app, and go through them once a week. 99% of them are just wildly inactionable anyway - things like reminding me that school is out, or that a student interest group that's irrelevant to me is having a meeting two Thursdays from now. If I was annoyed enough to take action, I'd rather just offer to build them a replacement app - shorten that feedback loop, ya know?
germinalphrase 3y ago
As a former teacher, my experience of education specific software was almost universally negative. We may even get an in-person meeting with the software vendor only for the reported issues to remain unpatched indefinitely.
- dahart 3y ago
  As a developer, I’m not at all surprised, I suspect it’s negative from almost all points of view except at the district or city or state level where they pat themselves on the back for bringing technology into classrooms via spending some money, what little money they have, on tech that they don’t use and don’t understand. I don’t even have that much experience with it, but I’ve tried making educational software, in the form of multi-user whiteboards teachers could use for their classes. The people approving purchasing decisions don’t use the software, and don’t ask teachers or students about the experience or value of the software. The insane sales cycles are a never-ending nightmare of requirements and delays, far beyond the reasonable concerns about kids & privacy. After years of run-arounds, we gave up trying to use any official channels and just let teachers use the software free. As a parent, the experience was also very negative. My kids hated almost everything the school forced them to use, because the software was terrible compared to non-educational software. Teachers didn’t get any training on the software, and didn’t have the resources or time to use it effectively. Hardware was always outdated or supremely cheap and crappy. Possible the single worst software experience was the automated essay grading software. It was beyond broken, actively harmful, and it literally didn’t work. I came home from work one day and my oldest kid was in tears and my wife at her wits end trying to finish a homework assignment to write a short concise 2 paragraph essay on a specific topic and achieve an arbitrary 90% score or whatever within 10 tries, and they’d used up 8 of them and scores were going down each time. This software isn’t even supposed to be used unsupervised for grading, but the teachers didn’t seem to understand, and weren’t following recommendations. I figured out quickly the software doesn’t understand english and would score higher if you fed it more words, regardless of their quality or relevance - the polar opposite of the assignment’s goal. So I gave it a wikipedia article on why people shouldn’t use software grading robots, which scored 100%, and then scheduled a meeting with the teacher, who proceeded to complain about the software and told me she didn’t have a choice because it was district rules and a state software contract.
cratermoon 3y ago
There's a whole thing around the nature of who makes the purchasing decisions vs who are the real users. For key organizational systems like Bloomz for schools, the decision is made at the senior executive level, and input from the people who will use the software on a day-to-day basis is rarely, if ever, taken into consideration. Sometimes the decision is made on the golf course between business buddies. In these situations, the vendor doesn't have to care much about user experience or feedback, all they need to do is be able to check all the boxes on the whatever list of features management cares about. It comes up on HN off and on, here's an example: https://news.ycombinator.com/item?id=21224209 https://news.ycombinator.com/item?id=21224209. Thread unrolled here: https://threadreaderapp.com/thread/1182635589604171776.html https://threadreaderapp.com/thread/1182635589604171776.html
rqtwteye 3y ago
This plagues most enterprise software. Decisions are made by people who will most likely never actually use the software. So the decisions are made by cost and ease of dealing with the vendor. Usability is way down the list of priorities.

makerdiety 3y ago

What about free software? That is, how can human society achieve good quality free software if it is physically impossible for free software to develop? Because personal income for free software developers is important. I guess since it's personal income, then it's a personal problem for free software developers then? That's fine, but income level coincidentally correlates with high quality software. I guess free software just doesn't matter then.

mantas 3y ago
Correlation between in one level and quality software? Heh, lol.
mch82 3y ago
Options include… Volunteer. Donate. Ask the elected officials that represent you to choose open source software for government use cases. Encourage grants to fund open source. If you’re at a company, contribute back to open source.
- makerdiety 3y ago
  Government officials for republics function to serve a greatest common denominator that doesn't even exist in reality. So there's no political representation at all for free software. Well, the greatest common denominator for political representation does exist. But it by its nature excludes free software as even a talking point. Namely that capitalism is mutually exclusive from free software. Capitalism hates free software to the point that free software cannot even literally get a word, a voice, or a conceptualization that can help refine it. That leaves us with the only realistic options for supporting quality efforts in free software being volunteer work that isn't empowered. Everything goes downhill from there. Grants and company lip service are only superficial reactions to the free software problem.
  - bdw5204 3y ago
    Capitalism is perfectly fine with FOSS software especially if it can use the idealism behind it to save on labor costs. Especially since the definitions of both "free software" and "open source" explicitly reject non-commercial licenses as heretical.

hiAndrewQuinn 3y ago

Most software exists to save someone somewhere time. I think this is the One True Metric by which we measure Software Quality. Software which sells well historically has done so by saving huge numbers of people huge amounts of time, for whatever other complicated business they're already involved in. Excel, ERP systems, and GPT-4 all do this to great commercial success. You only need to step back and wonder how many more man hours it would take to do what they currently do by hand to realise this. Software can also save the time of other developers by being simple, well-architected, reliably documented, etc. This too is an aspect of software quality. Its recursive nature can often become a competitive advantage in its own right: High quality code, can lead to faster change implementations (fixes and features alike), which can filter downward to a higher quality product for the end user.

yarekt 3y ago
I like the sentiment, but you can have quality, well architected software that’s pleasure to work in, and it being completely useless/obsolete to customers. Value != quality. I think software quality (and all the symptoms discussed in the post) are to do with how quickly the software can be changed: Lower tech debt results in software that’s quick to change to new requirements, or change in business domain. It’s also important to realise that just like with performance, you get diminishing returns at very high levels
- detourdog 3y ago
  I think software quality is more related fo how little it changes while remaining useful. I see the current software environment has different stake holders all defining quality from their own perspective.
  - yarekt 3y ago
    Ah I see, quality can also mean ingenuity, but in my case as a software developer ingenious solutions don’t always present themselves. I’d settle for old boring plain solution that works well enough, but is of enough quality that it doesn’t cause fires, and is quick to change with the business

doubloon 3y ago

when you make a post about software quality but did not test it on enough browsers to realize your constant width font ascii art is not rendering correctly in firefox or chrome, so the user has to grab an invisible scrollbar to see the picture... zooming out of course doesn't work.

adityaathalye 3y ago
One of the little ironies of life. I know about this (and some other annoying problems), and have been meaning to fix the typography on my site for a long while. But that would draw me into a rabbit hole of some other improvements I want to make to my site maker. And that will be, well, there's an xkcd for it. Two actually. Automation: https://xkcd.com/1319/ https://xkcd.com/1319/ Efficiency: https://xkcd.com/1445/ https://xkcd.com/1445/
- adityaathalye 3y ago
  P.S. Also, I did not tell them to make invisible scroll bars (nor have my CSS rules, as far as I can tell)!
  - mdaniel 3y ago
    I think some of the bad outcome may be due to macOS being sneaky with the scrollbar display <https://support.apple.com/guide/mac-help/change-appearance-settings-mchlp1225/mac#:~:text=Show%20scroll%20bars https://support.apple.com/guide/mac-help/change-appearance-s...>, but your style.css for sure isn't helping matters with its overflow: specifications. The tradeoff, as best I can tell, is whether macOS folks get hidden scrollbars (I don't know anything about the zoom complaint, since I'm on desktop) versus omitting the overflow: and having the asciiart clip outside of its bounding box due to some other CSS interaction that I didn't bother to track down https://www.evalapply.org/static/css/style.css#:~:text=overflow-x%3a%20hidden https://www.evalapply.org/static/css/style.css#:~:text=overf... and https://www.evalapply.org/static/css/style.css#:~:text=overflow-x%3a%20auto https://www.evalapply.org/static/css/style.css#:~:text=overf...

mojomark 3y ago

Good read (reminder) and love the reference list. Question for the OP. I suspect we're of similar tenure, and I'd love to det your perspective... I'm a mechanical systems engineer in an industry that is growing from industrial automation to true autonomy (robotics). I've always liked coding, but don't do it enough to be a serious contributer to "production quality" code. Nevertheless, since last year I've transitioned from a more mechanical/hardware design firm to a much more mathy/software engineering firm. I find myself in a role, bridging robotics hardware and software teams working on a ~10yr old codebase. We have some great engineers! I've tended to stay on the customer requirements and algorithm development/analysis side of things, but I struggle with the lack of benchmarking and testing our group does, and I'm continuously surprised at the simple bugs we find that cause application crashes. Moreover, when a developer shows me code, I'm stricken by the number of undocumented "magic numbers" that seem like forgotten patches popped in there at one time to make things work for a test or demo, and the number if/then statements I see that are holding things together. I'm also faced with a growing cry for "how can we best make use of these LLM things" (for which there are a thousand and one answers). I see/feel the suffering, and what looks to me (from my 1000ft perch) like thousands of cuts. Bottom line, I'm increasingly thinking that I really need to take a month and try tracing through the thousands of lines of C++ (which I haven't really written since grad school) in order to truly understand the state/configuration of pur code baseline so that I can provide some more explicit leadership in terms of - do we refactor everything that looks messy or the we don't 100% understand? Do we take a T.O. to build up our automated unit and system level testing? etc. As a manager of a SW team, from your experience is it a good idea for me to dig deep into the code to gain an intimate understanding? That feels an awful lot like micromanagement to me, but I will do it if that's what 'wisdom of experience' in this domain proves is the most value-add.

kushie 3y ago
i find myself in a similar sounding role, i don't have answers for you though. i'm wondering, do you use git for version control and repositories?
- deleted 3y ago
  [deleted]

teucris 3y ago

Minor nit regarding this thing that will destroy quality: “We'll fix it if customers complain." You can’t fix everything, and perfect is the enemy of good. While punting testing to users is a no-no, recognizing that edge case issues can be deprioritized is crucial to getting things out the door. This means shipping stuff with known defects that you can respond to if necessary.

marcosdumay 3y ago
> You can’t fix everything Oh, you certainly can't. But you can choose how many things needing fixing you will make. Doing some shit without thinking and waiting for people to complain about symptoms of deep issues is a very effective way to create lots and lots of need for fixing things.
- mdaniel 3y ago
  And my experience has been that a quality culture like that can "resource starve" annoyances for literally years due to while True: ship_feature() if bugs: to_fix = highest_priority(bugs) fix = make_patch(to_fix) fix = introduce_new_bug(fix) # defer bugs[1:] until they meet the complaint criteria and that's how meetings end up with "but, how hard can that possibly be to fix?" statements
  - NomDePlum 3y ago
    If you accept some bugs as "features" that you will accept, not fix, you can have a bug free policy. Doing this means you make decisions early and don't carry a ton of historical baggage that gets endlessly discussed.
  - hinkley 3y ago
    I knew a guy who said on a previous project they had a system that raised the priority of old bugs on some interval. I guess the idea was to either get them fixed or agree that they never would. There are morale and duplicate bug issues with keeping a backlog that grows faster than you can close them, so I guess that makes a bit of sense. And better than GitHub auto-shooting-the-messenger by clocking bugs after a month or two of inactivity.

sriharis 3y ago

This is well written, adi. I particularly like the book references.

adityaathalye 3y ago
Thank you :) If there is just one I would pick from the lot, it would be the Meadows book.

jiggawatts 3y ago

One thing I like about this is that the author doesn’t talk in terms of absolutes. He assumes (correctly) that advice is not universally applicable and that each process needs to be traced back to fundamental axioms or requirements. As a random example I’m right now trying to explain this to a large government enterprise customer. They write “cold blooded” software, the type where they have a development project with a fixed budget and a fixed scope and then the code goes into hibernation for five years until they can “find the budget” again. So of course, they picked the Angular framework, one of the fastest moving, most “hot blooded” frameworks in the world. Angular might be the right choice for a startup writing highly interactive software. It is definitely the wrong choice for an org that thinks of maintenance as something that occurs twice a decade. It’s these things that most fundamentally kill quality. Picking a framework, language, or even operating system because it’s “hip” or “cool” or the “buzzword of the year”. Not because it’s suitable for the project. The thousands of cuts come later, but they barely rate in comparison to having your legs chopped off at the knees right from the beginning.