4 ms·

It’s worth remembering that engineers don’t get paid to write tests, they get paid to produce software that supports some business need. In most circumstances,

by bww 7y ago

It’s worth remembering that engineers don’t get paid to write tests, they get paid to produce software that supports some business need. In most circumstances, some amount of automated testing makes it faster to reliably meet those business needs. If you’re building a lot of small, one-off tools for internal use, it may well be the case that some limited manual QA or UAT is sufficient to ensure that your work is good enough. If you’re working on larger, more complex projects that are frequently updated, the shorter feedback loop that some amount of automated tests provide will probably save time and money by catching problems earlier, avoiding regressions, and reducing the need for repetitive, time-intensive manual testing. But in any case, your testing needs will always be highly specific to the actual nature and needs of the project.

jacques_chester 7y ago
> It’s worth remembering that engineers don’t get paid to write tests, they get paid to produce software that supports some business need. I view it differently. We get paid to solve business problems, most often -- but not always -- with software. Software that's broken in preventable ways doesn't solve business problems, it just creates new ones. Outcomes, not output.
- BoiledCabbage 7y ago
  > Software that's broken in preventable ways doesn't solve business problems, it just creates new ones. I think you may be missing OPs point. Software "that's broken in preventable ways" very frequently does solve business problems. And solves them well. That's why the industry works the way it does w.r.t. testing. 6 months to deliver 2 applications that each work 80% often provides more business value than 6 months to provide one application that works 100%. More functionality almost always is more valuable to a business - including warts and all.
  - jacques_chester 7y ago
    I wasn't defining "broken" as "the engineers find it distasteful". I was defining it as "broken". I agree with this: > 6 months to deliver 2 applications that each work 80% often provides more business value than 6 months to provide one application that works 100%. Though this is a question of product management prioritisation, not engineering practice. "It's faster to not write tests" depends entirely on a definition of completion that ignores the cost of repairing defects and coping with faults and failures in production.
    - BoiledCabbage 7y ago
      > I wasn't defining "broken" as "the engineers find it distasteful". I was defining it as "broken". If a piece of software literally will not launch period, then it is broken. If it can do pretty much anything more than that, then it "works to some degree", and just has some number of scenarios that are broken. And my point is that if it has 80% of scenarios that work and 20% of scenarios that are completely functionally broken (ie do not work at all for the customer) that still provides business value. That's the point I'm making. It's not ideal, but in the real world half broken software is immensely valuable. > Software that's broken in preventable ways doesn't solve business problems, it just creates new ones. So software that is broken in preventable ways, absolutely does solve business problems. And just about the entire software industry is proof of this.
  - cosmie 7y ago
    To expand on the internal tool case from the GP: If your software is designed to support, augment, or automate some sort of internal need for non-IT teams, there's a significant chance that there's already an inefficient, error-prone process in place as-is. Because even without your software, they still have to get their job done. And they're used to having a "good enough" mindset because they're used to making due with whatever tools/capabilities they have, regardless of fit. As long as your software addresses their needs[1] and isn't any more error prone than whatever they're currently doing, they'll be happy. That said, whenever possible you do want to code your failure conditions or edge cases to fail hard and visibly or expose visible checks for the user. If it covers 80% of their needs, but they have to fall back to their old process for the other 20% of their needs, business users are generally happy with that. But if it pretends to work 100% of the time, but subtly and silently causes problems 20% of the time (and they don't know what 20%), then they'll be distrustful of it and resist (or resent) adopting it. For externally-exposed software, I'd be a bit more wary of not having any tests at all. But for internal tools, I've seen far more systems without tests than with them.
chriswarbo 7y ago
> engineers don’t get paid to write tests, they get paid to produce software I've heard this double-standard before, and it sounds plausible at first glance, but is quite demonstrably nonsense. 1) Automated tests are software 2) Engineers aren't paid to produce software (nobody would pay me to auto-generate a billion "hello world" programs per second). Engineers are paid to solve problems and/or create value. Tests solve problems (finding bugs, demonstrate usage, clarify thinking, document intent, etc.). 3) Software is a liability, not an asset. We should produce as little as needed, and delete it if possible. 4) Engineers aren't paid to perform manual testing (running commands over and over, clicking around web forms to see what happens, etc.)
- mempko 7y ago
  I find #4, exploratory testing far more bang for buck than automated testing. I would argue engineers are paid to make services (software is a service) and exploratory testing is far more valuable towards that goal than automated tests. a) users do crazy things. b) eating your own dogfood is worth it.
  - chriswarbo 7y ago
    If by "exploratory testing" you mean things like clicking around on a site manually, then sure it can be useful to spend a couple of minutes looking for anything dodgy. That doesn't mean engineers are paid to go through a list of regressions and edge-cases, look for broken links, double-check calculations with pen-and-paper, etc. All of that can be done across an entire site automatically in seconds; there's no excuse not to. > a) users do crazy things Again, thinking we can check such things better than a computer is misguided. Test frameworks like QuickCheck can stress-test a system with far more strange inputs/transitions than I could hope to think of.
    - gitgud 7y ago
      Totally agree, any repeatable action can/should be automated. It hurts to see software engineers clicking login to test it still works.... Every day

by bww 7y ago

jacques_chester 7y ago

> It’s worth remembering that engineers don’t get paid to write tests, they get paid to produce software that supports some business need. I view it differently. We get paid to solve business problems, most often -- but not always -- with software. Software that's broken in preventable ways doesn't solve business problems, it just creates new ones. Outcomes, not output.

BoiledCabbage 7y ago
> Software that's broken in preventable ways doesn't solve business problems, it just creates new ones. I think you may be missing OPs point. Software "that's broken in preventable ways" very frequently does solve business problems. And solves them well. That's why the industry works the way it does w.r.t. testing. 6 months to deliver 2 applications that each work 80% often provides more business value than 6 months to provide one application that works 100%. More functionality almost always is more valuable to a business - including warts and all.
- jacques_chester 7y ago
  I wasn't defining "broken" as "the engineers find it distasteful". I was defining it as "broken". I agree with this: > 6 months to deliver 2 applications that each work 80% often provides more business value than 6 months to provide one application that works 100%. Though this is a question of product management prioritisation, not engineering practice. "It's faster to not write tests" depends entirely on a definition of completion that ignores the cost of repairing defects and coping with faults and failures in production.
  - BoiledCabbage 7y ago
    > I wasn't defining "broken" as "the engineers find it distasteful". I was defining it as "broken". If a piece of software literally will not launch period, then it is broken. If it can do pretty much anything more than that, then it "works to some degree", and just has some number of scenarios that are broken. And my point is that if it has 80% of scenarios that work and 20% of scenarios that are completely functionally broken (ie do not work at all for the customer) that still provides business value. That's the point I'm making. It's not ideal, but in the real world half broken software is immensely valuable. > Software that's broken in preventable ways doesn't solve business problems, it just creates new ones. So software that is broken in preventable ways, absolutely does solve business problems. And just about the entire software industry is proof of this.
- cosmie 7y ago
  To expand on the internal tool case from the GP: If your software is designed to support, augment, or automate some sort of internal need for non-IT teams, there's a significant chance that there's already an inefficient, error-prone process in place as-is. Because even without your software, they still have to get their job done. And they're used to having a "good enough" mindset because they're used to making due with whatever tools/capabilities they have, regardless of fit. As long as your software addresses their needs[1] and isn't any more error prone than whatever they're currently doing, they'll be happy. That said, whenever possible you do want to code your failure conditions or edge cases to fail hard and visibly or expose visible checks for the user. If it covers 80% of their needs, but they have to fall back to their old process for the other 20% of their needs, business users are generally happy with that. But if it pretends to work 100% of the time, but subtly and silently causes problems 20% of the time (and they don't know what 20%), then they'll be distrustful of it and resist (or resent) adopting it. For externally-exposed software, I'd be a bit more wary of not having any tests at all. But for internal tools, I've seen far more systems without tests than with them.

chriswarbo 7y ago

> engineers don’t get paid to write tests, they get paid to produce software I've heard this double-standard before, and it sounds plausible at first glance, but is quite demonstrably nonsense. 1) Automated tests are software 2) Engineers aren't paid to produce software (nobody would pay me to auto-generate a billion "hello world" programs per second). Engineers are paid to solve problems and/or create value. Tests solve problems (finding bugs, demonstrate usage, clarify thinking, document intent, etc.). 3) Software is a liability, not an asset. We should produce as little as needed, and delete it if possible. 4) Engineers aren't paid to perform manual testing (running commands over and over, clicking around web forms to see what happens, etc.)

mempko 7y ago
I find #4, exploratory testing far more bang for buck than automated testing. I would argue engineers are paid to make services (software is a service) and exploratory testing is far more valuable towards that goal than automated tests. a) users do crazy things. b) eating your own dogfood is worth it.
- chriswarbo 7y ago
  If by "exploratory testing" you mean things like clicking around on a site manually, then sure it can be useful to spend a couple of minutes looking for anything dodgy. That doesn't mean engineers are paid to go through a list of regressions and edge-cases, look for broken links, double-check calculations with pen-and-paper, etc. All of that can be done across an entire site automatically in seconds; there's no excuse not to. > a) users do crazy things Again, thinking we can check such things better than a computer is misguided. Test frameworks like QuickCheck can stress-test a system with far more strange inputs/transitions than I could hope to think of.
  - gitgud 7y ago
    Totally agree, any repeatable action can/should be automated. It hurts to see software engineers clicking login to test it still works.... Every day