5 ms·

So is reviewing and verifying code. Maybe not twice as "hard" if you're skilled in such things. But most programmers I've worked with can't be bothered to write

So is reviewing and verifying code. Maybe not twice as "hard" if you're skilled in such things. But most programmers I've worked with can't be bothered to write tests let alone verify correctness by other means (good tests, property tests, types, model checking, etc). It's one thing to point out small trivialities like initialization and life time issues in a small piece of code. But it's quite another to prove they don't exist in a large code base. Kernigan is a good source of quotes and thinking on programming.

eweise 9mo ago
I haven't worked in a codebase in 20 years that didn't have some sort of tests.
- simonw 9mo ago
  Out of interest, what language ecosystems do you tend to work in? My guess is that some languages - like Go - have a more robust testing culture than other languages like PHP.
  - jama211 9mo ago
    Not who you asked but I think it comes down to risk/reward. The consequences of some user finding a big in most websites is low, compared to the risk of an astronaut finding a bug the hard way whilst attempting re-entry. There is genuinely a reasonable and rational argument to “testing requires more effort than fixing the issues as users find them” if the consequences are low. See video games being notorious for this. So, industry is more important than language I’d say.
    - simonw 9mo ago
      I don't see testing as a quality thing any more, I see it as a developer productivity thing. If my project has tests I can work so much faster on it, because I can confidently add tests and refactor and know that I didn't break existing functionality. You gotta pay that initial cost to to get the framework in place though. That takes early discipline.
      - smackeyacky 9mo ago
        It’s a lot faster and easier than it used to be. Things like xUnit in the .net world make setting up tests friction free to the point where I question a codebase that doesn’t have some kind of basic unit tests. It doesn’t make mock testing or integration testing easier but I would argue if you know the base code and logic is sound those tests are less relevant. One thing I found is that if testing is easy, your code structure does change a bit to aid with a “test first” approach and I don’t hate it. I thought it made me slower but it doesn’t, it ensures that when all the ground work is finished, the gnarly part of wiring everything up goes much faster.
        simonw 9mo ago
        Yeah I've found the same, having good test discipline influences my code design in a positive way because code that's easier to test is also code that's easier to integrate and understand.
      - GrumpyYoungMan 9mo ago
        Developer testing is checking whether the code does what the developer themself thinks it should. QA testing is checking whether the code does what the customers / users / rest of the world thinks it should.
      - jama211 9mo ago
        Depends again on the type of work, and it absolutely has something to do with quality. You’re caring about not breaking existing functionality, you’re refactoring, you’re moving forward with confidence in your code. That’s absolutely a quality thing. I can assure you that you could move a lot faster if you didn’t try and meet such standards, not that it’d be a good idea necessarily, but in isolation it proves the point.
hackthemack 9mo ago
I am fascinated by the prevalence of wanting "tests" from hacker news comments. Most of the code I have worked on in the past 20 years did not have tests. Most of it was shopping carts, custom data transformation code, orchestrating servers, plugin code functionality to change some aspect of a website. Now, I have had to do some salesforce apex coding and the framework requires tests. So I write up some dummy data of a user and a lead and pass it through the code, but it feels of limited value, almost like just additional ceremony. Most of the bugs I see are from a misconception of different users about what a flag means. I can not think of a time a test caught something. The organization is huge and people do not go and run all the code every time some other area of the system is changed. Maybe they should? But I doubt that would ever happen given the politics of the organization. So I am curious, what are the kinds of tests do people write in other areas of the industry?
- bluGill 9mo ago
  The value of tests is when the fail they show you of something you broke that you didn't realize. 80% (likely more, but I don't know how to measure) of the tests I write could safely be thrown away because they fail again - but I don't know which tests will fail and thus inform me that I broke things. The system I'm working on has been in production for 12 years - we have added a lot of new features over those years. Many of those needed us to hook into existing code, tests help us know that we didn't break something that used to work. Maybe that helps answer the question of why they are important to me. They might not be to your problems.
- tjr 9mo ago
  what are the kinds of tests do people write in other areas of the industry? Aerospace here. Roughly this would be typical: - comprehensive requirements on the software behavior, with tests to verify those requirements. Tests are automated as much as possible (e.g., scripts rather than manual testing) - tests are generally run first in a test suite in a completely virtual software environment - structural coverage analysis (depending on level of criticality) to show that all code in the subsystem was executed by the testing (or adequately explain why the testing can't hit that code) - then once that passes, run the same tests in a hardware lab environment, testing the software as it runs on the the actual physical component that will be installed on the plane - then test that actually on a plane, through a series of flight tests. (The flight testing would likely not be as entirely comprehensive as the previous steps) A full round of testing is very time-consuming and expensive, and as much as possible should be caught and fixed in the virtual software tests before it even gets to the hardware lab, much less to the plane.
tom_m 9mo ago
Absolutely. Well said.

eweise 9mo ago

I haven't worked in a codebase in 20 years that didn't have some sort of tests.

simonw 9mo ago
Out of interest, what language ecosystems do you tend to work in? My guess is that some languages - like Go - have a more robust testing culture than other languages like PHP.
- jama211 9mo ago
  Not who you asked but I think it comes down to risk/reward. The consequences of some user finding a big in most websites is low, compared to the risk of an astronaut finding a bug the hard way whilst attempting re-entry. There is genuinely a reasonable and rational argument to “testing requires more effort than fixing the issues as users find them” if the consequences are low. See video games being notorious for this. So, industry is more important than language I’d say.
  - simonw 9mo ago
    I don't see testing as a quality thing any more, I see it as a developer productivity thing. If my project has tests I can work so much faster on it, because I can confidently add tests and refactor and know that I didn't break existing functionality. You gotta pay that initial cost to to get the framework in place though. That takes early discipline.
    - smackeyacky 9mo ago
      It’s a lot faster and easier than it used to be. Things like xUnit in the .net world make setting up tests friction free to the point where I question a codebase that doesn’t have some kind of basic unit tests. It doesn’t make mock testing or integration testing easier but I would argue if you know the base code and logic is sound those tests are less relevant. One thing I found is that if testing is easy, your code structure does change a bit to aid with a “test first” approach and I don’t hate it. I thought it made me slower but it doesn’t, it ensures that when all the ground work is finished, the gnarly part of wiring everything up goes much faster.
      - simonw 9mo ago
        Yeah I've found the same, having good test discipline influences my code design in a positive way because code that's easier to test is also code that's easier to integrate and understand.
    - GrumpyYoungMan 9mo ago
      Developer testing is checking whether the code does what the developer themself thinks it should. QA testing is checking whether the code does what the customers / users / rest of the world thinks it should.
    - jama211 9mo ago
      Depends again on the type of work, and it absolutely has something to do with quality. You’re caring about not breaking existing functionality, you’re refactoring, you’re moving forward with confidence in your code. That’s absolutely a quality thing. I can assure you that you could move a lot faster if you didn’t try and meet such standards, not that it’d be a good idea necessarily, but in isolation it proves the point.

hackthemack 9mo ago

I am fascinated by the prevalence of wanting "tests" from hacker news comments. Most of the code I have worked on in the past 20 years did not have tests. Most of it was shopping carts, custom data transformation code, orchestrating servers, plugin code functionality to change some aspect of a website. Now, I have had to do some salesforce apex coding and the framework requires tests. So I write up some dummy data of a user and a lead and pass it through the code, but it feels of limited value, almost like just additional ceremony. Most of the bugs I see are from a misconception of different users about what a flag means. I can not think of a time a test caught something. The organization is huge and people do not go and run all the code every time some other area of the system is changed. Maybe they should? But I doubt that would ever happen given the politics of the organization. So I am curious, what are the kinds of tests do people write in other areas of the industry?

bluGill 9mo ago
The value of tests is when the fail they show you of something you broke that you didn't realize. 80% (likely more, but I don't know how to measure) of the tests I write could safely be thrown away because they fail again - but I don't know which tests will fail and thus inform me that I broke things. The system I'm working on has been in production for 12 years - we have added a lot of new features over those years. Many of those needed us to hook into existing code, tests help us know that we didn't break something that used to work. Maybe that helps answer the question of why they are important to me. They might not be to your problems.
tjr 9mo ago
what are the kinds of tests do people write in other areas of the industry? Aerospace here. Roughly this would be typical: - comprehensive requirements on the software behavior, with tests to verify those requirements. Tests are automated as much as possible (e.g., scripts rather than manual testing) - tests are generally run first in a test suite in a completely virtual software environment - structural coverage analysis (depending on level of criticality) to show that all code in the subsystem was executed by the testing (or adequately explain why the testing can't hit that code) - then once that passes, run the same tests in a hardware lab environment, testing the software as it runs on the the actual physical component that will be installed on the plane - then test that actually on a plane, through a series of flight tests. (The flight testing would likely not be as entirely comprehensive as the previous steps) A full round of testing is very time-consuming and expensive, and as much as possible should be caught and fixed in the virtual software tests before it even gets to the hardware lab, much less to the plane.

tom_m 9mo ago

Absolutely. Well said.