7 ms·

Serious question, I've been programming 15 odd years, somewhere around a million people used my code last year, I've written entire sites on my own that happily

Serious question, I've been programming 15 odd years, somewhere around a million people used my code last year, I've written entire sites on my own that happily run today. I don't write tests. Projects that already had tests, those tests caught a single bug in about 3.5 years of working on that code base. Yes, 1 bug. We'd have picked it up anyway when actually testing it. Why do you feel #1 or #16 are actually true? Am I (and colleagues) a magic snowflake, or are tests a massive waste of time, or is it somewhere in the middle?

youeseh 8y ago
It comes down to the cost of failure. If the cost of failure for a particular module say, payments processing or rocket guidance system, is high then writing tests could be much cheaper than waiting for a failure, especially in production.
pathseeker 8y ago
>I've written entire sites on my own that happily run today. This is probably the key issue. You're either working by yourself or you are on small enough team projects that it's easy to recognize when you are breaking things with a change. Tests are about rapidly detecting regressions in a programmatic way. It should not require manually testing every time you want to make a small change to the code base and if you depend on reviewers seeing that a change will break something, you are in for a world of hurt on any large project. Every large project I've been on that has skipped tests has ended up turning into a shitshow where master is broken for days at a time and you can't even tell the cause without hours of manual testing with bisection.
- beokop 8y ago
  > if you depend on reviewers seeing that a change will break something, you are in for a world of hurt on any large project. I don’t agree with this at all. Here’s a counter example: Linux. I think the real problem here is that people don’t take review seriously and/or just don’t care about the project. We build these complicated, expensive systems that do all the thinking for us, not because it’s better, but because we simply don’t care. Having tests is better than letting sloppy developers run amok. In my experience though tests aren’t really needed on projects with a maintainer that actually gives a damn.
  - y4mi 8y ago
    > I don’t agree with this at all. Here’s a counter example: Linux. are you sure about that? https://stackoverflow.com/questions/3177338/how-is-the-linux-kernel-tested https://stackoverflow.com/questions/3177338/how-is-the-linux...
  - pathseeker 8y ago
    Bad example because a massive chunk of linux is independent systems that have no overlap at all. The netlink code has jack-shit to do with a graphics driver from intel, etc. Within subsystems regressions sneak in all of the time that don't directly cause compile-time failures. http://linux-test-project.github.io/ http://linux-test-project.github.io/
guscost 8y ago
Many if not most tests are a waste of time (especially when the same guarantees are provided by, say, a type system). That said, automated tests catch a problem that would make it to production just about weekly in my experience. If you're trying to do any kind of HA/CI/CD they're really really helpful. Beyond that, isolated "unit tests" are somewhat helpful too in my experience, but only so far as they allow you to easily run your test suite (and rarely, find bugs faster).
mh8h 8y ago
Writing tests makes me write better software. The resulting test suite is just a nice side effect.
doubleunplussed 8y ago
I'm not a professional programmer, but I think it comes down to whether you expect to ever do major refactoring. If requirements change substantially, would you write new code, or would you modify the existing code and attempt to maintain some kind of backward compatibility? If the latter, tests can be worth it. I have a library that never had any tests until I wanted to re-architect it in a fundamental way whilst porting it from Python 2 to Python 3. Since it was a wire-protocol type of thing, the Python bytes–unicode distinction made this hard. So I wrote tests for each piece of functionality as I ported that piece. I keep the tests around nowadays even though the porting is complete, but am kind of lazy writing ones for new functionality. Of course tests also seem really important when you're accepting 3rd party contributions to your codebase. I've made what I thought were totally innocuous pull requests to projects only to see in the CI tests that my code causes regressions.
jrvarela56 8y ago
Broadly speaking, there are several steps in the development process where defects are caught: 1. During development: you code something, run it, see something fail and fix it. Most people don't consider this part of QA because it's coding/debugging. 2. Code review: a teammate reads your code and catches a bug or an opportunity for a bug to arise. 3. Manual QA: a teammate tries the new feature, finds a defect and reports it. 4. Production: a user reports the error. 5. Monitoring: tools like Sentry may warn you before a user reports the issue. With your experience, you probably have learned to avoid patterns in your code that make #1 occur often. Maybe you use static typing, linters, an IDE or some other tools to help you avoid silly mistakes. If you perform code review (#2), then you help less-experienced teammates avoid mistakes even if they haven't learned to avoid error-prone patterns or don't use tooling like you do. They spend more time in #1 than you do. If you care (above average for developer standards) about your job, you probably manually test the software you make to see how your users experience your creation (#3) before checking-in code, while code-reviewing, and/or afterwards to help testers. This takes time and if you find mistakes, that means other teammates that did this before you invested time also. If your company releases progressively (using feature flags) you get info from users (#4) and fix before it's considered an issue. Also, if using monitoring tools (#5) you may avoid users ever noticing there was an issue. These all lead to rework and are risky. I don't have enough information about what process you follow, so I tried making references to generic practices. In theory, writing tests first reduces the amount of time reacting in situations 1-5. I have felt the benefits in my own team and seen them when coaching other teams. If none of what I have written applies to you and your team, please let me know. Would love to buy you all a meal to hear about what you do to avoid defects in your software. I'm not being snarky here. Have genuinely been asking myself how do these developers I read about online manage to ship bug-free software really fast without writing tests.
- mattmanser 8y ago
  It's probably a combination of #3 (I always do that now, use the feature I just made/improved/fixed) and that I always make sure I can reliably repro a bug before I try and fix the bug, so I always know it's fixed. Also a significant majority of my code has been statically typed as you say and I'm moving even more over to static typing now I'm using TS over JS in all new code.
zeroname 8y ago
> Am I (and colleagues) a magic snowflake, or are tests a massive waste of time, or is it somewhere in the middle? Tests are definitely a massive waste of time if you never need them. Write and forget code exists and can certainly run happily for years. On the other hand, frequently changing something in a large existing codebase, especially without static typing, is almost certainly going to end up more expensive without automated tests.
fredifrum 8y ago
You are able to go without tests because you're writing entire apps on your own. You know what each file, class, and module is supposed to do, and how to test it for regressions manually. If you're working on a large team with a large suite of applications, this isn't the case. Very often I'll need to make changes to code that I don't explicitly own, or that I've never seen before today. If this code doesn't have tests, this is a recipe for disaster. I don't really understand what this code is supposed to do. How do I know that my change didn't break any existing functionality? Is it obvious how or even possible to test the functionality in this file manually? Maybe it's a background job that processes a file off of an SFTP server, how long will it take me to set up a manual test for that? It's also about iteration time. Automated tests allow to you check the correctness of every single change nearly instantaneously. I don't want to need switch to a browser and wait for a form to submit just to check that my code still works after a minor refactor, or to ensure that my typo was fixed successfully. Tests mean that code is much more likely to be refactored often, and will lead to much cleaner and easier to maintain codebases.
- candiodari 8y ago
  This argument, and most of the others at this same level, have a glaring error. They only justify integration tests. They do not justify unit tests (in nearly all cases). Unit tests wouldn't catch the examples you gave: > How do I know that my change didn't break any existing functionality? You don't. And with many unit tests ... you still don't. A software engineer with a few years experience will know this. > Maybe it's a background job that processes a file off of an SFTP server, how long will it take me to set up a manual test for that? This especially, no unit test is ever going to catch. This behavior would depend on so many things: encryption, networking, scheduling, starvation, graceful error handling, retry logic ... Not something any amount of unit tests could reasonably verify. So this must be an argument against unit tests ? And yet, I don't think so. In practice, you see the opposite. These arguments are designed to force people to write unit tests, and actually used against integration tests (which are complex tests that require tweaking in intelligent ways every time you make a serious change). Of course integration tests and "instantaneous" are usually opposites. So I don't understand this argument. It only makes sense on the surface. It does not actually work on real large code projects. So I don't understand that you make this argument. And I especially do not understand that nearly every large company follows advice like this. I mean, I can come up with cynical reasons: "more pointless work with zero verifiable results, but nice "quantifiable" (if meaningless) numbers ! Of course we're in favor !". But really, I cannot come up with a coherent to explain the behavior seen in developers in a large company. Most statistics about unit tests make no sense. Coverage for instance, says nothing about how many edge cases you handle. Says nothing about many guarantees (like bounded and reasonable allocation for "weird" inputs, same with cpu usage, that it still works together with the OS and other daemons on the system it'll be installed on, ...). Since tests have no "real" purpose for software (they're code that doesn't run) number of tests is meaningless too. Number of lines of test code ... that's beyond useless. The bigger a piece of software, the more useless unit tests become and the more critical realistic integration tests become ... and the less likely you are to actually find them. And frankly, for small pieces of code, I find that specialized ways to "test" them that provide actual guarantees of correctness (like Coq) actually work. Unit tests ... never catch edge cases, not even with the best of developers. Of course, even Coq sort-of only makes sure you KNOW about every edge case and requires you to verify it's correctness before letting you pass, so bugs in your understanding of the problem will still be bugs. But at least you have a theoretical guarantee that a developer actually thought about every edge case. Also, it's pretty hard to use. The language could be clearer, but that's not really the problem. The problem is that Coq WILL show you that no matter how well you think you understand a problem, your understanding is not complete. Often in surprising, interesting ways.
mixmastamyk 8y ago
As always, "it depends." Testing is usually about larger teams, cost of failure, and sleeping better at night during refactors. If the project is small, fits in one skull, and not critical enough to be maintained/redesigned for years on end tests buy you less. Also, technology can prevent some classes of problems, such as type checkers, linters, etc. One hopes you are at least using these. #1 gives the impression the author is using a dynamic "scripting" language, which may require a higher ratio of tests per line as a project matures.
JamesBarney 8y ago
I've found the sweet spot for tests are - complex code that will change with definable inputs/outputs that won't - large projects I've found they're not nearly as helpful in - small projects - UI intensive code - code where the requirements are ill defined and change often
deleted 8y ago
[deleted]
vharuck 8y ago
>We'd have picked it up anyway when actually testing it. Why not automate the tests you often run?
dougk16 8y ago
I'm in the same boat and have thought a lot about it. The biggest correlation I see from automated testing advocates is that they are usually using dynamic languages for non-trivial core business logic. I mean that's pretty much it. If you're using a dynamic language for such things, like anything past 10k lines of code, then I agree, you pretty much need automated tests since they're partially filling in the role of a compiler and type system. It begs the question of why to use dynamic languages for tricky business logic, but that's a whole different flamewar! If the testing advocates are using something with even a basic compiler/type-system (e.g. Java), then IME it often comes down to not being fluent with the design patterns, tricks, IDE shortcuts/plugins, tools, etc. that compilers and type-systems can provide to work with you to catch bugs at build time. I mention Java because it's a language that allows you to keep writing "dynamic" code and avoid the compiler/type-system to a good degree if you want (and boy do people want to). E.g. passing top-level Object types around and casting, using HashMaps instead of a proper Class, heavy reflection use, "stringly" typing, etc. Unlike say Haskell or Rust which force correctness down your throat to a much higher degree. Obviously no matter the language choice, their comes a point where automated testing becomes cost-effective (e.g. safety-critical software). At that point however there should be formal methods and such in play also. And I'd say for the majority of us working on CRUD apps, we're not at that cost-effectiveness point.
- rorykoehler 8y ago
  Testing is for business logic not for seeing if the project will build.
- arkh 8y ago
  Currently you read a lot about unit testing. Usually used badly (testing solo methods or classes). What you want is E2E tests: test that your product conforms to its specifications. Automate the test procedure your testers have.
- pathseeker 8y ago
  Type testing can't catch business logic regressions.
contingencies 8y ago
Test interfaces, not implementations.
Mc_Big_G 8y ago
Tests fail all the time during development. Test fails indicate a bug. You fix the bug before you commit. The bug never makes it to prod. It's impossible that only one test ever failed during development.
rorykoehler 8y ago
God help the person who has to pick up after you, do refactoring etc etc. Not writing tests is unprofessional as it introduces risk. If code is being used by 1 person or is some non-critical internal application then fine. If it is being used by 1 million people to earn a living the people responsible should lose their jobs. I spend a significant amount of my time tidying up after people who don't write tests and leave a mess and it's a nightmare (but keeps my family fed, housed and clothed so...).
arkh 8y ago
How much maintenance have those sites seen? I usually focus on end-to-end tests: when I had a new feature or do some refactoring I don't have to try things manually. I just launch my test suite go prepare some coffee and if the suite passes I'm confident in not having broken anything. Also writing new tests for new features helps thinking about the edge-cases you'll encounter. When someone reports a bug I reproduce it. Then I write a test reproducing the problem. And at last I correct it. Now I know 6 months or one year from now I won't have a regression and see this bug coming back. People really hate when something they reported months ago is a problem again.
- mattmanser 8y ago
  Huge amounts, you don't have a million users for failed projects. I still maintain various code bases for multiple clients that are over 10 years old, at varying levels of TLOC (10k, 100k, million k). The truth is bugs almost never come back like that. I know because of the thousands, maybe 10s of thousand, of bugs I've fixed over the years.
HereBeBeasties 8y ago
Your code is full of bugs, you just haven't noticed yet. Whether that matters or not (to you or your end users) is your grey ground in the middle. It's possible you've only ever worked on things that are simple or obvious enough to not have the need for tests, and that code that never changes. Especially if that's UI code, where you're testing it manually as you write it, you may be able to live with this on small code bases. If you have never worked in an environment where the benefit of at least some level of automated testing is not blindingly obvious, my advice to you is to stop doing the same thing over and over and find a new job.
- mattmanser 8y ago
  Projects with unit tests are also riddled with bugs. I have absolutely no respect for your attitude or your comment. Sanctimonious zealotry like yours should have no place in our industry. It's disgusting that you think you can tell someone they're shit or trivialize their work because you disagree with their workflow.
  - HereBeBeasties 8y ago
    I hardly think that encouraging someone to seek out alternative positions from which to view the world, and a suggestion that there is a grey area for all this constitutes zealotry. Quite the opposite, in fact. And in my experience projects with unit tests are far _less_ riddled with bugs, which is the whole point, no? And I don't know understand why on earth you think I'm telling anyone they are "shit". I'd advise someone to broaden their horizons if they said they thought functional programming was worthless, or that assembly language was always pointlessly low level, or any one of a number of possible assertions that wouldn't sit well with me. That is far from "you are shit" - it's much more assuming they are not shit, but haven't had the right context yet.
suraj 8y ago
When I was a rookie programmer 12 years ago, I was assigned a task to optimise a routine. It took current vehicle speed as input, performed integration and calculated total distance traveled. I made a mistake that was let through code review and was caught by an automated test. My calculations were off by 1% which accumulated to a larger amount in 10 minutes. After that incident we never had another bug for 5 years in that module. Does this make the tests invalid? Our does this improve confidence in our code?
azdacha 8y ago
There is something in the middle actually. Just write test for the hard tasks or the algos that you implemented your self. (Some stuff that I don't undertand end up much easier to write once I've created some tests to help me iterate faster) Testing basic class re-use (I'm thinking about serializers, or ViewSets like in django) doesn't make much sense to me, and in fact was 100% automatised. Those "simple" test end-up being useful when you start hitting your app with the hammer to change a lot of stuff in it. I havn't had any use for these otherwhise.

youeseh 8y ago

It comes down to the cost of failure. If the cost of failure for a particular module say, payments processing or rocket guidance system, is high then writing tests could be much cheaper than waiting for a failure, especially in production.

pathseeker 8y ago

>I've written entire sites on my own that happily run today. This is probably the key issue. You're either working by yourself or you are on small enough team projects that it's easy to recognize when you are breaking things with a change. Tests are about rapidly detecting regressions in a programmatic way. It should not require manually testing every time you want to make a small change to the code base and if you depend on reviewers seeing that a change will break something, you are in for a world of hurt on any large project. Every large project I've been on that has skipped tests has ended up turning into a shitshow where master is broken for days at a time and you can't even tell the cause without hours of manual testing with bisection.

beokop 8y ago
> if you depend on reviewers seeing that a change will break something, you are in for a world of hurt on any large project. I don’t agree with this at all. Here’s a counter example: Linux. I think the real problem here is that people don’t take review seriously and/or just don’t care about the project. We build these complicated, expensive systems that do all the thinking for us, not because it’s better, but because we simply don’t care. Having tests is better than letting sloppy developers run amok. In my experience though tests aren’t really needed on projects with a maintainer that actually gives a damn.
- y4mi 8y ago
  > I don’t agree with this at all. Here’s a counter example: Linux. are you sure about that? https://stackoverflow.com/questions/3177338/how-is-the-linux-kernel-tested https://stackoverflow.com/questions/3177338/how-is-the-linux...
- pathseeker 8y ago
  Bad example because a massive chunk of linux is independent systems that have no overlap at all. The netlink code has jack-shit to do with a graphics driver from intel, etc. Within subsystems regressions sneak in all of the time that don't directly cause compile-time failures. http://linux-test-project.github.io/ http://linux-test-project.github.io/

guscost 8y ago

Many if not most tests are a waste of time (especially when the same guarantees are provided by, say, a type system). That said, automated tests catch a problem that would make it to production just about weekly in my experience. If you're trying to do any kind of HA/CI/CD they're really really helpful. Beyond that, isolated "unit tests" are somewhat helpful too in my experience, but only so far as they allow you to easily run your test suite (and rarely, find bugs faster).

mh8h 8y ago

Writing tests makes me write better software. The resulting test suite is just a nice side effect.

doubleunplussed 8y ago

I'm not a professional programmer, but I think it comes down to whether you expect to ever do major refactoring. If requirements change substantially, would you write new code, or would you modify the existing code and attempt to maintain some kind of backward compatibility? If the latter, tests can be worth it. I have a library that never had any tests until I wanted to re-architect it in a fundamental way whilst porting it from Python 2 to Python 3. Since it was a wire-protocol type of thing, the Python bytes–unicode distinction made this hard. So I wrote tests for each piece of functionality as I ported that piece. I keep the tests around nowadays even though the porting is complete, but am kind of lazy writing ones for new functionality. Of course tests also seem really important when you're accepting 3rd party contributions to your codebase. I've made what I thought were totally innocuous pull requests to projects only to see in the CI tests that my code causes regressions.

jrvarela56 8y ago

Broadly speaking, there are several steps in the development process where defects are caught: 1. During development: you code something, run it, see something fail and fix it. Most people don't consider this part of QA because it's coding/debugging. 2. Code review: a teammate reads your code and catches a bug or an opportunity for a bug to arise. 3. Manual QA: a teammate tries the new feature, finds a defect and reports it. 4. Production: a user reports the error. 5. Monitoring: tools like Sentry may warn you before a user reports the issue. With your experience, you probably have learned to avoid patterns in your code that make #1 occur often. Maybe you use static typing, linters, an IDE or some other tools to help you avoid silly mistakes. If you perform code review (#2), then you help less-experienced teammates avoid mistakes even if they haven't learned to avoid error-prone patterns or don't use tooling like you do. They spend more time in #1 than you do. If you care (above average for developer standards) about your job, you probably manually test the software you make to see how your users experience your creation (#3) before checking-in code, while code-reviewing, and/or afterwards to help testers. This takes time and if you find mistakes, that means other teammates that did this before you invested time also. If your company releases progressively (using feature flags) you get info from users (#4) and fix before it's considered an issue. Also, if using monitoring tools (#5) you may avoid users ever noticing there was an issue. These all lead to rework and are risky. I don't have enough information about what process you follow, so I tried making references to generic practices. In theory, writing tests first reduces the amount of time reacting in situations 1-5. I have felt the benefits in my own team and seen them when coaching other teams. If none of what I have written applies to you and your team, please let me know. Would love to buy you all a meal to hear about what you do to avoid defects in your software. I'm not being snarky here. Have genuinely been asking myself how do these developers I read about online manage to ship bug-free software really fast without writing tests.

mattmanser 8y ago
It's probably a combination of #3 (I always do that now, use the feature I just made/improved/fixed) and that I always make sure I can reliably repro a bug before I try and fix the bug, so I always know it's fixed. Also a significant majority of my code has been statically typed as you say and I'm moving even more over to static typing now I'm using TS over JS in all new code.

zeroname 8y ago

> Am I (and colleagues) a magic snowflake, or are tests a massive waste of time, or is it somewhere in the middle? Tests are definitely a massive waste of time if you never need them. Write and forget code exists and can certainly run happily for years. On the other hand, frequently changing something in a large existing codebase, especially without static typing, is almost certainly going to end up more expensive without automated tests.

fredifrum 8y ago

You are able to go without tests because you're writing entire apps on your own. You know what each file, class, and module is supposed to do, and how to test it for regressions manually. If you're working on a large team with a large suite of applications, this isn't the case. Very often I'll need to make changes to code that I don't explicitly own, or that I've never seen before today. If this code doesn't have tests, this is a recipe for disaster. I don't really understand what this code is supposed to do. How do I know that my change didn't break any existing functionality? Is it obvious how or even possible to test the functionality in this file manually? Maybe it's a background job that processes a file off of an SFTP server, how long will it take me to set up a manual test for that? It's also about iteration time. Automated tests allow to you check the correctness of every single change nearly instantaneously. I don't want to need switch to a browser and wait for a form to submit just to check that my code still works after a minor refactor, or to ensure that my typo was fixed successfully. Tests mean that code is much more likely to be refactored often, and will lead to much cleaner and easier to maintain codebases.

candiodari 8y ago
This argument, and most of the others at this same level, have a glaring error. They only justify integration tests. They do not justify unit tests (in nearly all cases). Unit tests wouldn't catch the examples you gave: > How do I know that my change didn't break any existing functionality? You don't. And with many unit tests ... you still don't. A software engineer with a few years experience will know this. > Maybe it's a background job that processes a file off of an SFTP server, how long will it take me to set up a manual test for that? This especially, no unit test is ever going to catch. This behavior would depend on so many things: encryption, networking, scheduling, starvation, graceful error handling, retry logic ... Not something any amount of unit tests could reasonably verify. So this must be an argument against unit tests ? And yet, I don't think so. In practice, you see the opposite. These arguments are designed to force people to write unit tests, and actually used against integration tests (which are complex tests that require tweaking in intelligent ways every time you make a serious change). Of course integration tests and "instantaneous" are usually opposites. So I don't understand this argument. It only makes sense on the surface. It does not actually work on real large code projects. So I don't understand that you make this argument. And I especially do not understand that nearly every large company follows advice like this. I mean, I can come up with cynical reasons: "more pointless work with zero verifiable results, but nice "quantifiable" (if meaningless) numbers ! Of course we're in favor !". But really, I cannot come up with a coherent to explain the behavior seen in developers in a large company. Most statistics about unit tests make no sense. Coverage for instance, says nothing about how many edge cases you handle. Says nothing about many guarantees (like bounded and reasonable allocation for "weird" inputs, same with cpu usage, that it still works together with the OS and other daemons on the system it'll be installed on, ...). Since tests have no "real" purpose for software (they're code that doesn't run) number of tests is meaningless too. Number of lines of test code ... that's beyond useless. The bigger a piece of software, the more useless unit tests become and the more critical realistic integration tests become ... and the less likely you are to actually find them. And frankly, for small pieces of code, I find that specialized ways to "test" them that provide actual guarantees of correctness (like Coq) actually work. Unit tests ... never catch edge cases, not even with the best of developers. Of course, even Coq sort-of only makes sure you KNOW about every edge case and requires you to verify it's correctness before letting you pass, so bugs in your understanding of the problem will still be bugs. But at least you have a theoretical guarantee that a developer actually thought about every edge case. Also, it's pretty hard to use. The language could be clearer, but that's not really the problem. The problem is that Coq WILL show you that no matter how well you think you understand a problem, your understanding is not complete. Often in surprising, interesting ways.

mixmastamyk 8y ago

As always, "it depends." Testing is usually about larger teams, cost of failure, and sleeping better at night during refactors. If the project is small, fits in one skull, and not critical enough to be maintained/redesigned for years on end tests buy you less. Also, technology can prevent some classes of problems, such as type checkers, linters, etc. One hopes you are at least using these. #1 gives the impression the author is using a dynamic "scripting" language, which may require a higher ratio of tests per line as a project matures.

JamesBarney 8y ago

I've found the sweet spot for tests are - complex code that will change with definable inputs/outputs that won't - large projects I've found they're not nearly as helpful in - small projects - UI intensive code - code where the requirements are ill defined and change often

deleted 8y ago

[deleted]

vharuck 8y ago

>We'd have picked it up anyway when actually testing it. Why not automate the tests you often run?

dougk16 8y ago

I'm in the same boat and have thought a lot about it. The biggest correlation I see from automated testing advocates is that they are usually using dynamic languages for non-trivial core business logic. I mean that's pretty much it. If you're using a dynamic language for such things, like anything past 10k lines of code, then I agree, you pretty much need automated tests since they're partially filling in the role of a compiler and type system. It begs the question of why to use dynamic languages for tricky business logic, but that's a whole different flamewar! If the testing advocates are using something with even a basic compiler/type-system (e.g. Java), then IME it often comes down to not being fluent with the design patterns, tricks, IDE shortcuts/plugins, tools, etc. that compilers and type-systems can provide to work with you to catch bugs at build time. I mention Java because it's a language that allows you to keep writing "dynamic" code and avoid the compiler/type-system to a good degree if you want (and boy do people want to). E.g. passing top-level Object types around and casting, using HashMaps instead of a proper Class, heavy reflection use, "stringly" typing, etc. Unlike say Haskell or Rust which force correctness down your throat to a much higher degree. Obviously no matter the language choice, their comes a point where automated testing becomes cost-effective (e.g. safety-critical software). At that point however there should be formal methods and such in play also. And I'd say for the majority of us working on CRUD apps, we're not at that cost-effectiveness point.

rorykoehler 8y ago
Testing is for business logic not for seeing if the project will build.
arkh 8y ago
Currently you read a lot about unit testing. Usually used badly (testing solo methods or classes). What you want is E2E tests: test that your product conforms to its specifications. Automate the test procedure your testers have.
pathseeker 8y ago
Type testing can't catch business logic regressions.

contingencies 8y ago

Test interfaces, not implementations.

Mc_Big_G 8y ago

Tests fail all the time during development. Test fails indicate a bug. You fix the bug before you commit. The bug never makes it to prod. It's impossible that only one test ever failed during development.

rorykoehler 8y ago

God help the person who has to pick up after you, do refactoring etc etc. Not writing tests is unprofessional as it introduces risk. If code is being used by 1 person or is some non-critical internal application then fine. If it is being used by 1 million people to earn a living the people responsible should lose their jobs. I spend a significant amount of my time tidying up after people who don't write tests and leave a mess and it's a nightmare (but keeps my family fed, housed and clothed so...).

arkh 8y ago

How much maintenance have those sites seen? I usually focus on end-to-end tests: when I had a new feature or do some refactoring I don't have to try things manually. I just launch my test suite go prepare some coffee and if the suite passes I'm confident in not having broken anything. Also writing new tests for new features helps thinking about the edge-cases you'll encounter. When someone reports a bug I reproduce it. Then I write a test reproducing the problem. And at last I correct it. Now I know 6 months or one year from now I won't have a regression and see this bug coming back. People really hate when something they reported months ago is a problem again.

mattmanser 8y ago
Huge amounts, you don't have a million users for failed projects. I still maintain various code bases for multiple clients that are over 10 years old, at varying levels of TLOC (10k, 100k, million k). The truth is bugs almost never come back like that. I know because of the thousands, maybe 10s of thousand, of bugs I've fixed over the years.

HereBeBeasties 8y ago

Your code is full of bugs, you just haven't noticed yet. Whether that matters or not (to you or your end users) is your grey ground in the middle. It's possible you've only ever worked on things that are simple or obvious enough to not have the need for tests, and that code that never changes. Especially if that's UI code, where you're testing it manually as you write it, you may be able to live with this on small code bases. If you have never worked in an environment where the benefit of at least some level of automated testing is not blindingly obvious, my advice to you is to stop doing the same thing over and over and find a new job.

mattmanser 8y ago
Projects with unit tests are also riddled with bugs. I have absolutely no respect for your attitude or your comment. Sanctimonious zealotry like yours should have no place in our industry. It's disgusting that you think you can tell someone they're shit or trivialize their work because you disagree with their workflow.
- HereBeBeasties 8y ago
  I hardly think that encouraging someone to seek out alternative positions from which to view the world, and a suggestion that there is a grey area for all this constitutes zealotry. Quite the opposite, in fact. And in my experience projects with unit tests are far _less_ riddled with bugs, which is the whole point, no? And I don't know understand why on earth you think I'm telling anyone they are "shit". I'd advise someone to broaden their horizons if they said they thought functional programming was worthless, or that assembly language was always pointlessly low level, or any one of a number of possible assertions that wouldn't sit well with me. That is far from "you are shit" - it's much more assuming they are not shit, but haven't had the right context yet.

suraj 8y ago

When I was a rookie programmer 12 years ago, I was assigned a task to optimise a routine. It took current vehicle speed as input, performed integration and calculated total distance traveled. I made a mistake that was let through code review and was caught by an automated test. My calculations were off by 1% which accumulated to a larger amount in 10 minutes. After that incident we never had another bug for 5 years in that module. Does this make the tests invalid? Our does this improve confidence in our code?

azdacha 8y ago

There is something in the middle actually. Just write test for the hard tasks or the algos that you implemented your self. (Some stuff that I don't undertand end up much easier to write once I've created some tests to help me iterate faster) Testing basic class re-use (I'm thinking about serializers, or ViewSets like in django) doesn't make much sense to me, and in fact was 100% automatised. Those "simple" test end-up being useful when you start hitting your app with the hammer to change a lot of stuff in it. I havn't had any use for these otherwhise.