13 ms·

How deep are your unit tests?

62 points by thallavajhula 14y ago

MojoJolo 14y ago
I agree with the first answer. I'm not a fan of unit testing. I'm just doing it because it's a requirement (and we still have no QA). Why I'm not a fan? Because I'm doing my own tests. And probably it has a bias all over it. I know when it will have a successful run and I expect where it will fail. This is not the scenario that I wanted. (I know this is not a good testing scenario) In my opinion, a programmer must write a good code. And leave the testing to the QA.
- selectnull 14y ago
  In my opinion, a programmer must write a good code. I always write good code, but somehow that code turns to shit after six months, and I promise no one touched it. So, I guess I don't need the tests when I'm writing the code but I need them badly when I maintain it.
  - Vivtek 14y ago
    Oh dear Lord, how true this is.
  - Ygg2 14y ago
    That's quite opposite for me, I write code with lots of tiny errors (most of them 0,1 based array sizes or not understanding how API works), so I test as much as possible. But tests true value is regression testing. I can refactor something, and test knowing that if I messed something up, test will flare up. I still try to avoid errors, but it's pretty much nigh impossible.
- AndrewDucker 14y ago
  And when someone else is maintaining your code, and changes something deep within in it that has an unexpected side effect, how will you tell whether it's broken something or not?
  - rjd 14y ago
    That is the only time I write unit tests. I never write them first run of software, but when I start changing the product down the track maintenance/upgrades/bugs/what ever I'll add tests to make sure things don't explode, and hopefully I'm not introducing problems. That said often things are rushed and they get ignored, even when I'm low on confidence about said introductions of errors :P
- philbo 14y ago
  I suspect there are some subtleties to that first answer which you've overlooked. It is written by Kent Beck. He pretty much invented the concept of test-driven development. He's probably the most well-known advocate of developers writing their own tests there is. What his answer demonstrates (to me) is that the conclusions he and the other XP guys arrived at - to write code in tiny chunks, each one preceded by a tiny failing test - is based entirely in pragmatism. He does it because it works better than anything else he's come across, including leaving the tests up to a dedicated QA team.
  - dalke 14y ago
    "... because it works better..." Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html http://third-bit.com/blog/archives/4529.html: "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile." There is very little research which shows that TDD is better than other approaches. Perhaps a bit, for some cases, but not enough to be certain about its general applicability. (Or has this changed in the last three years since I researched this topic?) I hate writing code in "tiny chunks, each one preceded by a tiny failing test." I prefer to use a small set of burn tests until I have the main structure in place. Only then do I do the tiny tests - through the high-level API where possible (so I can refactor later) - and I use coverage analysis to make sure I'm not missing any obvious tests. Coverage analysis is not part of TDD, which is a shame. (Technically that's because TDD is a design methodology, while coverage analysis is part of testing. While TDD generates tests, these are not a complete set of tests. For example, security analysis and scalability are outside the scope of TDD.)
    - stonemetal 14y ago
      Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html http://third-bit.com/blog/archives/4529.html: "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile." Oh good, evidence against is just as preachy and unscientific as the evidence for. http://evidencebasedse.com/?q=taxonomy/term/2 http://evidencebasedse.com/?q=taxonomy/term/2 Research suggests that unit testing has a very strong correlation with correct software. Research suggests TDD has no correlation with correct software, except for the fact that TDDers tend to write 2x more tests than non TDDers.
- deleted 14y ago
  [deleted]
- tikhonj 14y ago
  This is one of the reasons I really like property-based testing à la QuickCheck[1]. The core idea is simple: you come up with an invariant for your code and the testing framework checks this invariant with randomly generated inputs. So when you're writing a test, you just have to come up with interesting invariants; you do not have to guess which inputs are edge cases. [1]: http://www.haskell.org/haskellwiki/Introduction_to_QuickCheck http://www.haskell.org/haskellwiki/Introduction_to_QuickChec... This may sound complicated and seem like overkill, but I've actually found it easier to use this style of tests for much of my code. Having randomly generated inputs can help find edge cases I did not even consider when writing the code in the first place--this addresses the bias issue you are worried about. I also find that this style of test results in concise, easy to read tests. The usual "hello world" example uses the reverse function for lists. We want to ensure that for all lists, reversing it twice does nothing. The test would look like this: prop_reverseTwice ls = reverse (reverse ls) == ls It's very easy to tell what invariant you're testing for! (The prop_* name is a convention for this sort of test.) While real invariants are often more complex, I find they are still usually easy to read right from the code for the test. Anyhow: you should really try QuickCheck or something like it for your language of choice.
  - danieldk 14y ago
    I agree that QuickCheck is really nice. It helped me spot some errors in e.g. tree diff'ing code that I wrote. QuickCheck especially works well for fundamental data structures and algorithms, where invariants are clear and simple. However, in other cases, the usage is less obvious. For instance, suppose that you are writing, say a URL parser. What do you want QuickCheck to generate and check? If you generate too broadly, the invariant check becomes a parser itself, if you generate too specifically you might introduce bias again.
    - tikhonj 14y ago
      That's a good point. I remember having exactly that problem when writing tests for my toy language's parser. Generating random valid programs was actually tricky for several different reasons. In fact, I ultimately gave up on QuickCheck completely for that project, but mostly out of laziness. Amusingly, I also wrote some tree diff code in Haskell recently. However, I did it rather poorly so I'm going to have to rewrite it soon (probably some time during winter break). I definitely agree that algorithms like this are particularly well-suited for QuickCheck.
- Firehed 14y ago
  > In my opinion, a programmer must write a good code. And leave the testing to the QA. Aside from the "not my problem" vibe that gives off, I strongly disagree. QA is helpful for covering weird user interactions and writing tests against bugs discovered in production, but you as the programmer should know what your code is supposed to do while you're writing it. Best write tests then rather than six months later when you've forgotten what it was for.
deleted 14y ago
[deleted]
mewmoo 14y ago
A lot of the comments say tests are needed for edge cases and discovered bugs. How does this compare with TDD? Has anyone tried both? How did it work out for you? I implemented TDD for a project and I thought it was overkill and took a considerable amount of my time.
- fruiapps 14y ago
  do it for more projects, don't you think the first time you did anything it took helluva time??
clarle 14y ago
A good strategy I've found was suggested by Reid Burke in response to this answer: http://reidburke.com/2012/09/27/write-code-that-works/ http://reidburke.com/2012/09/27/write-code-that-works/ Your code isn't uniform, and so you shouldn't be expecting 100% code coverage on everything either. Instead, assign to particular sections of your code levels of stability, and for the code that has the highest stability/least amount of expected future change, prioritize writing the most unit tests for those.
raverbashing 14y ago
Several successful projects exist without TDD (or even unit tests for that matter) The whole of GNU+Linux for example (or at least most of it) On the other hand I've seen several projects claim "very good TDD coverage" and then crash and burned when put into production (usually posted to HN) Real world, real usage testing is essential. TDD is good for keeping you on track and avoid regressions It's also good for complicated (small) pieces of software that do something complex and is prone to having its behaviour adjusted with time (think: reports, data consolidation or analysis, calculations, etc)
- fruiapps 14y ago
  if rms read it, he would suggest editing GNU+Linux to GNU/Linux.
  - raverbashing 14y ago
    Yes, I wrote it like this to be clear that I mean the Linux kernel and the GNU userspace
  - jiggy2011 14y ago
    I don't think RMS reads HN.
Negitivefrags 14y ago
The thing I really dislike are unit testing styles that use mock objects a lot. I see some people say that you should test each object in isolation, mocking out any dependencies. That seems wrong headed to me. I prefer my unit tests to test everything all the way down. The only thing I would mock is the file system, which is useful for testing file loading code.
- dale-cooper 14y ago
  I think that's more towards some kind of integration testing. You're supposed to test one unit independently, and then write integration tests that actually tests that the whole system works. Not saying that's the best way but i think that's how people think.
- azylman 14y ago
  I prefer my unit tests to test everything all the way down. They already have that, that's called integration testing. Unit testing is trying to test as small of a unit of code as possible - this helps you identify exactly where the error is. If all of your tests are integration tests and something fails, you have no idea which part of your stack the problem is in. Obviously different projects require different kinds and amounts of test cases, but I prefer a mixture of both unit tests and integration tests (unit tests for small, complex blocks of logic, integration tests for most of the other stuff).
  - Silhouette 14y ago
    If all of your tests are integration tests and something fails, you have no idea which part of your stack the problem is in. Not necessarily, because if you have a lot of integration tests then probably multiple ones will fail given a bug in a certain module, and the pattern of the failures might be all you need to locate the source of the problem. One of the most successful projects I ever worked on, in robustness/quality terms, didn't really have any unit tests at all, but it had a comprehensive suite of end-to-end test cases that could be run automatically. Many of those didn't (and couldn't) have an absolute true/false outcome, either, but looking at the results they generated and applying various heuristics developed from experience, they were remarkably useful for similar reasons to unit tests.
huhtenberg 14y ago
I find that simply assert'ing all non-trivial assumptions and invariants in the code is the form of unit testing that works the best in heck of a lot of cases. If there's need for explicit tests, just use the code in the right way and see it exit cleanly. Then use the code in the wrong way and see it assert (first replacing the assert() with a throw/longjmp to catch failures without aborting). Asserts also double as a concise context documentation. When an assert is triggered, it's typically easy to see what its condition means and what has gone wrong. So it's a win-win all around :)
- FooBarWidget 14y ago
  I've found asserts to be useful in addition to unit testing. The advantage of automated tests is that you don't have to run tests manually. Asserts merely tell you that something is wrong, they don't run test cases for you.
- meaty 14y ago
  I do this. In fact, for a lot of what I do, assertions are good enough to catch any issues early. That and integration tests are probably more valuable than religious unit tests. The advantage with assertions as well, is if you leave them on in production, when an issue hits, you know about what it is straight away and can fix. Many of our system errors are fixed and deployed before the user has finished raising the issue.
davidw 14y ago
I try and cover the easy 80% where it's helpful, and then specific things in the difficult 20% on an as-needed basis, or when bugs are discovered. It's an asymptotic curve: you want to spend the effort for the low-hanging fruit, but not get caught up chasing after that time-eating final bit of coverage.
jiggy2011 14y ago
> "If I don't typically make a kind of mistake (like setting the wrong variables in a constructor), I don't test for it." Hmm, that sounds like the old "You don't need to test , just be an awesome programmer duh!" argument. The kind of mistakes you never anticipate making are probably the sort of stuff you should be testing for. The amount of times I've had something fail on an edge case that turned out to be a typo in a var name on some edge case somewhere..
- jiggy2011 14y ago
  don't know why this has so many downvotes.
  - pekk 14y ago
    Your tests routinely fail because of typos?
    - jiggy2011 14y ago
      I've seen production code fail because of typos after months in deployment (because that's how long it took that else { branch to be hit in practice). The code in question was only subject to very high level smoke tests that looked like they were basically cherry picked best case scenarios intended to pass. A unit test would have picked this up before it even got to production.
    - TeMPOraL 14y ago
      In dynamic languages? Sure they may. The PHP script you thought works crashes, the Lisp routine out of the sudden enters the debugger, things like that.
      - pekk 14y ago
        The question isn't what "may" happen, but what routinely does happen for you. If typos are even a significant problem in your code then the solution is to type more slowly, have your IDE flag unknown symbols, use autocomplete, etc. As a solution to this problem, strong typing is both disproportionate and inadequate.
fruiapps 14y ago
Testing is something I do a lot, and like talking about a lot as well. There are no hard and fast answers to testing, like other stuff, there are lots of opinions(they are like assholes, everyone has them, and the others' stinks) about testing. So here goes mine(not necessarily correct, but worth putting them here). I am putting them as one liners, let me know your thoughts. 1. What do you mean by unit in unit tests? An implementation of very basic feature, probably not more than 100 lines of code. 2. What do you test when you write a unit test? We test the basic correctness, eg. given a url, if it exists assert 200, if not assert 404. 3. What should not be tested? The inbuilt libraries should not be tested. Eg. If you are using a library say django, you should assume it comes pre tested and should not spend time writing tests for them. 4. What should be the depth of the tests? There are no rules, but you should stop when you feel like you are humming inception theme. If you want to read in slight more detail its http://www.blog.fruiapps.com/2012/09/An-intro-tutorial-to-test-driven-development-in-python-django http://www.blog.fruiapps.com/2012/09/An-intro-tutorial-to-te...
chris_wot 14y ago
Kent Beck's response is interesting. How does one know that you write correct code until you realise there is a bug in it? I thought unit tests were developed for when you refactor code - at this point you run the unit tests and if any fail, you have refactored in a bug...
kurjam 14y ago
Well. I use a lot of unit testing because the idea of testing small pieces of code works for the way I develop software at work - modified pair programming. Instead of reviewing Holmes's code Watson writes tests (mostly unit ones). At spare time, tho, I tend to be too lazy to write a lot of tests(most of my "projects" are too small and never get released to the public anyway)
TeMPOraL 14y ago
It's partially an answer to jiggy2011's question from http://news.ycombinator.com/item?id=4898861 http://news.ycombinator.com/item?id=4898861, but I think it's worth mentioning separately. There's a very good sentence in SO thread - j_random_hacker's: "Every programmer has a probability distribution of bug locations; the smart approach is to focus your energy on testing regions where you estimate the bug probability to be high." So Kent Beck is not skipping some tests because he thinks he's awesome (as jiggy2011 said it sounds like); he does that because he can anticipate, from experience, that a particular type of bug is of very low probability for him, because he knows he's unlikely to make it. Like, for example, I tend not to put + instead of ++ in C++, so I can be pretty confident my code is free of these types of errors. Everyone needs to estimate his probability distribution for oneself (factoring for unknown unknowns) and test accordingly. The more experience one has, the better the estimate.
ruggeri 14y ago
Is it me, or does Bill the Lizard close every SO thread linked from HN?

62 points by thallavajhula 14y ago

I agree with the first answer. I'm not a fan of unit testing. I'm just doing it because it's a requirement (and we still have no QA). Why I'm not a fan? Because I'm doing my own tests. And probably it has a bias all over it. I know when it will have a successful run and I expect where it will fail. This is not the scenario that I wanted. (I know this is not a good testing scenario) In my opinion, a programmer must write a good code. And leave the testing to the QA.

selectnull 14y ago
In my opinion, a programmer must write a good code. I always write good code, but somehow that code turns to shit after six months, and I promise no one touched it. So, I guess I don't need the tests when I'm writing the code but I need them badly when I maintain it.
- Vivtek 14y ago
  Oh dear Lord, how true this is.
- Ygg2 14y ago
  That's quite opposite for me, I write code with lots of tiny errors (most of them 0,1 based array sizes or not understanding how API works), so I test as much as possible. But tests true value is regression testing. I can refactor something, and test knowing that if I messed something up, test will flare up. I still try to avoid errors, but it's pretty much nigh impossible.
AndrewDucker 14y ago
And when someone else is maintaining your code, and changes something deep within in it that has an unexpected side effect, how will you tell whether it's broken something or not?
- rjd 14y ago
  That is the only time I write unit tests. I never write them first run of software, but when I start changing the product down the track maintenance/upgrades/bugs/what ever I'll add tests to make sure things don't explode, and hopefully I'm not introducing problems. That said often things are rushed and they get ignored, even when I'm low on confidence about said introductions of errors :P
philbo 14y ago
I suspect there are some subtleties to that first answer which you've overlooked. It is written by Kent Beck. He pretty much invented the concept of test-driven development. He's probably the most well-known advocate of developers writing their own tests there is. What his answer demonstrates (to me) is that the conclusions he and the other XP guys arrived at - to write code in tiny chunks, each one preceded by a tiny failing test - is based entirely in pragmatism. He does it because it works better than anything else he's come across, including leaving the tests up to a dedicated QA team.
- dalke 14y ago
  "... because it works better..." Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html http://third-bit.com/blog/archives/4529.html: "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile." There is very little research which shows that TDD is better than other approaches. Perhaps a bit, for some cases, but not enough to be certain about its general applicability. (Or has this changed in the last three years since I researched this topic?) I hate writing code in "tiny chunks, each one preceded by a tiny failing test." I prefer to use a small set of burn tests until I have the main structure in place. Only then do I do the tiny tests - through the high-level API where possible (so I can refactor later) - and I use coverage analysis to make sure I'm not missing any obvious tests. Coverage analysis is not part of TDD, which is a shame. (Technically that's because TDD is a design methodology, while coverage analysis is part of testing. While TDD generates tests, these are not a complete set of tests. For example, security analysis and scalability are outside the scope of TDD.)
  - stonemetal 14y ago
    Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html http://third-bit.com/blog/archives/4529.html: "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile." Oh good, evidence against is just as preachy and unscientific as the evidence for. http://evidencebasedse.com/?q=taxonomy/term/2 http://evidencebasedse.com/?q=taxonomy/term/2 Research suggests that unit testing has a very strong correlation with correct software. Research suggests TDD has no correlation with correct software, except for the fact that TDDers tend to write 2x more tests than non TDDers.
deleted 14y ago
[deleted]
tikhonj 14y ago
This is one of the reasons I really like property-based testing à la QuickCheck[1]. The core idea is simple: you come up with an invariant for your code and the testing framework checks this invariant with randomly generated inputs. So when you're writing a test, you just have to come up with interesting invariants; you do not have to guess which inputs are edge cases. [1]: http://www.haskell.org/haskellwiki/Introduction_to_QuickCheck http://www.haskell.org/haskellwiki/Introduction_to_QuickChec... This may sound complicated and seem like overkill, but I've actually found it easier to use this style of tests for much of my code. Having randomly generated inputs can help find edge cases I did not even consider when writing the code in the first place--this addresses the bias issue you are worried about. I also find that this style of test results in concise, easy to read tests. The usual "hello world" example uses the reverse function for lists. We want to ensure that for all lists, reversing it twice does nothing. The test would look like this: prop_reverseTwice ls = reverse (reverse ls) == ls It's very easy to tell what invariant you're testing for! (The prop_* name is a convention for this sort of test.) While real invariants are often more complex, I find they are still usually easy to read right from the code for the test. Anyhow: you should really try QuickCheck or something like it for your language of choice.
- danieldk 14y ago
  I agree that QuickCheck is really nice. It helped me spot some errors in e.g. tree diff'ing code that I wrote. QuickCheck especially works well for fundamental data structures and algorithms, where invariants are clear and simple. However, in other cases, the usage is less obvious. For instance, suppose that you are writing, say a URL parser. What do you want QuickCheck to generate and check? If you generate too broadly, the invariant check becomes a parser itself, if you generate too specifically you might introduce bias again.
  - tikhonj 14y ago
    That's a good point. I remember having exactly that problem when writing tests for my toy language's parser. Generating random valid programs was actually tricky for several different reasons. In fact, I ultimately gave up on QuickCheck completely for that project, but mostly out of laziness. Amusingly, I also wrote some tree diff code in Haskell recently. However, I did it rather poorly so I'm going to have to rewrite it soon (probably some time during winter break). I definitely agree that algorithms like this are particularly well-suited for QuickCheck.
Firehed 14y ago
> In my opinion, a programmer must write a good code. And leave the testing to the QA. Aside from the "not my problem" vibe that gives off, I strongly disagree. QA is helpful for covering weird user interactions and writing tests against bugs discovered in production, but you as the programmer should know what your code is supposed to do while you're writing it. Best write tests then rather than six months later when you've forgotten what it was for.

deleted 14y ago

[deleted]

mewmoo 14y ago

A lot of the comments say tests are needed for edge cases and discovered bugs. How does this compare with TDD? Has anyone tried both? How did it work out for you? I implemented TDD for a project and I thought it was overkill and took a considerable amount of my time.

fruiapps 14y ago
do it for more projects, don't you think the first time you did anything it took helluva time??

clarle 14y ago

A good strategy I've found was suggested by Reid Burke in response to this answer: http://reidburke.com/2012/09/27/write-code-that-works/ http://reidburke.com/2012/09/27/write-code-that-works/ Your code isn't uniform, and so you shouldn't be expecting 100% code coverage on everything either. Instead, assign to particular sections of your code levels of stability, and for the code that has the highest stability/least amount of expected future change, prioritize writing the most unit tests for those.

raverbashing 14y ago

Several successful projects exist without TDD (or even unit tests for that matter) The whole of GNU+Linux for example (or at least most of it) On the other hand I've seen several projects claim "very good TDD coverage" and then crash and burned when put into production (usually posted to HN) Real world, real usage testing is essential. TDD is good for keeping you on track and avoid regressions It's also good for complicated (small) pieces of software that do something complex and is prone to having its behaviour adjusted with time (think: reports, data consolidation or analysis, calculations, etc)

fruiapps 14y ago
if rms read it, he would suggest editing GNU+Linux to GNU/Linux.
- raverbashing 14y ago
  Yes, I wrote it like this to be clear that I mean the Linux kernel and the GNU userspace
- jiggy2011 14y ago
  I don't think RMS reads HN.

Negitivefrags 14y ago

The thing I really dislike are unit testing styles that use mock objects a lot. I see some people say that you should test each object in isolation, mocking out any dependencies. That seems wrong headed to me. I prefer my unit tests to test everything all the way down. The only thing I would mock is the file system, which is useful for testing file loading code.

dale-cooper 14y ago
I think that's more towards some kind of integration testing. You're supposed to test one unit independently, and then write integration tests that actually tests that the whole system works. Not saying that's the best way but i think that's how people think.
azylman 14y ago
I prefer my unit tests to test everything all the way down. They already have that, that's called integration testing. Unit testing is trying to test as small of a unit of code as possible - this helps you identify exactly where the error is. If all of your tests are integration tests and something fails, you have no idea which part of your stack the problem is in. Obviously different projects require different kinds and amounts of test cases, but I prefer a mixture of both unit tests and integration tests (unit tests for small, complex blocks of logic, integration tests for most of the other stuff).
- Silhouette 14y ago
  If all of your tests are integration tests and something fails, you have no idea which part of your stack the problem is in. Not necessarily, because if you have a lot of integration tests then probably multiple ones will fail given a bug in a certain module, and the pattern of the failures might be all you need to locate the source of the problem. One of the most successful projects I ever worked on, in robustness/quality terms, didn't really have any unit tests at all, but it had a comprehensive suite of end-to-end test cases that could be run automatically. Many of those didn't (and couldn't) have an absolute true/false outcome, either, but looking at the results they generated and applying various heuristics developed from experience, they were remarkably useful for similar reasons to unit tests.

huhtenberg 14y ago

I find that simply assert'ing all non-trivial assumptions and invariants in the code is the form of unit testing that works the best in heck of a lot of cases. If there's need for explicit tests, just use the code in the right way and see it exit cleanly. Then use the code in the wrong way and see it assert (first replacing the assert() with a throw/longjmp to catch failures without aborting). Asserts also double as a concise context documentation. When an assert is triggered, it's typically easy to see what its condition means and what has gone wrong. So it's a win-win all around :)

FooBarWidget 14y ago
I've found asserts to be useful in addition to unit testing. The advantage of automated tests is that you don't have to run tests manually. Asserts merely tell you that something is wrong, they don't run test cases for you.
meaty 14y ago
I do this. In fact, for a lot of what I do, assertions are good enough to catch any issues early. That and integration tests are probably more valuable than religious unit tests. The advantage with assertions as well, is if you leave them on in production, when an issue hits, you know about what it is straight away and can fix. Many of our system errors are fixed and deployed before the user has finished raising the issue.

davidw 14y ago

I try and cover the easy 80% where it's helpful, and then specific things in the difficult 20% on an as-needed basis, or when bugs are discovered. It's an asymptotic curve: you want to spend the effort for the low-hanging fruit, but not get caught up chasing after that time-eating final bit of coverage.

jiggy2011 14y ago

> "If I don't typically make a kind of mistake (like setting the wrong variables in a constructor), I don't test for it." Hmm, that sounds like the old "You don't need to test , just be an awesome programmer duh!" argument. The kind of mistakes you never anticipate making are probably the sort of stuff you should be testing for. The amount of times I've had something fail on an edge case that turned out to be a typo in a var name on some edge case somewhere..

jiggy2011 14y ago
don't know why this has so many downvotes.
- pekk 14y ago
  Your tests routinely fail because of typos?
  - jiggy2011 14y ago
    I've seen production code fail because of typos after months in deployment (because that's how long it took that else { branch to be hit in practice). The code in question was only subject to very high level smoke tests that looked like they were basically cherry picked best case scenarios intended to pass. A unit test would have picked this up before it even got to production.
  - TeMPOraL 14y ago
    In dynamic languages? Sure they may. The PHP script you thought works crashes, the Lisp routine out of the sudden enters the debugger, things like that.
    - pekk 14y ago
      The question isn't what "may" happen, but what routinely does happen for you. If typos are even a significant problem in your code then the solution is to type more slowly, have your IDE flag unknown symbols, use autocomplete, etc. As a solution to this problem, strong typing is both disproportionate and inadequate.

fruiapps 14y ago

Testing is something I do a lot, and like talking about a lot as well. There are no hard and fast answers to testing, like other stuff, there are lots of opinions(they are like assholes, everyone has them, and the others' stinks) about testing. So here goes mine(not necessarily correct, but worth putting them here). I am putting them as one liners, let me know your thoughts. 1. What do you mean by unit in unit tests? An implementation of very basic feature, probably not more than 100 lines of code. 2. What do you test when you write a unit test? We test the basic correctness, eg. given a url, if it exists assert 200, if not assert 404. 3. What should not be tested? The inbuilt libraries should not be tested. Eg. If you are using a library say django, you should assume it comes pre tested and should not spend time writing tests for them. 4. What should be the depth of the tests? There are no rules, but you should stop when you feel like you are humming inception theme. If you want to read in slight more detail its http://www.blog.fruiapps.com/2012/09/An-intro-tutorial-to-test-driven-development-in-python-django http://www.blog.fruiapps.com/2012/09/An-intro-tutorial-to-te...

chris_wot 14y ago

Kent Beck's response is interesting. How does one know that you write correct code until you realise there is a bug in it? I thought unit tests were developed for when you refactor code - at this point you run the unit tests and if any fail, you have refactored in a bug...

kurjam 14y ago

Well. I use a lot of unit testing because the idea of testing small pieces of code works for the way I develop software at work - modified pair programming. Instead of reviewing Holmes's code Watson writes tests (mostly unit ones). At spare time, tho, I tend to be too lazy to write a lot of tests(most of my "projects" are too small and never get released to the public anyway)

TeMPOraL 14y ago

It's partially an answer to jiggy2011's question from http://news.ycombinator.com/item?id=4898861 http://news.ycombinator.com/item?id=4898861, but I think it's worth mentioning separately. There's a very good sentence in SO thread - j_random_hacker's: "Every programmer has a probability distribution of bug locations; the smart approach is to focus your energy on testing regions where you estimate the bug probability to be high." So Kent Beck is not skipping some tests because he thinks he's awesome (as jiggy2011 said it sounds like); he does that because he can anticipate, from experience, that a particular type of bug is of very low probability for him, because he knows he's unlikely to make it. Like, for example, I tend not to put + instead of ++ in C++, so I can be pretty confident my code is free of these types of errors. Everyone needs to estimate his probability distribution for oneself (factoring for unknown unknowns) and test accordingly. The more experience one has, the better the estimate.

ruggeri 14y ago

Is it me, or does Bill the Lizard close every SO thread linked from HN?