5 ms·

Unit tests don't help you determine whether a given behavior is incidental or necessary. For a lot of code, including the statistics code in the article, correc

Unit tests don't help you determine whether a given behavior is incidental or necessary. For a lot of code, including the statistics code in the article, correctness is a dynamic property: dependent on its context, environment, clients, etc. So working with legacy code is as much project management and communication as it is engineering best practices. Consider Chrome's recent change to mark sites as insecure if they have a password field without TLS. Will my grandma's bank page be reported as insecure? There's no way to know a priori, or through isolated unit tests. The Chrome team has to go out into the Internet jungle, reaching out to the site owners and pushing until the major sites are ready, and any holdouts can be considered acceptable casualties. With that understanding, whether code is "legacy" isn't about age or budget or level of support; it's about how much influence you have over your clients. Some code is never used and so never becomes legacy, no matter how old it is. But other code is born legacy: it gets clients immediately and pays the price for being popular.

dagw 10y ago
Another problem with fixing something like that statistics code is that it makes it to impossible compare to new results to old results. I have a piece of old code like that myself kicking around that tends to overestimate a certain value. I also have some new code that uses a better model and gives a more accurate value. The problem is I cannot really roll out the new code since it will make all the new things look worse than all the old things, even if they're actually the same or perhaps even better. So I use the old code, because even if it's wrong, at least it's wrong in that same way as all the other results and people can make relevant comparisons.
josteink 10y ago
> Unit tests don't help you determine whether a given behavior is incidental or necessary. That's his point. If you have a test-case for the behaviour, you know that this is how it should actually behave. Anything not covered in tests however, that will naturally be covered in uncertainty for any weirdness, quirks or odd behaviour. Should it be there or not? Hard to tell when it's not formalized in a unit-test.
- flukus 10y ago
  Assuming they were testing behavior and not shooting for some coverage metric. I come across plenty of bugs with tests against them. I have a rather strict rule personally, every test should start with a line that says what it's testing.

dagw 10y ago

Another problem with fixing something like that statistics code is that it makes it to impossible compare to new results to old results. I have a piece of old code like that myself kicking around that tends to overestimate a certain value. I also have some new code that uses a better model and gives a more accurate value. The problem is I cannot really roll out the new code since it will make all the new things look worse than all the old things, even if they're actually the same or perhaps even better. So I use the old code, because even if it's wrong, at least it's wrong in that same way as all the other results and people can make relevant comparisons.

josteink 10y ago

> Unit tests don't help you determine whether a given behavior is incidental or necessary. That's his point. If you have a test-case for the behaviour, you know that this is how it should actually behave. Anything not covered in tests however, that will naturally be covered in uncertainty for any weirdness, quirks or odd behaviour. Should it be there or not? Hard to tell when it's not formalized in a unit-test.

flukus 10y ago
Assuming they were testing behavior and not shooting for some coverage metric. I come across plenty of bugs with tests against them. I have a rather strict rule personally, every test should start with a line that says what it's testing.