3 ms·

Having tests can help you ensure that, when you do change one part of your application, unrelated parts don't start breaking. And I don't really buy the argumen

by st3v3r 11y ago

Having tests can help you ensure that, when you do change one part of your application, unrelated parts don't start breaking. And I don't really buy the argument that, because something might be invalidated later, we shouldn't put in the time now to make sure we get it right based on the info we do have.

nostrademons 11y ago
You should know which part of the software lifecycle you're in and adjust your development practice accordingly. When I first started my current startup idea, the product design was changing literally multiple times a day. I didn't even bother writing any code, it was all in pencil & paper notes & diagrams. Things have slowed down to about a requirements change every 2-3 days now, and I write code but no tests. If you spend 20% of your time fixing bugs and 80% of your time writing or rewriting features, tests are not the bottleneck; it's not an improvement to spend 50% of your time writing & rewriting tests so you can avoid the 20% spent tracking down bugs. When I was at Google Search, working on a mature product with billions of users, every change got 100% test coverage, and you were supposed to break a test with every change because that's how you know your test suite is comprehensive. And then it went out through a QA department and full release process. But adding a link to the page then took 2 weeks and 600 lines of code, while adding one to my startup takes 5 minutes and about 3 lines of code.
- karmelapple 11y ago
  I'd challenge you to write some tests, even high level acceptance tests. There must be a non-zero number of requirements that are not changing every 2-3 days. Every time you don't have a test to capture a desired behavior that will probably stick around for awhile, that's a risk. Granted, everything is risky in an early stage startup, but I've found all my coding is so much better when I have tests that define the requirements - even vague ones - on which to hang my code. Otherwise, why am I writing that function I just made? Is it really the most minimum thing required to meet the requirement change that just cropped up? If I shouldn't be writing the most minimum change possible, what other requirement am I silently signing up for?
- dennisgorelik 11y ago
  > If you spend 20% of your time fixing bugs and 80% of your time writing or rewriting features, tests are not the bottleneck; it's not an improvement to spend 50% of your time writing & rewriting tests so you can avoid the 20% spent tracking down bugs. That is a good line of reasoning about when auto-tests are needed. However you did not take into account time that is required to discover that something is broken. Without tests you do not know if your code change breaks important features. So you should also consider how expensive it is to test your product after every code change. You do not have to test yourself and delegate testing to end users. But turning users into testers could be pretty expensive too (loosing potential customers). So auto-tests should be introduced much earlier than when you hit "50% maintenance" threshold.
  - nostrademons 11y ago
    You don't need to worry about losing customers if you have none. At the stage I'm talking about, you'll know something is broken when you go to demo it and it doesn't work, and presumably you'll be walking through your demo manually before putting it in front of customers.
- rileymat2 11y ago
  > But adding a link to the page then took 2 weeks and 600 lines of code That is interesting, without giving away in proprietary info, what sort of changes did you have to make to add a link?
  - nostrademons 11y ago
    It's hard to really describe without giving away proprietary info, but some things that are public-but-not-commonly-thought-of include: how do you propagate "sticky" settings like SafeSearch, a user-defined current location, or multi-login? Is it an ad-click, in which case it needs to be tracked rigorously because real money is changing hands for it? Or is it a result click, where we want to do anything possible to avoid introducing latency in getting the user to where they need to be? Or a feature click, where the destination is our own servers? Should the browser do a full page refresh, or will it do an AJAX fetch for only part of the page? If it's an AJAX fetch, what needs to be refreshed? Does a history entry need to be pushed? Are we going to display something special if the user hits "back", like Facebook does for "More stories like this" (the feature in question was actually one of the first sites that did this, before Facebook)? What happens if we've changed the code on the server while the user has an outdated page open in their browser? What are all the browser differences in handling these?

by st3v3r 11y ago

nostrademons 11y ago

You should know which part of the software lifecycle you're in and adjust your development practice accordingly. When I first started my current startup idea, the product design was changing literally multiple times a day. I didn't even bother writing any code, it was all in pencil & paper notes & diagrams. Things have slowed down to about a requirements change every 2-3 days now, and I write code but no tests. If you spend 20% of your time fixing bugs and 80% of your time writing or rewriting features, tests are not the bottleneck; it's not an improvement to spend 50% of your time writing & rewriting tests so you can avoid the 20% spent tracking down bugs. When I was at Google Search, working on a mature product with billions of users, every change got 100% test coverage, and you were supposed to break a test with every change because that's how you know your test suite is comprehensive. And then it went out through a QA department and full release process. But adding a link to the page then took 2 weeks and 600 lines of code, while adding one to my startup takes 5 minutes and about 3 lines of code.

karmelapple 11y ago
I'd challenge you to write some tests, even high level acceptance tests. There must be a non-zero number of requirements that are not changing every 2-3 days. Every time you don't have a test to capture a desired behavior that will probably stick around for awhile, that's a risk. Granted, everything is risky in an early stage startup, but I've found all my coding is so much better when I have tests that define the requirements - even vague ones - on which to hang my code. Otherwise, why am I writing that function I just made? Is it really the most minimum thing required to meet the requirement change that just cropped up? If I shouldn't be writing the most minimum change possible, what other requirement am I silently signing up for?
dennisgorelik 11y ago
> If you spend 20% of your time fixing bugs and 80% of your time writing or rewriting features, tests are not the bottleneck; it's not an improvement to spend 50% of your time writing & rewriting tests so you can avoid the 20% spent tracking down bugs. That is a good line of reasoning about when auto-tests are needed. However you did not take into account time that is required to discover that something is broken. Without tests you do not know if your code change breaks important features. So you should also consider how expensive it is to test your product after every code change. You do not have to test yourself and delegate testing to end users. But turning users into testers could be pretty expensive too (loosing potential customers). So auto-tests should be introduced much earlier than when you hit "50% maintenance" threshold.
- nostrademons 11y ago
  You don't need to worry about losing customers if you have none. At the stage I'm talking about, you'll know something is broken when you go to demo it and it doesn't work, and presumably you'll be walking through your demo manually before putting it in front of customers.
rileymat2 11y ago
> But adding a link to the page then took 2 weeks and 600 lines of code That is interesting, without giving away in proprietary info, what sort of changes did you have to make to add a link?
- nostrademons 11y ago
  It's hard to really describe without giving away proprietary info, but some things that are public-but-not-commonly-thought-of include: how do you propagate "sticky" settings like SafeSearch, a user-defined current location, or multi-login? Is it an ad-click, in which case it needs to be tracked rigorously because real money is changing hands for it? Or is it a result click, where we want to do anything possible to avoid introducing latency in getting the user to where they need to be? Or a feature click, where the destination is our own servers? Should the browser do a full page refresh, or will it do an AJAX fetch for only part of the page? If it's an AJAX fetch, what needs to be refreshed? Does a history entry need to be pushed? Are we going to display something special if the user hits "back", like Facebook does for "More stories like this" (the feature in question was actually one of the first sites that did this, before Facebook)? What happens if we've changed the code on the server while the user has an outdated page open in their browser? What are all the browser differences in handling these?