4 ms·

There seems to be a general assumption that writing [unit?] tests is always the "right thing" to do, even at a startup, and no matter what the code in question

There seems to be a general assumption that writing [unit?] tests is always the "right thing" to do, even at a startup, and no matter what the code in question does. In my experience that's absolutely not the case.

timr 15y ago
Just to be clear: Bill's opinion on this isn't universally shared at JTV. When you're two guys in a garage, unit tests probably slow you down. But at some point, they begin to dramatically speed things up -- working on someone other person's code becomes orders of magnitude easier when there's a comprehensive test suite to validate your changes. The problem is that at some point between first employee and a dozen (or more) engineers, you start to wish you'd spent a bit more time on testing code. Writing those tests from scratch becomes "a project", and it only gets done with herculean effort.
- jdeseno 15y ago
  Having been at JTV, looking back it almost seems insane to try to do as much work as we we're without good coverage. If it matters, it should get a test. I think a lot of people also see tests as slowing things down but, they also provide a lot of value in documenting intentions, make upgrades easier, etc.
  - timr 15y ago
    It also matters a lot more when you're hiring relatively new/inexperienced people, and throwing them in the deep end. I disagree with Bill mostly because I see the purpose of testing differently than he: the point is not to catch All Of The Bugs; the point is to make it easier to confidently work on code that you didn't write. There's a lot of counter-intuitive stuff that happens in production code for a big website, and it's not totally fair to throw inexperienced people at it without at least some safety belts in place.
    - abstractbill 15y ago
      Yeah, this is a fair point. I think, as much as I dislike pair-programming overall, that's something I would prefer to use as a safety belt here (because it's only expensive for the period during which a new employee is still learning). Pair programming hasn't ever been tried at justin.tv either though, as far as I recall. I'm probably going to give it a try at ZeroCater for new engineers actually (we're hiring!).
mikeryan 15y ago
While I'm not a unit test nazi by any means, if you don't have a QA and deployment process in place to catch things like this unit, and functional tests may be time well spent.
steve-howard 15y ago
And as a lack of testing endangered the payment system, I'd say it would have been better for them to have some tests in place.
ctide 15y ago
The only time tests aren't the right thing to do is when you can guarantee that you will completely trash that system at some point and that code is never going to see the light of a production machine. Most code gets refactored out of it's initial state, and that's where your tests offer huge benefits. I can say that I didn't truly appreciate the full value of tests until the first time I had to do a massive refactoring of a well tested app. Being able to change core fundamental components of a system and trust that everything will Just Work™ when you're done is an awesome feeling.

timr 15y ago

Just to be clear: Bill's opinion on this isn't universally shared at JTV. When you're two guys in a garage, unit tests probably slow you down. But at some point, they begin to dramatically speed things up -- working on someone other person's code becomes orders of magnitude easier when there's a comprehensive test suite to validate your changes. The problem is that at some point between first employee and a dozen (or more) engineers, you start to wish you'd spent a bit more time on testing code. Writing those tests from scratch becomes "a project", and it only gets done with herculean effort.

jdeseno 15y ago
Having been at JTV, looking back it almost seems insane to try to do as much work as we we're without good coverage. If it matters, it should get a test. I think a lot of people also see tests as slowing things down but, they also provide a lot of value in documenting intentions, make upgrades easier, etc.
- timr 15y ago
  It also matters a lot more when you're hiring relatively new/inexperienced people, and throwing them in the deep end. I disagree with Bill mostly because I see the purpose of testing differently than he: the point is not to catch All Of The Bugs; the point is to make it easier to confidently work on code that you didn't write. There's a lot of counter-intuitive stuff that happens in production code for a big website, and it's not totally fair to throw inexperienced people at it without at least some safety belts in place.
  - abstractbill 15y ago
    Yeah, this is a fair point. I think, as much as I dislike pair-programming overall, that's something I would prefer to use as a safety belt here (because it's only expensive for the period during which a new employee is still learning). Pair programming hasn't ever been tried at justin.tv either though, as far as I recall. I'm probably going to give it a try at ZeroCater for new engineers actually (we're hiring!).

mikeryan 15y ago

While I'm not a unit test nazi by any means, if you don't have a QA and deployment process in place to catch things like this unit, and functional tests may be time well spent.

steve-howard 15y ago

And as a lack of testing endangered the payment system, I'd say it would have been better for them to have some tests in place.

ctide 15y ago

The only time tests aren't the right thing to do is when you can guarantee that you will completely trash that system at some point and that code is never going to see the light of a production machine. Most code gets refactored out of it's initial state, and that's where your tests offer huge benefits. I can say that I didn't truly appreciate the full value of tests until the first time I had to do a massive refactoring of a well tested app. Being able to change core fundamental components of a system and trust that everything will Just Work™ when you're done is an awesome feeling.