7 ms·

> My first employer, the online dating site OkCupid, didn’t harp on testing either. In part, this was because our company was so small. Only seven fellow engine

by gred 8y ago

> My first employer, the online dating site OkCupid, didn’t harp on testing either. In part, this was because our company was so small. Only seven fellow engineers and I maintained all the code running on our servers—and making tests work was time-consuming and error-prone. “We can’t sacrifice forward momentum for technical debt,” then-CEO Mike Maxim told me, referring to the cost of engineers building behind-the-scenes tech instead of user-facing features. “Users don’t care.” He thought of testing frameworks as somewhat academic, more lofty than practical. I found this paragraph interesting because my experience with my one-man side project is the opposite: I'm so small, and time is so scarce (I have a separate full-time job), that I can't afford to fix bugs on somebody else's schedule when they find them... instead I spend a large amount of time writing tests up front for new features so that I find the bugs on my schedule (i.e. late at night when I don't have other commitments).

toss1 8y ago
Yup, classic. "...no time/resources to do it right in the first place, but plenty of time/resources to fix it when the customers complain..." I always thought it better to find the bugs in-house before shipping, but so many others don't see it...
- nitrogen 8y ago
  The logic kind of makes sense if you consider that customers will only find a small fraction of the bugs that your team will. Sometimes the cost of removing all bugs is greater than the cost of losing customers to the few unlucky bugs that get found. But you really need experienced people deciding where that tradeoff lies for each company and project.
  - cs02rm0 8y ago
    I think that's the bottom line. I've done virtually zero unit testing on a project before when there was barely time to write the core code. There wasn't much business logic, lots of I/O and no user inputs or browsers to worry about and I was leading a small team of four so it was relatively well suited to it. Still, I was nervous as it was quite high profile and unit test coverage was about the only metric management knew to ask for. It was replacing a critical third party system being turned off on a fixed date and we barely made it. 6 months after being switched on in production we'd had zero bugs reported and I moved on. I'm confident in hindsight that I made the right call for the circumstances, but having it funded properly would have made me sweat a bit less.
  - toss1 8y ago
    Yes, each individual customer is likely to find only a fraction of the bugs that your team finds. But, many customers will collectively find much more. the cost is more than an individual customer. It is everyone that customer comes into contact with and spreads word of the grief your bugs caused them. Reputation can be very fragile. The "deciding where the tradeoff lies" can easily lead to the Ford Pinto fiasco, where it was decided that the costs of the few lawsuits were less than the costs of the fix. After they'd _killed_ dozens of people and then got massive fines, that decision didn't look so good. Quality is its own excuse. And yes, you cannot let the Perfect be the enemy of the Good and insist in fixing every nit, but Good should be Damn Good, not merely 'just ok, ship it infested with known bugs'. You are supposed to make people's lives easier with a product, not merely extract their money.
- megablast 8y ago
  You are acting like most bugs aren't already caught in the dev process. No on writes code and just deploys. Writing extra tests that cover everything for a small project that you are working on yourself is a waste of time.
  - aprdm 8y ago
    > No on writes code and just deploys. You would be - very - surprised how some big non tech companies (FANG) run.
  - LandR 8y ago
    > No on writes code and just deploys. Oh yes they do! Hell, I've seen dlls copied from a server, disassambled into code, code changed, rebuilt and copied back to server.... Most devs here write code then copy their bin folder to a production server and deploy it.
  - rhencke 8y ago
    > No on writes code and just deploys. I wish this was true. I stopped keeping track of how many times I've seen people break stuff because they deploy code that simply doesn't work, because they figured they could just eyeball it rather than run it and test it out. Not everyone cares deeply about the quality of their work.
  - jkubicek 8y ago
    Writing tests for small, personal projects is hugely valuable for me. Running a well-written test lets me iterate much faster than running the entire script or booting up the whole app and poking at the UI.
spamizbad 8y ago
This is a smart way to think of testing and will discover bugs when your understanding of the code is the most lucid. But it does require some nuance. Unfortunately, some people think writing tests or test harnesses are wasteful because they think of shops that insist on 90%+ coverage and have test suites that are easily 5-10x the size of the application's codebase.... so they decide to forgo them completely. When in reality, the answer is probably something like: you have at least 25% of your codebase that would greatly benefit from testing, even if they're just "smoke tests"
- deleted 8y ago
  [deleted]
sidstling 8y ago
Don’t you take this bit about out of context? If the premise the article builds around faulty tests is correct, then stuff like unit-testing is mostly a waste of time since you’ll still need to fix things when the servers are burning. We’ve done TDD, projects with coverage only vital parts and no software-tests where I work and there is no difference in production on smaller projects. Maybe we’re terrible at writing tests, but if so, then it’s still something we have to deal with. So I think it’s hard to outline the correct amount of testing for every case. I don’t think I would want to work on a major system with multiple contributors if there weren’t automatic tests, so it’s not like I’m against testing either, I just don’t think it’s always a holy grail.
- tinco 8y ago
  I think the advantages of tests, TDD or otherwise, are not easily measured. Everyone knows cowboy coding up small projects works and through manual testing they can quickly be deployed with reasonable quality, especially if the developers are experienced. The tests shine in the follow-up phases, when code needs to be changed, when inexperienced hands have to touch the code base, and when there's simply too much going on to fit in one mind. If you build your product with the future in mind, then you write tests. If you've got VC money and you need to sprint to some goal before the puck gets in front of your stick, then I can see the logic in skipping tests. If the tech debt hits you before you reach that point though, you're in deep shit. It might also be the reason that innovative companies often seem to technologically stall right after hitting mainstream.

by gred 8y ago

toss1 8y ago

Yup, classic. "...no time/resources to do it right in the first place, but plenty of time/resources to fix it when the customers complain..." I always thought it better to find the bugs in-house before shipping, but so many others don't see it...

nitrogen 8y ago
The logic kind of makes sense if you consider that customers will only find a small fraction of the bugs that your team will. Sometimes the cost of removing all bugs is greater than the cost of losing customers to the few unlucky bugs that get found. But you really need experienced people deciding where that tradeoff lies for each company and project.
- cs02rm0 8y ago
  I think that's the bottom line. I've done virtually zero unit testing on a project before when there was barely time to write the core code. There wasn't much business logic, lots of I/O and no user inputs or browsers to worry about and I was leading a small team of four so it was relatively well suited to it. Still, I was nervous as it was quite high profile and unit test coverage was about the only metric management knew to ask for. It was replacing a critical third party system being turned off on a fixed date and we barely made it. 6 months after being switched on in production we'd had zero bugs reported and I moved on. I'm confident in hindsight that I made the right call for the circumstances, but having it funded properly would have made me sweat a bit less.
- toss1 8y ago
  Yes, each individual customer is likely to find only a fraction of the bugs that your team finds. But, many customers will collectively find much more. the cost is more than an individual customer. It is everyone that customer comes into contact with and spreads word of the grief your bugs caused them. Reputation can be very fragile. The "deciding where the tradeoff lies" can easily lead to the Ford Pinto fiasco, where it was decided that the costs of the few lawsuits were less than the costs of the fix. After they'd _killed_ dozens of people and then got massive fines, that decision didn't look so good. Quality is its own excuse. And yes, you cannot let the Perfect be the enemy of the Good and insist in fixing every nit, but Good should be Damn Good, not merely 'just ok, ship it infested with known bugs'. You are supposed to make people's lives easier with a product, not merely extract their money.
megablast 8y ago
You are acting like most bugs aren't already caught in the dev process. No on writes code and just deploys. Writing extra tests that cover everything for a small project that you are working on yourself is a waste of time.
- aprdm 8y ago
  > No on writes code and just deploys. You would be - very - surprised how some big non tech companies (FANG) run.
- LandR 8y ago
  > No on writes code and just deploys. Oh yes they do! Hell, I've seen dlls copied from a server, disassambled into code, code changed, rebuilt and copied back to server.... Most devs here write code then copy their bin folder to a production server and deploy it.
- rhencke 8y ago
  > No on writes code and just deploys. I wish this was true. I stopped keeping track of how many times I've seen people break stuff because they deploy code that simply doesn't work, because they figured they could just eyeball it rather than run it and test it out. Not everyone cares deeply about the quality of their work.
- jkubicek 8y ago
  Writing tests for small, personal projects is hugely valuable for me. Running a well-written test lets me iterate much faster than running the entire script or booting up the whole app and poking at the UI.

spamizbad 8y ago

This is a smart way to think of testing and will discover bugs when your understanding of the code is the most lucid. But it does require some nuance. Unfortunately, some people think writing tests or test harnesses are wasteful because they think of shops that insist on 90%+ coverage and have test suites that are easily 5-10x the size of the application's codebase.... so they decide to forgo them completely. When in reality, the answer is probably something like: you have at least 25% of your codebase that would greatly benefit from testing, even if they're just "smoke tests"

deleted 8y ago
[deleted]

sidstling 8y ago

Don’t you take this bit about out of context? If the premise the article builds around faulty tests is correct, then stuff like unit-testing is mostly a waste of time since you’ll still need to fix things when the servers are burning. We’ve done TDD, projects with coverage only vital parts and no software-tests where I work and there is no difference in production on smaller projects. Maybe we’re terrible at writing tests, but if so, then it’s still something we have to deal with. So I think it’s hard to outline the correct amount of testing for every case. I don’t think I would want to work on a major system with multiple contributors if there weren’t automatic tests, so it’s not like I’m against testing either, I just don’t think it’s always a holy grail.

tinco 8y ago
I think the advantages of tests, TDD or otherwise, are not easily measured. Everyone knows cowboy coding up small projects works and through manual testing they can quickly be deployed with reasonable quality, especially if the developers are experienced. The tests shine in the follow-up phases, when code needs to be changed, when inexperienced hands have to touch the code base, and when there's simply too much going on to fit in one mind. If you build your product with the future in mind, then you write tests. If you've got VC money and you need to sprint to some goal before the puck gets in front of your stick, then I can see the logic in skipping tests. If the tech debt hits you before you reach that point though, you're in deep shit. It might also be the reason that innovative companies often seem to technologically stall right after hitting mainstream.