10 ms·

Software Tests Are for Millionaires (2020)

72 points by jimhi 4y ago

danuker 4y ago
For this argument to follow, you'd have to know that Angry Birds did not use tests. I know I would have gotten tired of manually launching Angry Birds before getting the physics right.
faut_reflechir 4y ago
I appreciate the polemical framing -- not many people are comfortable taking the "wrong" side of this issue publicly and it's easy to preach purity when you have a nine-to-five. I think, though, that even if you grant that speed is more important than all else, and even if you are willing to accept a 200% interest rate on tech debt to get your releases out, you still want unit tests from day one. Done right, writing unit testing in parallel with your code speeds things up rather than slowing them down. Personally, I find this is the case for even tiny projects like a Project Euler problem unless they can be done in under about an hour.
- pclmulqdq 4y ago
  Personally, I have found that unit tests speed you up a lot, while integration tests often slow you down. In other words, if you're writing a "mock_[X]" class/function, it's probably slower than just testing with X.
  - alexgrover 4y ago
    Typically people refer to those by the opposite naming convention - unit tests just test one thing and mock any dependencies, while integration tests integrate multiple things and generally involve less mocking
commandlinefan 4y ago
I don't think I've ever seen a harder missing of a point than this one - the point of writing automated tests is to actually test scenarios that are hard to manually test. Don't be hesitant to delete tests that are no longer useful, but if you're not writing automated tests, you're not testing ENORMOUS swaths of your functionality.
- hummus_bae 4y ago
  It says "hard to manually test" which obviously includes easy test cases that you don't want to spend time on automating.
MrGilbert 4y ago
I first read "Software Tests Are for Millenials" and had to giggle. And then I thought about it, and wonder if there is a point to be made. Anyways… Yes, tests are expensive. However, once the first million pours in, you are more busy ensuring that more millions come in by building features over features. And a solid foundation will help you here.
- commandlinefan 4y ago
  > Yes, tests are expensive Yeah, but so is manual testing. You can save a fortune by eliminating the entire QA team!
  - justin_oaks 4y ago
    I know this is partially a joke, but for those who may be taking it seriously: The QA team's main value shouldn't be just repeating the testing that dev team should do in unit testing. QA's value should be to do exploratory testing. This involves trying to break things in as many ways as possible and finding things that the dev team didn't think of. Is there a text field? Try putting Chinese characters and emoji in it to see what happens. Try to add null characters. Try adding the entire text of "War and Peace". Can I upload files? See what happens when I try to upload a 1TB file, or millions of tiny files. Should the dev team be writing unit tests to handle these cases? Sure, but it helps to have someone who is explicitly trying to find all the cases the dev team missed.
- randomdata 4y ago
  > Yes, tests are expensive. Software is expensive. But do tests increase the cost? In my experience adding tests to a legacy project not designed for testing is never worth the cost, but a new project designed with testing in mind can be developed significantly quicker having tests providing valuable data about the work in progress.
- djks422 4y ago
  > once the first million pours in That was the point of the article, saying that it’s harder to get to "once" if you write "sufficient test.
gregors 4y ago
I'm a huge proponent of testing. I also believe the 20/80 rule seems to apply here. You gain enormous benefits from just a few tests, and I feel that benefit reduces as you head towards unrealistic goals of 100% coverage. Either you're writing tests or you're manually running the software (manual testing) or you're letting your users test your software in production. It might be that all of those might be the correct option as some point. However, in my experience, success in shipping code that works and finding and fixing bugs takes considerably less time when there are at least some automated tests available.
- RangerScience 4y ago
  > Either you're writing tests or you're... manual testing 100% strong agree. One of the main reasons I write tests now is just to put the code to use without having to also write all the UI/front-end/usage code. > unrealistic 100% coverage I'm not entirely sure how I've managed it (I've got a few ideas), but for the last year or so, I've been able to almost trivially reach 100% test coverage, and without a crazy ratio - one recent project that I actually counted, it was 3 LOC of test code for every line of "product" code. Time spent was more like 1:1, although I wasn't counting, and it's very blurry anyway, and testing manually wouldn't have taken all that much less time anyway. Note that that's simple code execution - that's not including anything like "100% coverage of possible inputs" - and the LOC is counting pretty formatting lines, etc. I'm at about 4x Rubocop's default maximums on function lengths, and I think abut 1.5x it's maximums on "comlexity" metrics, both for the test and the product code. Write testable code, and make sure your test code itself is code good, and everything gets better. (At least, in Ruby) Edit: Part of how I got "here", with what (AFAIK) is trivial 100% coverage, was at prior garage startup, we read Martin's "Clean Code" as a team, book-club style. Our CTO led it, so we also skipped some chapters, since we were a Ruby shop so not everything was applicable. There are some really good principles in that book, no matter your language.
  - darttree 4y ago
    100% coverage means nothing. Testing every bit is easy if you ask me but coming up with edge cases takes so much time.
RangerScience 4y ago
Yikes hard no on this. I see the point it's trying to make, but, anecdotally, for me, it's very much the case that: 1. Writing tests is a forcing function on writing better code. 2. Better code is significantly more maintainable and adaptable 3. Tests are a cheap way to invoke your code to just simply see it run. Definitely cheaper than you clicking/tapping around your product. 4. Tests suites are an extremely effective way to coordinate work with (at least) yourself. Pending and failing tests make for a great TODO list. It is absolutely possible - and easy - to write tests that get in the way. It's also easy (once you find The Way) to write tests that a force multiplier. It is a combination of writing both good code, and good test code.
- CharlieDigital 4y ago
  Agree. Testable code is inherently more decoupled and easier to swap out and toss away. Exactly what you want as a startup.
  - hbrn 4y ago
    > Testable code is inherently more decoupled I.e. more abstract with more layers. Exactly the opposite what you want as a startup. As a startup, your pivot is not going to be switching from MySQL to Postgres. Your pivot is going to be switching from consumer app to business APIs.
exabrial 4y ago
I'm surprised people still think tests _slow them down_. You do tests _all the time_ when you write software. It's a matter of making said tests repeatable.
- danuker 4y ago
  You probably repeat a large subset of them manually anyway. Automate them and save time. For instance, how many times did someone from Rovio manually shoot an Angry Bird, instead of code doing it?
indymike 4y ago
I used to think like the author, until I realized the road to $1M was paved in forward progress, and not constantly fixing regressions.
- fuzzfactor 4y ago
  >Software Tests Are for Millionaires Sounds good to me, if you're already a millionaire you can afford it, if you want to become a millionaire through software, how else is it going to be more reliable?
- hbrn 4y ago
  Automated tests are not a solution to constant regressions. The are a solution to lots of manual tests. If your team ships lots of regressions, it means your team prefers regressions over manual testing. It's a culture problem. You can't fix it with tests, only hide it temporarily.
  - indymike 4y ago
    We did.
namuol 4y ago
I mean I get that it’s supposed to be a provocative statement, but it really is just a very shallow, false dichotomy. If writing tests slows you down -this much- then yes, you’re spending too much time writing tests. But they often save you time, both as a tool for modeling your thinking about the software (being explicit about how it should work) and also in, yes, testing it. Here’s my provocation: If you don’t know how to tell the difference and make the frankly boring judgement call of whether or not you’re wasting time writing some tests, then maybe don’t work on software. “Pivot”, as they say.
- coffeebeqn 4y ago
  Usually if I don’t write a test for the core functionality it just means I’m testing in prod and end up wasting 3x more time dealing with the fire
revskill 4y ago
People blamed they can't write tests is an excuse for some configuration issues i think: It's harder to configure correct testing than testing itself. Writing tests should be the simplest thing to do. If you're onboarding a well testing setup project, then it's hard for you not to follow the TDD.
bosch_mind 4y ago
This is a stupid article. Writing tests makes me code faster AND prevents bugs. Maybe this is true if you hire junior coders?
alanfranz 4y ago
> Claiming everything must be polished and tested Polished, perfect software with 100% coverage is something that you may not want at the beginning of a business, I can agree, because of opportunity costs. But that’s quite different from zero tests. 0% coverage will impact an MVP or even a simple POC sometimes, because software is “never done”, and requires constant change.
ofcrpls 4y ago
* Regulatory Industry walks into the bar *
- not_the_fda 4y ago
  Yeah tests are less expansive than a regulatory body shutting you down or facing a wrongful death lawsuit.
turnsout 4y ago
I used to have the same attitude toward unit tests for my solo projects… But a few years ago I was working on library that had to do a lot of vector math, and I was doing SO much manual checking after every change. I wrote a bunch of tests so that I could see what was breaking when. What I didn't expect is that the tests gave me so much confidence to improve, extend and refactor the code. Before it felt so fragile, like the whole thing could crumble if I don't quadruple check my work. With the tests in place, I changed the entire underlying data type from a struct to a native simd vector in an hour. Made the changes, fixed a couple of bugs, merged the branch, boom. I don't strive for anything close to 100% coverage, but the handful of tests that cover my critical code have saved me so much time.
- Jtsummers 4y ago
  "Confidence" is a good word for it. When teams don't have good test suites (meaning mostly automated, if not entirely, and reasonably comprehensive), then you can watch as over time you get either increasing error rates or a massive slow down in team performance. Even for small changes, having no tests or primarily manual end-to-end/integration style tests means that you have little chance, unless it's a particularly good code base, of determining the actual effects statically (by analysis) or dynamically (because your tests will take too long and are likely mostly ad hoc so you will miss edge cases). Teams that don't care (or are pressured into not caring) will deliver crappier and crappier software over time. Teams that do care will eventually stop delivering software at all, or at least no more meaningful updates. All for lack of confidence.
  - deleted 4y ago
    [deleted]
  - hbrn 4y ago
    Unit tests can never give you "confidence". > you can watch as over time you get either increasing error rates or a massive slow down in team performance I hear this story repeated over and over and yet I consistently witness the opposite of what you're describing: teams start investing in unit testing and they noticeably slow down and ship more bugs at the same time. What can give you confidence is simplicity. Being able to fit the entire system in your head. Being able to say what are the side effects of your changes. If you feel like you're losing those, it's time to take a step back and rethink your architecture. But of course, nobody has the balls to take a step back and think, to admit that you failed at something. It's so much easier to just slap a bunch of unit tests on your big ball of mud and hope for the best.
moomoo11 4y ago
Is it really that hard? Write tests where you think they add value. Anyone who wants 0% or 100% are nuts and you can ignore them. Either they’re noobs or work at big tech and they want to get promoted. Money is made somewhere in the middle.
paulpauper 4y ago
The startup story is a cliché now for a reason. The guy who throws up a 3-day MVP every week for a year gets rich. Rovio made 51 failed games before they got billions from Angry Birds This is a sample size of 1. Buggy projects can lower trust, cost users. Getting users costs money so it's cheaper to make a good first impression. I always test thoroughly.
Brian_K_White 4y ago
In another thread just today was the question "what's the best advice you ever recieved about anything?" "If you don't think you have time to do it right, what makes you think you have time to do it over?"
chrismarlow9 4y ago
I mean I guess it's one way to close your tickets haha.
deleted 4y ago
[deleted]
adamwong246 4y ago
What good is velocity when the final output is a mass of bad code and broken coders? Well, it's good for _somebody_, certainly not the poor soul who was forced to mechanical turk forevermore.
Brian_K_White 4y ago
Maybe he's hiring and just looking to see who disagrees and articulates why. Ok no but it was fun to imagine.
none_to_remain 4y ago
Half the time I can't even get the bastard thing written without tests
DasCorCor 4y ago
“Software tests are for Millionaires” Yep! So if you want to be a millionaire, write tests. :)
- rvnx 4y ago
  To make a lot of money as a startup, you have to leverage debt. Weak code or weak testing are just a loan that you have to repay later (or never if the product shuts down or gets replaced by another one).
p-o 4y ago
For small projects and startups that are still figuring things out, tests are useful to prevent regressions, and I would spend my energy doing exactly that. Code coverage should only be an after thought. Whenever a bug shows up, writing a test to cover that edge case is going to help you ship with more confidence as more bugs are found and fixed.
ascotan 4y ago
Just remember that there is nothing more permanent than the prototype.
drewcoo 4y ago
It's apparently this guy: https://www.forbes.com/profile/james-steinberg/?sh=211dfcf5b34f https://www.forbes.com/profile/james-steinberg/?sh=211dfcf5b... And he can build businesses without worrying about testing. Except slow, manual testing end-to-end. Cool for him. Obviously, based on comments here, not everyone agrees. I'm a test dev. I live my work-life in this stuff: figuring out how to efficiently, effectively give non-test devs the best feedback at the right times and in the right ways to speed them up. They are individual flames and I am gasoline. Many companies start like Steinberg. And many of those automate those exact manual steps through the UI, testing everything end-to-end. That's where it stops being cool and starts being a really stupid waste of time and money. E2E tests are the least effective, slowest, flakiest way to test, it's often done really inefficiently, and I see all kinds of companies invest ridiculous sums in it. Most of those efforts slow down development. Boo! More power to Steinberg for cutting waste and doing only the testing he needs. When your next company has millions, look me up. I'm a tester. I solve business problems.
irjustin 4y ago
I largely agree with the sentiment of the article. For startups, feature tests - and mostly happy paths. Integration tests for anything super complex that you cannot reason in your head easily. The feature test catches everything breaking the happy path. If it breaks, write a spec. If you find yourself manually retesting things, SURE add a test! But 100% agreed, dogmatic TDD will kill your startup.
gavinhoward 4y ago
Absolutely not. I'm no millionaire. I wrote tests. A lot of them. I have a project that ships with FreeBSD, Mac OSX, and several Linux distros. I get probably one bug report every three months on average. Why? Because I test the snot out of that project. Writing tests means that this FOSS project of mine does not consume my life like so many can. I'll spend the time upfront. Saves gobs of time later. Oh, and time is money, so maybe this philosophy will actually help me become a millionaire.
kledru 4y ago
You may skip writing tests when building a prototype or experimenting with new ideas. Otherwise always write tests. I think it really is that easy.
ccll 4y ago
Disappointed to see that most comments here missed the point in the article. All these comments claim that automated tests make their code stable, fast, easy to refactor, etc. But the article is not talking about those characters of the code, it’s about pivoting fast and making money. Does your stable, easy-to-refactor code directly make you good money? If yes, then please talk more about it, this is the correct response to the article, please don’t just claim your code is stable and easy-to-refactoring, and in the meanwhile have nothing to do with making money.
- whateveracct 4y ago
  I don't want to maximize money up the food chain. I want my job to not be a pain in the ass so I don't have to work so hard. That effectively makes me money.
  - ccll 4y ago
    That’s alright and a very valid point, I too will appreciate it if I’m working for an employer. But the OP’s context is about creating your own startup, then things work very differently now, and that’s the reason I feel disappointed, because people are not thinking and replying in the OP’s context, but rather in their own, this makes their point pointless, including yours I’m afraid.
    - whateveracct 4y ago
      I've founded my own startup and even then I wish I had spent more time on good code. I wasn't alone in it, and definitely got pushed away from that by cofounders. But the startup lasted a long time and it would've been worth it.
- draw_down 4y ago
  [dead]
draw_down 4y ago
[dead]
ccll 4y ago
Some how I agree with the author. In the last decade I’ve known and worked with dozens of one-man tech startup founders, they basically fall into two categories, the ‘product’ group and the ‘engineer’ group. The ‘product’ guys tend to use boring technology like PHP or Python to duct tape their product and ship it ASAP and pivot very quickly if things didn’t work out, they NEVER write a single test. The ‘engineer’ guys tend to use bleeding edge technology like meteor/nextjs/svelt/cypress/terraform/k8s/nomad/netlify/cloudfunction/(insert a hundred tech here)/… and polish their product alot before launching, including writing a lot tests. To be honest, the ‘product’ group are way more successful financially than the ‘engineer’ group in my small sample dataset. I’m in the ‘engineer’ group myself, and I tend to polish a lot, lately I’ve been thinking about the two different mindsets, and I get to the same conclusion with the OP’s article, that code quality and maintenancibility are great ideas, but unfortunately *for a startup* they are not directly related with making money, at least not linearly directly related. Startup is about exploring undiscovered market opportunities, and it’s full of uncertainties and assumptions, your job is to find out whether the assumptions are true or false, not to write the most stable and easy-to-refactor code. Guess what? Tests are about certainty, about known rules. In week 1 you write lots of tests to cover your 80% business logic, and in week 2 you find out your original idea does not work and need to change course, and all the written tests are garbage now, then week 3 same thing happen again…and all your precious time are wasted chasing the wrong target. One thing I disagree though is that you don’t need to be literally rich to begin writing tests, as soon as your business logic and model is proven, like making positive cash flow or user growth is promising, then it’s the right time to write some tests, to improve the code quality and your confidence of future refactoring.
satisfice 4y ago
This article is insight free. It’s essentially saying that being responsible and caring is for suckers. Of course that is true. So what? I am a sucker because I want to do good work instead of sticking it to my clients and users. Yep. Testing is about learning what your wishful and arrogant ambitions have produced. Of course you can shrug and say you’d rather gamble and get rich before your users and investors find out that your product has terrible performance or security or functionality problems. This is not news. I can make more money by mistreating my wife and family, and cheating on my taxes, too. I don’t do this because I have self-respect, and the respect of others matters to me. Companies never test more than they think they need to. There is no pandemic of over-testing. This article is a waste of good disk space.
pixel_tracing 4y ago
I’d love to see this attitude in a fintech application. You will lose your users very quickly and not to mention potential fines from the SEC or <insert_other_regulatory_body>
AtNightWeCode 4y ago
With the tools of today, writing automated tests is a very fast process especially with the help of Github Copilot and similar. Running the tests is also fast and you can alter the code while debugging the tests in some langs making it a fast process. I use it for POCs all the time. At the end of the day automated tests is a tool among others. Use it when it makes sense. Just don't fall into the trap of maintaining large sets of more or less useless tests. Delete them.

72 points by jimhi 4y ago

For this argument to follow, you'd have to know that Angry Birds did not use tests. I know I would have gotten tired of manually launching Angry Birds before getting the physics right.

faut_reflechir 4y ago

I appreciate the polemical framing -- not many people are comfortable taking the "wrong" side of this issue publicly and it's easy to preach purity when you have a nine-to-five. I think, though, that even if you grant that speed is more important than all else, and even if you are willing to accept a 200% interest rate on tech debt to get your releases out, you still want unit tests from day one. Done right, writing unit testing in parallel with your code speeds things up rather than slowing them down. Personally, I find this is the case for even tiny projects like a Project Euler problem unless they can be done in under about an hour.

pclmulqdq 4y ago
Personally, I have found that unit tests speed you up a lot, while integration tests often slow you down. In other words, if you're writing a "mock_[X]" class/function, it's probably slower than just testing with X.
- alexgrover 4y ago
  Typically people refer to those by the opposite naming convention - unit tests just test one thing and mock any dependencies, while integration tests integrate multiple things and generally involve less mocking

commandlinefan 4y ago

I don't think I've ever seen a harder missing of a point than this one - the point of writing automated tests is to actually test scenarios that are hard to manually test. Don't be hesitant to delete tests that are no longer useful, but if you're not writing automated tests, you're not testing ENORMOUS swaths of your functionality.

hummus_bae 4y ago
It says "hard to manually test" which obviously includes easy test cases that you don't want to spend time on automating.

MrGilbert 4y ago

I first read "Software Tests Are for Millenials" and had to giggle. And then I thought about it, and wonder if there is a point to be made. Anyways… Yes, tests are expensive. However, once the first million pours in, you are more busy ensuring that more millions come in by building features over features. And a solid foundation will help you here.

commandlinefan 4y ago
> Yes, tests are expensive Yeah, but so is manual testing. You can save a fortune by eliminating the entire QA team!
- justin_oaks 4y ago
  I know this is partially a joke, but for those who may be taking it seriously: The QA team's main value shouldn't be just repeating the testing that dev team should do in unit testing. QA's value should be to do exploratory testing. This involves trying to break things in as many ways as possible and finding things that the dev team didn't think of. Is there a text field? Try putting Chinese characters and emoji in it to see what happens. Try to add null characters. Try adding the entire text of "War and Peace". Can I upload files? See what happens when I try to upload a 1TB file, or millions of tiny files. Should the dev team be writing unit tests to handle these cases? Sure, but it helps to have someone who is explicitly trying to find all the cases the dev team missed.
randomdata 4y ago
> Yes, tests are expensive. Software is expensive. But do tests increase the cost? In my experience adding tests to a legacy project not designed for testing is never worth the cost, but a new project designed with testing in mind can be developed significantly quicker having tests providing valuable data about the work in progress.
djks422 4y ago
> once the first million pours in That was the point of the article, saying that it’s harder to get to "once" if you write "sufficient test.

gregors 4y ago

I'm a huge proponent of testing. I also believe the 20/80 rule seems to apply here. You gain enormous benefits from just a few tests, and I feel that benefit reduces as you head towards unrealistic goals of 100% coverage. Either you're writing tests or you're manually running the software (manual testing) or you're letting your users test your software in production. It might be that all of those might be the correct option as some point. However, in my experience, success in shipping code that works and finding and fixing bugs takes considerably less time when there are at least some automated tests available.

RangerScience 4y ago
> Either you're writing tests or you're... manual testing 100% strong agree. One of the main reasons I write tests now is just to put the code to use without having to also write all the UI/front-end/usage code. > unrealistic 100% coverage I'm not entirely sure how I've managed it (I've got a few ideas), but for the last year or so, I've been able to almost trivially reach 100% test coverage, and without a crazy ratio - one recent project that I actually counted, it was 3 LOC of test code for every line of "product" code. Time spent was more like 1:1, although I wasn't counting, and it's very blurry anyway, and testing manually wouldn't have taken all that much less time anyway. Note that that's simple code execution - that's not including anything like "100% coverage of possible inputs" - and the LOC is counting pretty formatting lines, etc. I'm at about 4x Rubocop's default maximums on function lengths, and I think abut 1.5x it's maximums on "comlexity" metrics, both for the test and the product code. Write testable code, and make sure your test code itself is code good, and everything gets better. (At least, in Ruby) Edit: Part of how I got "here", with what (AFAIK) is trivial 100% coverage, was at prior garage startup, we read Martin's "Clean Code" as a team, book-club style. Our CTO led it, so we also skipped some chapters, since we were a Ruby shop so not everything was applicable. There are some really good principles in that book, no matter your language.
- darttree 4y ago
  100% coverage means nothing. Testing every bit is easy if you ask me but coming up with edge cases takes so much time.

RangerScience 4y ago

Yikes hard no on this. I see the point it's trying to make, but, anecdotally, for me, it's very much the case that: 1. Writing tests is a forcing function on writing better code. 2. Better code is significantly more maintainable and adaptable 3. Tests are a cheap way to invoke your code to just simply see it run. Definitely cheaper than you clicking/tapping around your product. 4. Tests suites are an extremely effective way to coordinate work with (at least) yourself. Pending and failing tests make for a great TODO list. It is absolutely possible - and easy - to write tests that get in the way. It's also easy (once you find The Way) to write tests that a force multiplier. It is a combination of writing both good code, and good test code.

CharlieDigital 4y ago
Agree. Testable code is inherently more decoupled and easier to swap out and toss away. Exactly what you want as a startup.
- hbrn 4y ago
  > Testable code is inherently more decoupled I.e. more abstract with more layers. Exactly the opposite what you want as a startup. As a startup, your pivot is not going to be switching from MySQL to Postgres. Your pivot is going to be switching from consumer app to business APIs.

exabrial 4y ago

I'm surprised people still think tests _slow them down_. You do tests _all the time_ when you write software. It's a matter of making said tests repeatable.

danuker 4y ago
You probably repeat a large subset of them manually anyway. Automate them and save time. For instance, how many times did someone from Rovio manually shoot an Angry Bird, instead of code doing it?

indymike 4y ago

I used to think like the author, until I realized the road to $1M was paved in forward progress, and not constantly fixing regressions.

fuzzfactor 4y ago
>Software Tests Are for Millionaires Sounds good to me, if you're already a millionaire you can afford it, if you want to become a millionaire through software, how else is it going to be more reliable?
hbrn 4y ago
Automated tests are not a solution to constant regressions. The are a solution to lots of manual tests. If your team ships lots of regressions, it means your team prefers regressions over manual testing. It's a culture problem. You can't fix it with tests, only hide it temporarily.
- indymike 4y ago
  We did.

namuol 4y ago

I mean I get that it’s supposed to be a provocative statement, but it really is just a very shallow, false dichotomy. If writing tests slows you down -this much- then yes, you’re spending too much time writing tests. But they often save you time, both as a tool for modeling your thinking about the software (being explicit about how it should work) and also in, yes, testing it. Here’s my provocation: If you don’t know how to tell the difference and make the frankly boring judgement call of whether or not you’re wasting time writing some tests, then maybe don’t work on software. “Pivot”, as they say.

coffeebeqn 4y ago
Usually if I don’t write a test for the core functionality it just means I’m testing in prod and end up wasting 3x more time dealing with the fire

revskill 4y ago

People blamed they can't write tests is an excuse for some configuration issues i think: It's harder to configure correct testing than testing itself. Writing tests should be the simplest thing to do. If you're onboarding a well testing setup project, then it's hard for you not to follow the TDD.

bosch_mind 4y ago

This is a stupid article. Writing tests makes me code faster AND prevents bugs. Maybe this is true if you hire junior coders?

alanfranz 4y ago

> Claiming everything must be polished and tested Polished, perfect software with 100% coverage is something that you may not want at the beginning of a business, I can agree, because of opportunity costs. But that’s quite different from zero tests. 0% coverage will impact an MVP or even a simple POC sometimes, because software is “never done”, and requires constant change.

ofcrpls 4y ago

* Regulatory Industry walks into the bar *

not_the_fda 4y ago
Yeah tests are less expansive than a regulatory body shutting you down or facing a wrongful death lawsuit.

turnsout 4y ago

I used to have the same attitude toward unit tests for my solo projects… But a few years ago I was working on library that had to do a lot of vector math, and I was doing SO much manual checking after every change. I wrote a bunch of tests so that I could see what was breaking when. What I didn't expect is that the tests gave me so much confidence to improve, extend and refactor the code. Before it felt so fragile, like the whole thing could crumble if I don't quadruple check my work. With the tests in place, I changed the entire underlying data type from a struct to a native simd vector in an hour. Made the changes, fixed a couple of bugs, merged the branch, boom. I don't strive for anything close to 100% coverage, but the handful of tests that cover my critical code have saved me so much time.

Jtsummers 4y ago
"Confidence" is a good word for it. When teams don't have good test suites (meaning mostly automated, if not entirely, and reasonably comprehensive), then you can watch as over time you get either increasing error rates or a massive slow down in team performance. Even for small changes, having no tests or primarily manual end-to-end/integration style tests means that you have little chance, unless it's a particularly good code base, of determining the actual effects statically (by analysis) or dynamically (because your tests will take too long and are likely mostly ad hoc so you will miss edge cases). Teams that don't care (or are pressured into not caring) will deliver crappier and crappier software over time. Teams that do care will eventually stop delivering software at all, or at least no more meaningful updates. All for lack of confidence.
- deleted 4y ago
  [deleted]
- hbrn 4y ago
  Unit tests can never give you "confidence". > you can watch as over time you get either increasing error rates or a massive slow down in team performance I hear this story repeated over and over and yet I consistently witness the opposite of what you're describing: teams start investing in unit testing and they noticeably slow down and ship more bugs at the same time. What can give you confidence is simplicity. Being able to fit the entire system in your head. Being able to say what are the side effects of your changes. If you feel like you're losing those, it's time to take a step back and rethink your architecture. But of course, nobody has the balls to take a step back and think, to admit that you failed at something. It's so much easier to just slap a bunch of unit tests on your big ball of mud and hope for the best.

moomoo11 4y ago

Is it really that hard? Write tests where you think they add value. Anyone who wants 0% or 100% are nuts and you can ignore them. Either they’re noobs or work at big tech and they want to get promoted. Money is made somewhere in the middle.

paulpauper 4y ago

The startup story is a cliché now for a reason. The guy who throws up a 3-day MVP every week for a year gets rich. Rovio made 51 failed games before they got billions from Angry Birds This is a sample size of 1. Buggy projects can lower trust, cost users. Getting users costs money so it's cheaper to make a good first impression. I always test thoroughly.

Brian_K_White 4y ago

In another thread just today was the question "what's the best advice you ever recieved about anything?" "If you don't think you have time to do it right, what makes you think you have time to do it over?"

chrismarlow9 4y ago

I mean I guess it's one way to close your tickets haha.

deleted 4y ago

[deleted]

adamwong246 4y ago

What good is velocity when the final output is a mass of bad code and broken coders? Well, it's good for _somebody_, certainly not the poor soul who was forced to mechanical turk forevermore.

Brian_K_White 4y ago

Maybe he's hiring and just looking to see who disagrees and articulates why. Ok no but it was fun to imagine.

none_to_remain 4y ago

Half the time I can't even get the bastard thing written without tests

DasCorCor 4y ago

“Software tests are for Millionaires” Yep! So if you want to be a millionaire, write tests. :)

rvnx 4y ago
To make a lot of money as a startup, you have to leverage debt. Weak code or weak testing are just a loan that you have to repay later (or never if the product shuts down or gets replaced by another one).

p-o 4y ago

For small projects and startups that are still figuring things out, tests are useful to prevent regressions, and I would spend my energy doing exactly that. Code coverage should only be an after thought. Whenever a bug shows up, writing a test to cover that edge case is going to help you ship with more confidence as more bugs are found and fixed.

ascotan 4y ago

Just remember that there is nothing more permanent than the prototype.

drewcoo 4y ago

It's apparently this guy: https://www.forbes.com/profile/james-steinberg/?sh=211dfcf5b34f https://www.forbes.com/profile/james-steinberg/?sh=211dfcf5b... And he can build businesses without worrying about testing. Except slow, manual testing end-to-end. Cool for him. Obviously, based on comments here, not everyone agrees. I'm a test dev. I live my work-life in this stuff: figuring out how to efficiently, effectively give non-test devs the best feedback at the right times and in the right ways to speed them up. They are individual flames and I am gasoline. Many companies start like Steinberg. And many of those automate those exact manual steps through the UI, testing everything end-to-end. That's where it stops being cool and starts being a really stupid waste of time and money. E2E tests are the least effective, slowest, flakiest way to test, it's often done really inefficiently, and I see all kinds of companies invest ridiculous sums in it. Most of those efforts slow down development. Boo! More power to Steinberg for cutting waste and doing only the testing he needs. When your next company has millions, look me up. I'm a tester. I solve business problems.

irjustin 4y ago

I largely agree with the sentiment of the article. For startups, feature tests - and mostly happy paths. Integration tests for anything super complex that you cannot reason in your head easily. The feature test catches everything breaking the happy path. If it breaks, write a spec. If you find yourself manually retesting things, SURE add a test! But 100% agreed, dogmatic TDD will kill your startup.

gavinhoward 4y ago

Absolutely not. I'm no millionaire. I wrote tests. A lot of them. I have a project that ships with FreeBSD, Mac OSX, and several Linux distros. I get probably one bug report every three months on average. Why? Because I test the snot out of that project. Writing tests means that this FOSS project of mine does not consume my life like so many can. I'll spend the time upfront. Saves gobs of time later. Oh, and time is money, so maybe this philosophy will actually help me become a millionaire.

kledru 4y ago

You may skip writing tests when building a prototype or experimenting with new ideas. Otherwise always write tests. I think it really is that easy.

ccll 4y ago

Disappointed to see that most comments here missed the point in the article. All these comments claim that automated tests make their code stable, fast, easy to refactor, etc. But the article is not talking about those characters of the code, it’s about pivoting fast and making money. Does your stable, easy-to-refactor code directly make you good money? If yes, then please talk more about it, this is the correct response to the article, please don’t just claim your code is stable and easy-to-refactoring, and in the meanwhile have nothing to do with making money.

whateveracct 4y ago
I don't want to maximize money up the food chain. I want my job to not be a pain in the ass so I don't have to work so hard. That effectively makes me money.
- ccll 4y ago
  That’s alright and a very valid point, I too will appreciate it if I’m working for an employer. But the OP’s context is about creating your own startup, then things work very differently now, and that’s the reason I feel disappointed, because people are not thinking and replying in the OP’s context, but rather in their own, this makes their point pointless, including yours I’m afraid.
  - whateveracct 4y ago
    I've founded my own startup and even then I wish I had spent more time on good code. I wasn't alone in it, and definitely got pushed away from that by cofounders. But the startup lasted a long time and it would've been worth it.
draw_down 4y ago
[dead]

draw_down 4y ago

[dead]

ccll 4y ago

Some how I agree with the author. In the last decade I’ve known and worked with dozens of one-man tech startup founders, they basically fall into two categories, the ‘product’ group and the ‘engineer’ group. The ‘product’ guys tend to use boring technology like PHP or Python to duct tape their product and ship it ASAP and pivot very quickly if things didn’t work out, they NEVER write a single test. The ‘engineer’ guys tend to use bleeding edge technology like meteor/nextjs/svelt/cypress/terraform/k8s/nomad/netlify/cloudfunction/(insert a hundred tech here)/… and polish their product alot before launching, including writing a lot tests. To be honest, the ‘product’ group are way more successful financially than the ‘engineer’ group in my small sample dataset. I’m in the ‘engineer’ group myself, and I tend to polish a lot, lately I’ve been thinking about the two different mindsets, and I get to the same conclusion with the OP’s article, that code quality and maintenancibility are great ideas, but unfortunately *for a startup* they are not directly related with making money, at least not linearly directly related. Startup is about exploring undiscovered market opportunities, and it’s full of uncertainties and assumptions, your job is to find out whether the assumptions are true or false, not to write the most stable and easy-to-refactor code. Guess what? Tests are about certainty, about known rules. In week 1 you write lots of tests to cover your 80% business logic, and in week 2 you find out your original idea does not work and need to change course, and all the written tests are garbage now, then week 3 same thing happen again…and all your precious time are wasted chasing the wrong target. One thing I disagree though is that you don’t need to be literally rich to begin writing tests, as soon as your business logic and model is proven, like making positive cash flow or user growth is promising, then it’s the right time to write some tests, to improve the code quality and your confidence of future refactoring.

satisfice 4y ago

This article is insight free. It’s essentially saying that being responsible and caring is for suckers. Of course that is true. So what? I am a sucker because I want to do good work instead of sticking it to my clients and users. Yep. Testing is about learning what your wishful and arrogant ambitions have produced. Of course you can shrug and say you’d rather gamble and get rich before your users and investors find out that your product has terrible performance or security or functionality problems. This is not news. I can make more money by mistreating my wife and family, and cheating on my taxes, too. I don’t do this because I have self-respect, and the respect of others matters to me. Companies never test more than they think they need to. There is no pandemic of over-testing. This article is a waste of good disk space.

pixel_tracing 4y ago

I’d love to see this attitude in a fintech application. You will lose your users very quickly and not to mention potential fines from the SEC or <insert_other_regulatory_body>

AtNightWeCode 4y ago

With the tools of today, writing automated tests is a very fast process especially with the help of Github Copilot and similar. Running the tests is also fast and you can alter the code while debugging the tests in some langs making it a fast process. I use it for POCs all the time. At the end of the day automated tests is a tool among others. Use it when it makes sense. Just don't fall into the trap of maintaining large sets of more or less useless tests. Delete them.