9 ms·

Conventional wisdom as an anti-pattern

53 points by qudat 4y ago

codeulike 4y ago
I've written so many automated tests, but how many have actually caught something important? A couple dozen? Yeah I'm glad TDD isn't the dominant religion any more. If you write a test and it never fails throughout the lifetime of the project, thats wasted time.
- moron4hire 4y ago
  The last time I found TDD to be useful was when I was implementing some code for processing of data feeds according to a pretty well-defined specification. The spec translated to tests quite easily and then the tests very readily pointed out where the data did not match the spec 100%. Luckily, we were in a position to modify the data-generating side of things, too, to make sure everything was in-spec. But most of the time, I'm dealing with user interfaces, service integrations, and data munging. I have "test benches" instead of unit tests for these. They're like "minimal reproductions", fully stand-alone programs that exercise the code in the "expected" way. The big issue is that the "expected way" could change at practically any minute, depending on business need. There's no "spec" other than, "are people's expectations for a usable, comfortable system being met?" We want them to be changeable at the drop of a hat. And those test benches help make those changes possible, by not having to work in the full UI to change the behavior of one control, one service, etc.
- dllthomas 4y ago
  I want to push back on this in two ways. First, it's my understanding (as someone who usually doesn't TDD and hasn't read/thought that deeply about it in a while) that the core of TDD (vs other testing doctrines) was to iterate by making a failing test, then making it pass. If that's correct, then if you're writing tests that never fail throughout the lifetime of the project then you're not doing TDD, though culture does weird things to practices and definitions and I'm sure there's places that called any damned thing TDD when it had more hype. Second, I think a test (or static check) that never fails can be useful if it lets the programmer pay less attention to making sure the program is correct in that particular way (freeing up bandwidth for other important considerations), and as a communication tool demonstrating that the system does in fact work the way it does. Whether those benefits are worth it depends on a bunch of features of the particular context, including how long it takes to write/maintain the test (although if it never fails then at least you probably don't need to update it) and how long it takes to run the test.
  - qudat 4y ago
    > If that's correct, then if you're writing tests that never fail throughout the lifetime of the project then you're not doing TDD Agreed. > [tests that never fail can be useful] as a communication tool demonstrating that the system does in fact work the way it does. Agreed as well. That's what makes the section I wrote on testing sting for me. I read about all the benefits and see how it can be helpful as a form of documentation. I see all the upsides but at the end of the day, we still need to calculate if automated tests are really worth it. The maintenance burden on them can be huge. A couple line change and confirming it works with a manual test can take a couple of minutes, but fixing an integration test that is now failing can take the entire day or worse. It's tough, but my main goal was to challenge the dogma of "always write automated tests."
    - justin_oaks 4y ago
      > It's tough, but my main goal was to challenge the dogma of "always write automated tests." It's perfectly fine to challenge dogma and ask people figure out whether the automated tests make sense for their use case. I think the push for automated tests came when there were so many others who grossly underestimate the benefits of automated tests and dismiss them off-hand. It's hard to see the benefits of automated testing. It's easy to see the time spent writing the tests while it's hard to see the time saved by catching bugs before they happen. It's also easier to refactor if you have tests to confirm that your refactoring didn't break something.
    - dllthomas 4y ago
      Right, just because it has value doesn't mean it has enough value. My objection to the parent (not the article) was that it wound up dismissing certain kinds of value.
  - codeulike 4y ago
    the core of TDD was to iterate by making a failing test, then making it pass Yes and when I first heard that 15 years ago (or whenever it was) it sounded like genius and I did it on a few projects. Its a useful crutch maybe if you're doing something very unfamiliar But as a dogma that must always be followed? And having a test for every single 'specification' and edge case? No - screw that, you end up faffing around for ages setting up mocks and testing stuff that doesnt deserve it. And you end up with a massive library of tests that get on your nerves. Also everything got completely bent out of shape for a few years because everything had to be injectable and testable. https://dhh.dk/2014/tdd-is-dead-long-live-testing.html https://dhh.dk/2014/tdd-is-dead-long-live-testing.html http://web.archive.org/web/20180215225218/https://iansommerville.com/systems-software-and-technology/giving-up-on-test-first-development/ http://web.archive.org/web/20180215225218/https://iansommerv... http://blog.cleancoder.com/uncle-bob/2016/03/19/GivingUpOnTDD.html http://blog.cleancoder.com/uncle-bob/2016/03/19/GivingUpOnTD... https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste...
- kazinator 4y ago
  Just like if you wear a seatbelt all your life, but never have an accident, what a waste of effort. Not to mention $$$ spent on car insurance. Programmers should peer into their crystal balls and predict which code is going to need to be maintained in ways that will break it and only write regression tests for that. For all the rest, just poke at it manually in your REPL and debugger and call it tested. You heard it here.
  - codeulike 4y ago
    Yes, I'll write a test for something that I know from experience is likely to give me trouble later on with regression problems Writing a test when you find a bug and then fixing the bug is also quite handy sometimes And I'll write tests upfront when there's well defined outcomes and I know the code is going to be tricky But I'm not going to write tests for dumb stuff that I know wont break
- zcmack 4y ago
  i think TDD is a great method, but not dogma. likewise with code coverage. being prescriptive about one or the other just leads to angry people who get fed up with the principle and rebel.
- taeric 4y ago
  A problem with calling it wasted time, is that implies you could have recaptured that time in some other, more productive, way. And, I'd wager odds are high that you could not have.
  - codeulike 4y ago
    Why not?
    - taeric 4y ago
      There ultimately is no "reason." Time just isn't completely fungible between useful and wasteful. As much as we wish it could be.
      - codeulike 4y ago
        So there's no difference between doing things that have a point and doing things that dont have a point?
        taeric 4y ago
        Post fact, there can be. But we are terrible at knowing one from the other ahead of time. More, we don't always have "meaningful" things to do at any given moment. This is why we don't all work to type as fast as court reporters. It would not gain us much if any meaningful time. More, you can give meaning to things. Puzzles and such.
        codeulike 4y ago
        I know what you mean But if I was managing a bunch of programmers (thankfully, I dont have to do that) I would be thinking about their productivity and which activities contributed to productivity. TDD seems like a bell curve, having some can really help with productivity and regression and bugs. But too much and it just starts to be counterproductive.
kazinator 4y ago
The "convention over configuration" can be regarded as self-referential. These guidelines are often good default choices in situation where you don't have a strong reason to go either way. You wouldn't write a program which is configurable between those choices (e.g. exhibits more code repetition or less based on a run-time setting): so you go with a good default convention. If you can avoid repeating yourself, that's usually good; machines should do the repetitive work rather than people. If you know that two or more repetitions of something are only initially that way and soon going to diverge, then might as well fork those copies now. Or maybe do allow yourself to repeat yourself, but via macro. Some compiler optimizations violate DRY by design: function inlining, and loop unrolling. It's invisible to humans who aren't disassembling the output, or measuring code size changes.
- __derek__ 4y ago
  Right. They're razors, not commandments. Given a choice between two reasonable alternatives, it might be wise to pick the one that's {unrepetitive|simple|explicit|extensible|testable}.
- pjdesno 4y ago
  I'm not sure the opposite of "configuration" should be called "convention" - the worst abuses I've seen of punting to user configuration have been ones where the best solution was only determinable at runtime. (e.g. compare user-configured fixed window sizes in the doomed ISO protocol stack with dynamic window control in TCP) Typically the user doesn't know best - they copied a config that worked for someone else, years ago on a different machine and workload, and don't know what any of the parameters actually mean. In the worst case (sendmail?) you have O(0) people who actually know how to use the configuration language, and 10 competing higher-level config generators.
Veuxdo 4y ago
> We are a hammer and every problem is a nail that can be solved by the act of thinking. We fall into the trap of overthinking every scenario because we are paid to poke holes in systems and then patch them before it gets to the user. I was thinking about this yesterday, but more in terms of how small teams (or solo devs like in my case) can out-perform larger teams just by not even considering every edge-case. I've discovered these overlooked cases later and been kind of embarrassed, but also happy that I didn't catch them right away, since fixing them wouldn't have been worth it, and the urge to fix them would have been high.
- qudat 4y ago
  Exactly! I try to be intentional about what edge-cases I care about and maybe more importantly the ones I don't care about -- for now. I also try to be intentional about time-boxing how long I spend thinking about edge-cases. Again I think context is important here. I think this overall mentality is the contrarian in me resisting the status quo.
- taeric 4y ago
  Hubris is an amazingly effective tool that more managers and senior engineers could leverage from junior members. There are obvious problems and pitfalls that one needs to look out for. But all too often companies and teams grow to a size where their main product no longer has a place for people to try otherwise bad ideas.
- planede 4y ago
  It's easy to fall into the trap of introducing even more edge cases by handling some other edge cases. The correct way to handle edge cases is to not have them in the design. Easier said than done. And sometimes the edge cases are in the spec. Sometimes they are justified, but often they are not. These should be eliminated before the spec is finalized.
- esailija 4y ago
  Yeah I admit I have wasted too much time solving edge cases that either never happen or happen but the effect is not so big of a deal that it was worth to handle it.
xyzelement 4y ago
// Tests waste a lot of time Back when I was a developer, I had a manager who was super religious about unit tests (this seemed incongruous to me, I was a senior lead at the time and it was my vibe that such tests weren't what the project really needed the most at the time) The basic "problem" I have with tests is that - at least maybe in my experience - things tend to go wrong in cases that you haven't thought about so your tests don't cover them anyway. Like, what's the point of asserting that add(2,2) will yield 4 if you've already been thinking about that. And then, if you've already thought more complicated cases (eg negative numbers, cases that overflow) then your code already very likely handles them. And if you hadn't thought about them, your tests won't cover them either. I get it, it's cool to be able to run a test suite especially if I refactored something and be like "yup, add(2,2) is still 4!) but not sure how much actual value vs cost/annoyance I've seen from it.
- pixl97 4y ago
  I'm not really a developer here, more of the wedge between the customer and developer when things go wrong, but don't these tests cover the continuous updating of the application as specifications change? Especially as multiple people are working on the code and may have different ideas of what's going on. May not catch everything, but can trigger a warning in many cases.
  - xyzelement 4y ago
    Ah - I can see those kinds of tests as useful. I guess they are something like "integration tests" or something like that which tests the user-visible outputs of the system. My comment was about very low-level unit testing, where it's literally every function of code must be covered, isolated from other parts of code.
  - crazygringo 4y ago
    Exactly this. Also in codebases where the authors seemed opposed to writing good comments, tests are extremely useful in documenting expected behavior in edge cases, how an optional argument is supposed to be dealt with when missing, etc. Not to mention that people are often a lot less rigorous with "manual testing" than they should be. They don't think of scenarios, they skip scenarios, because nobody's checking on them. While when you write tests, you're creating a permanent record where you can be held accountable if you miss something.
jmull 4y ago
I don't think conventional wisdom is an anti-pattern, exactly. Better to think of it this way: It's a starting point, not an ending point. E.g., for DRY, if you notice yourself copying and pasting the same code around you might spend a few seconds thinking about whether there's some commonality worth capturing or abstraction worth building right now or not. The answer could be yes or no (for a variety of reasons), but it costs very little to reflect on it. (I'm not a fan of DRY though, as it's generally presented. Whether something is a good abstraction point or not doesn't really directly depend on how many times you're writing that code. WET, as presented here is even worse in this regard. If you're copying code around there's probably some useful abstraction to make, but simply barfing out a utility function every time probably leads to a lot of bad abstractions. Code is expensive -- to write, test and maintain, so it's good to minimize it. But abstractions are expensive too, since they directly affect the code written in terms of them. Bad abstractions don't end up making anything better, and probably make them worse since they will force you into a form of spaghetti code you might otherwise have avoided.)
- bsder 4y ago
  > E.g., for DRY, if you notice yourself copying and pasting the same code around you might spend a few seconds thinking about whether there's some commonality worth capturing or abstraction worth building right now or not. DRY doesn't stop the kind of people who blindly copypasta, anyway. To me, DRY is more about avoiding desync. I should not define a constant in two different places--that will desync into a bug. I should not define a data structure in two different places--that will desync into a bug. A function, on the other hand, might actually want to desync after copypasta.
  - aleksiy123 4y ago
    Pretty sure this is what DRY is supposed to be. Should be renamed really. If something needs to stay in sync it should be defined once. Things that looks sort of similar, don't necessarily need to be defined only once because its okay for them to diverge or desync and they probably eventually will.
    - 4y ago
Jtsummers 4y ago
Manual testing is for chumps (the US government is a good source of chumps). It's a great way to stretch a project out and balloon the budget because you need a separate team (maybe even multiple teams) to do in a few weeks or months what could have been done in a few hours or days by a couple computers. Why anyone would prefer manual testing for anything is beyond me, it's a great source of errors because it becomes your big time crunch in large projects. You end up cutting out tests because "that one never fails" (until you cut it) and rushing through the tests handwaving issues with "I'm sure we just fat-fingered it". Automated testing takes less time, fewer people, less hardware (normally, but not always), and can be kicked off by anyone in the project team (if you're sane about your project management) at any time. So you can run your suite today and know (worst case on huge projects) by Friday or next week that it's actually passing or failing. If you stick with manual testing, you won't know about failures for weeks, months, and sometimes years. Genius if you're going for government money, moronic if you're spending your own.
- logicalmonster 4y ago
  > Why anyone would prefer manual testing for anything is beyond me, Formal software testing is great, particularly if you're working on a hard problem with clear goals like building some hypothetical API to do some financial calculations or transactions where an error might cost tens of thousands of dollars or more. But there's some cases where literal manual testing is the best business decision, even if it's not the best software engineering decision. To give one example: have you ever worked in an agency environment? What do clients do when you hand them off any scale of project? Some people within their organization will then go over every inch of the software and see if it works as they expected. They will literally never ask once what your test coverage is. But they will notice if some test data they input themselves doesn't give the right answer, or if some minor UI element they never really thought through doesn't seem to work the way they like. Writing tests in this environment is 99% intellectual masturbation.
  - kazinator 4y ago
    > But they will notice if some test data they input themselves doesn't give the right answer, or if some minor UI element they never really thought through doesn't seem to work the way they like. Those people are going to not stop at one; so you will be foolish if you don't write a regression test for the behavior they wanted. When they find the next thing they think is broken, and you try to fix it, you could regress your earlier fix for the first item. In reacting to the user reports, you may be breaking other things that are not tested, but at least you pin down the behavior that the users are reporting. UI element not working right in some way could be a genuine "pass" for testing. I mean, writing some monkey test that feeds events into a UI to check that it's in exactly some expected state could be a big waste of time. Probably if you were writing a reusable widget framework, you might want that sort of testing, because you could make an inadvertant change which makes every instance of some widget behave differently in downstream applications. If you have some complex behavior in your UI that doesn't come from the underlying widgets, and has to be right in certain ways, then that could be worth testing.
poszlem 4y ago
You know what's not given enough credit? The fact that those so-called "patterns" are really just trendy ideas pretending to be "knowledge". It's like fashion - it goes in and out of style. Now we have all these blog posts saying "DRY considered harmful", but back in the day, everyone was all about "DRY considered good". It's just a trendy thing, man. And all this backlash against it? Yup, you guessed it - it's just another trend (because it's not cool to follow the rules anymore). To be real, most of those ideas probably have some truth to them, but like always, it's about weighing the pros and cons. It's all about those tradeoffs.
- esailija 4y ago
  There is a lot of truth to DRY but it has been distorted by becoming completely obsessed with it as if it was some end to reach, rather than any other property with its own tradeoffs. And its not just developers, but even some companies are also obsessed with removing "waste" by sacrificing everything just to minimize any kind of seemingly duplicated effort without ever considering if it's actually cheaper to do so.. When DRY is shown as high ideal somewhere, how often do they also mention the tradeoffs like tight coupling and premature abstraction?
EGreg 4y ago
I don’t write automated tests and none of my developers do, except the guy building smart contracts. And yet I am not sure I am doing the right thing. We only did it because we couldn’t afford the extra time and money to do it. If a test never fails, is it a good test? If I have to change a test everytime I change the implementation, is it a good test? Writing good tests is really difficult. Yes — just like good interface is not noticed, or how good security which prevents issues isn’t noticed when things go well.
- justin_oaks 4y ago
  I had a hard time writing tests because nobody else around me did it. Nobody taught me how to implement good tests or implement testable code. I had to figure it out by myself through trial and error, and through reading articles and watching videos. Ultimately, I found writing tests were immensely helpful for the work I was doing. I was writing the API backend for desktop/mobile apps to call. When one of the guys working on the client apps would claim my stuff wasn't working right, I demanded an example HTTP request that I could use to reproduce the problem. Most of the time they never got back to me. Often the problem was with their (untested) code and they wanted to throw the problem over the wall for someone else to investigate. I had the confidence that my code was behaving well because of the tests. So when someone else came over and said "Your stuff is broken" I had the confidence to say "Prove it". With that said, I recognize that there are cases that are lower stakes that may not require such thorough testing. Right now I have some APIs I maintain that I don't write tests for. These APIs are of the type where if there's a bug then the user of the API says "Hey, it's broken" and I say "Ok, I'll fix it in a couple days" and he says "Sure, let me know when it's done". This is quite different than the API that gets called hundreds of times a second and customers are screaming "I want my data now!"
kposehn 4y ago
Conventional Wisdom is just Convenient Assumption in experience. We need to regularly check our assumptions, lest we find ourselves operating at right angles to reality.
kunalgupta 4y ago
Luv manual tests. With everything that ships I bullet point a list of tests that theoretically would be written. Then I test the most important ones by hand. I’ve never had to retest yet, but theoretically if something was wrong i can start debugging by poking at these tests. If something broke at all i could even automate that specific test
acmegeek 4y ago
When it comes to "conventional" wisdom, I think there's simply a confusion between methods and principles. Am sure many here have heard the quote by Harrington Emerson[0] that roughly goes: "Methods are many, but principles are few. Methods always change. Principles never do." So I think that many things, like DRY and KISS, are really simply principles that were distilled after many attempted much and learned a lot. Things go sideways quickly when these are mistaken for methods. Whenever some rule or method doesn't really seem to fit, I wonder if this is really more of a principle to help guide me to the most contextually appropriate method. So, don't necessarily throw out DRY and similar, just treat them as guiding principles that will absolutely make sense in the right situations. The above quote is the most commonly known form, and is often misattributed to Ralph Waldo Emerson. Here is the full original quote: [1] "As to methods there may be a million and then some, but principles are few. The man who grasps principles can successfully select his own methods. The man who tries methods, ignoring principles, is sure to have trouble." [0] https://en.wikiquote.org/wiki/Harrington_Emerson https://en.wikiquote.org/wiki/Harrington_Emerson [1] https://www.goodreads.com/quotes/346365-as-to-methods-there-may-be-a-million-and-then https://www.goodreads.com/quotes/346365-as-to-methods-there-...
alexpetralia 4y ago
You should basically follow conventional wisdom if you are doing conventional things, and follow unconventional wisdom if you are doing unconventional things.
tonymet 4y ago
People are confusing wisdom and rules. It takes wisdom and experience to understand "DRY" , but that doesn't mean people should DRY in every possible case. Following a rule blindly isn't wise. The Wisdom is in the understanding of why the rule matters along with when & how it should be applied.
malcolmstill 4y ago
> If a test never fails, is it a good test? If I have to change a test everytime I change the implementation, is it a good test? Writing good tests is really difficult. If you don't like writing / maintaining tests or don't have the time, let the computer write them for you! [0][1] [0] https://insta.rs/ https://insta.rs/ [1] https://vitest.dev/guide/snapshot.html https://vitest.dev/guide/snapshot.html
GMoromisato 4y ago
You cannot reduce software engineering to an algorithm[1], because if you could, you could (duh) write a program to automate it and not have to worry about it. That's why the job is so hard: you have to think about the specific case and decide the best course of action. Sometimes (maybe often), DRY is the right solution; sometimes not. Same for all the other "rules" of software engineering. I think the OP is just reinforcing this: don't go on autopilot--think! [1] Obligatory disclaimer: Eventually AGI will reduce software engineering to an algorithm, but not today.
0xbadcafebee 4y ago
Ahhhhh. This is what I come to HN for. Pseudointellectual contrarian thinkpieces by bored silicon valley people and the massive upvotes by other pseudointellectual contrarian bored silicon valley people debating the details of a ridiculous premise.
muyuu 4y ago
assuming conventional wisdom to be an anti-pattern is an exceedingly common anti-pattern
DrewADesign 4y ago
When people use meaningful guiding principles dogmatically, the principles aren't the problem. As a chef, I often adopted the "measure twice, cut once" principle from carpentry. Saved big batches of sauce or whatever several times during prep. Working the line on Saturday night, however, doing that would cost someone their job within half an hour. Don't replace problem-solving with rules and always use the right tool for the job. If you realize you aren't, it's usually a lot easier to switch the tool than modify the job.

53 points by qudat 4y ago

I've written so many automated tests, but how many have actually caught something important? A couple dozen? Yeah I'm glad TDD isn't the dominant religion any more. If you write a test and it never fails throughout the lifetime of the project, thats wasted time.

moron4hire 4y ago
The last time I found TDD to be useful was when I was implementing some code for processing of data feeds according to a pretty well-defined specification. The spec translated to tests quite easily and then the tests very readily pointed out where the data did not match the spec 100%. Luckily, we were in a position to modify the data-generating side of things, too, to make sure everything was in-spec. But most of the time, I'm dealing with user interfaces, service integrations, and data munging. I have "test benches" instead of unit tests for these. They're like "minimal reproductions", fully stand-alone programs that exercise the code in the "expected" way. The big issue is that the "expected way" could change at practically any minute, depending on business need. There's no "spec" other than, "are people's expectations for a usable, comfortable system being met?" We want them to be changeable at the drop of a hat. And those test benches help make those changes possible, by not having to work in the full UI to change the behavior of one control, one service, etc.
dllthomas 4y ago
I want to push back on this in two ways. First, it's my understanding (as someone who usually doesn't TDD and hasn't read/thought that deeply about it in a while) that the core of TDD (vs other testing doctrines) was to iterate by making a failing test, then making it pass. If that's correct, then if you're writing tests that never fail throughout the lifetime of the project then you're not doing TDD, though culture does weird things to practices and definitions and I'm sure there's places that called any damned thing TDD when it had more hype. Second, I think a test (or static check) that never fails can be useful if it lets the programmer pay less attention to making sure the program is correct in that particular way (freeing up bandwidth for other important considerations), and as a communication tool demonstrating that the system does in fact work the way it does. Whether those benefits are worth it depends on a bunch of features of the particular context, including how long it takes to write/maintain the test (although if it never fails then at least you probably don't need to update it) and how long it takes to run the test.
- qudat 4y ago
  > If that's correct, then if you're writing tests that never fail throughout the lifetime of the project then you're not doing TDD Agreed. > [tests that never fail can be useful] as a communication tool demonstrating that the system does in fact work the way it does. Agreed as well. That's what makes the section I wrote on testing sting for me. I read about all the benefits and see how it can be helpful as a form of documentation. I see all the upsides but at the end of the day, we still need to calculate if automated tests are really worth it. The maintenance burden on them can be huge. A couple line change and confirming it works with a manual test can take a couple of minutes, but fixing an integration test that is now failing can take the entire day or worse. It's tough, but my main goal was to challenge the dogma of "always write automated tests."
  - justin_oaks 4y ago
    > It's tough, but my main goal was to challenge the dogma of "always write automated tests." It's perfectly fine to challenge dogma and ask people figure out whether the automated tests make sense for their use case. I think the push for automated tests came when there were so many others who grossly underestimate the benefits of automated tests and dismiss them off-hand. It's hard to see the benefits of automated testing. It's easy to see the time spent writing the tests while it's hard to see the time saved by catching bugs before they happen. It's also easier to refactor if you have tests to confirm that your refactoring didn't break something.
  - dllthomas 4y ago
    Right, just because it has value doesn't mean it has enough value. My objection to the parent (not the article) was that it wound up dismissing certain kinds of value.
- codeulike 4y ago
  the core of TDD was to iterate by making a failing test, then making it pass Yes and when I first heard that 15 years ago (or whenever it was) it sounded like genius and I did it on a few projects. Its a useful crutch maybe if you're doing something very unfamiliar But as a dogma that must always be followed? And having a test for every single 'specification' and edge case? No - screw that, you end up faffing around for ages setting up mocks and testing stuff that doesnt deserve it. And you end up with a massive library of tests that get on your nerves. Also everything got completely bent out of shape for a few years because everything had to be injectable and testable. https://dhh.dk/2014/tdd-is-dead-long-live-testing.html https://dhh.dk/2014/tdd-is-dead-long-live-testing.html http://web.archive.org/web/20180215225218/https://iansommerville.com/systems-software-and-technology/giving-up-on-test-first-development/ http://web.archive.org/web/20180215225218/https://iansommerv... http://blog.cleancoder.com/uncle-bob/2016/03/19/GivingUpOnTDD.html http://blog.cleancoder.com/uncle-bob/2016/03/19/GivingUpOnTD... https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste...
kazinator 4y ago
Just like if you wear a seatbelt all your life, but never have an accident, what a waste of effort. Not to mention $$$ spent on car insurance. Programmers should peer into their crystal balls and predict which code is going to need to be maintained in ways that will break it and only write regression tests for that. For all the rest, just poke at it manually in your REPL and debugger and call it tested. You heard it here.
- codeulike 4y ago
  Yes, I'll write a test for something that I know from experience is likely to give me trouble later on with regression problems Writing a test when you find a bug and then fixing the bug is also quite handy sometimes And I'll write tests upfront when there's well defined outcomes and I know the code is going to be tricky But I'm not going to write tests for dumb stuff that I know wont break
zcmack 4y ago
i think TDD is a great method, but not dogma. likewise with code coverage. being prescriptive about one or the other just leads to angry people who get fed up with the principle and rebel.
taeric 4y ago
A problem with calling it wasted time, is that implies you could have recaptured that time in some other, more productive, way. And, I'd wager odds are high that you could not have.
- codeulike 4y ago
  Why not?
  - taeric 4y ago
    There ultimately is no "reason." Time just isn't completely fungible between useful and wasteful. As much as we wish it could be.
    - codeulike 4y ago
      So there's no difference between doing things that have a point and doing things that dont have a point?
      - taeric 4y ago
        Post fact, there can be. But we are terrible at knowing one from the other ahead of time. More, we don't always have "meaningful" things to do at any given moment. This is why we don't all work to type as fast as court reporters. It would not gain us much if any meaningful time. More, you can give meaning to things. Puzzles and such.
        codeulike 4y ago
        I know what you mean But if I was managing a bunch of programmers (thankfully, I dont have to do that) I would be thinking about their productivity and which activities contributed to productivity. TDD seems like a bell curve, having some can really help with productivity and regression and bugs. But too much and it just starts to be counterproductive.

kazinator 4y ago

The "convention over configuration" can be regarded as self-referential. These guidelines are often good default choices in situation where you don't have a strong reason to go either way. You wouldn't write a program which is configurable between those choices (e.g. exhibits more code repetition or less based on a run-time setting): so you go with a good default convention. If you can avoid repeating yourself, that's usually good; machines should do the repetitive work rather than people. If you know that two or more repetitions of something are only initially that way and soon going to diverge, then might as well fork those copies now. Or maybe do allow yourself to repeat yourself, but via macro. Some compiler optimizations violate DRY by design: function inlining, and loop unrolling. It's invisible to humans who aren't disassembling the output, or measuring code size changes.

__derek__ 4y ago
Right. They're razors, not commandments. Given a choice between two reasonable alternatives, it might be wise to pick the one that's {unrepetitive|simple|explicit|extensible|testable}.
pjdesno 4y ago
I'm not sure the opposite of "configuration" should be called "convention" - the worst abuses I've seen of punting to user configuration have been ones where the best solution was only determinable at runtime. (e.g. compare user-configured fixed window sizes in the doomed ISO protocol stack with dynamic window control in TCP) Typically the user doesn't know best - they copied a config that worked for someone else, years ago on a different machine and workload, and don't know what any of the parameters actually mean. In the worst case (sendmail?) you have O(0) people who actually know how to use the configuration language, and 10 competing higher-level config generators.

Veuxdo 4y ago

> We are a hammer and every problem is a nail that can be solved by the act of thinking. We fall into the trap of overthinking every scenario because we are paid to poke holes in systems and then patch them before it gets to the user. I was thinking about this yesterday, but more in terms of how small teams (or solo devs like in my case) can out-perform larger teams just by not even considering every edge-case. I've discovered these overlooked cases later and been kind of embarrassed, but also happy that I didn't catch them right away, since fixing them wouldn't have been worth it, and the urge to fix them would have been high.

qudat 4y ago
Exactly! I try to be intentional about what edge-cases I care about and maybe more importantly the ones I don't care about -- for now. I also try to be intentional about time-boxing how long I spend thinking about edge-cases. Again I think context is important here. I think this overall mentality is the contrarian in me resisting the status quo.
taeric 4y ago
Hubris is an amazingly effective tool that more managers and senior engineers could leverage from junior members. There are obvious problems and pitfalls that one needs to look out for. But all too often companies and teams grow to a size where their main product no longer has a place for people to try otherwise bad ideas.
planede 4y ago
It's easy to fall into the trap of introducing even more edge cases by handling some other edge cases. The correct way to handle edge cases is to not have them in the design. Easier said than done. And sometimes the edge cases are in the spec. Sometimes they are justified, but often they are not. These should be eliminated before the spec is finalized.
esailija 4y ago
Yeah I admit I have wasted too much time solving edge cases that either never happen or happen but the effect is not so big of a deal that it was worth to handle it.

xyzelement 4y ago

// Tests waste a lot of time Back when I was a developer, I had a manager who was super religious about unit tests (this seemed incongruous to me, I was a senior lead at the time and it was my vibe that such tests weren't what the project really needed the most at the time) The basic "problem" I have with tests is that - at least maybe in my experience - things tend to go wrong in cases that you haven't thought about so your tests don't cover them anyway. Like, what's the point of asserting that add(2,2) will yield 4 if you've already been thinking about that. And then, if you've already thought more complicated cases (eg negative numbers, cases that overflow) then your code already very likely handles them. And if you hadn't thought about them, your tests won't cover them either. I get it, it's cool to be able to run a test suite especially if I refactored something and be like "yup, add(2,2) is still 4!) but not sure how much actual value vs cost/annoyance I've seen from it.

pixl97 4y ago
I'm not really a developer here, more of the wedge between the customer and developer when things go wrong, but don't these tests cover the continuous updating of the application as specifications change? Especially as multiple people are working on the code and may have different ideas of what's going on. May not catch everything, but can trigger a warning in many cases.
- xyzelement 4y ago
  Ah - I can see those kinds of tests as useful. I guess they are something like "integration tests" or something like that which tests the user-visible outputs of the system. My comment was about very low-level unit testing, where it's literally every function of code must be covered, isolated from other parts of code.
- crazygringo 4y ago
  Exactly this. Also in codebases where the authors seemed opposed to writing good comments, tests are extremely useful in documenting expected behavior in edge cases, how an optional argument is supposed to be dealt with when missing, etc. Not to mention that people are often a lot less rigorous with "manual testing" than they should be. They don't think of scenarios, they skip scenarios, because nobody's checking on them. While when you write tests, you're creating a permanent record where you can be held accountable if you miss something.

jmull 4y ago

I don't think conventional wisdom is an anti-pattern, exactly. Better to think of it this way: It's a starting point, not an ending point. E.g., for DRY, if you notice yourself copying and pasting the same code around you might spend a few seconds thinking about whether there's some commonality worth capturing or abstraction worth building right now or not. The answer could be yes or no (for a variety of reasons), but it costs very little to reflect on it. (I'm not a fan of DRY though, as it's generally presented. Whether something is a good abstraction point or not doesn't really directly depend on how many times you're writing that code. WET, as presented here is even worse in this regard. If you're copying code around there's probably some useful abstraction to make, but simply barfing out a utility function every time probably leads to a lot of bad abstractions. Code is expensive -- to write, test and maintain, so it's good to minimize it. But abstractions are expensive too, since they directly affect the code written in terms of them. Bad abstractions don't end up making anything better, and probably make them worse since they will force you into a form of spaghetti code you might otherwise have avoided.)

bsder 4y ago
> E.g., for DRY, if you notice yourself copying and pasting the same code around you might spend a few seconds thinking about whether there's some commonality worth capturing or abstraction worth building right now or not. DRY doesn't stop the kind of people who blindly copypasta, anyway. To me, DRY is more about avoiding desync. I should not define a constant in two different places--that will desync into a bug. I should not define a data structure in two different places--that will desync into a bug. A function, on the other hand, might actually want to desync after copypasta.
- aleksiy123 4y ago
  Pretty sure this is what DRY is supposed to be. Should be renamed really. If something needs to stay in sync it should be defined once. Things that looks sort of similar, don't necessarily need to be defined only once because its okay for them to diverge or desync and they probably eventually will.
  - 4y ago

Jtsummers 4y ago

Manual testing is for chumps (the US government is a good source of chumps). It's a great way to stretch a project out and balloon the budget because you need a separate team (maybe even multiple teams) to do in a few weeks or months what could have been done in a few hours or days by a couple computers. Why anyone would prefer manual testing for anything is beyond me, it's a great source of errors because it becomes your big time crunch in large projects. You end up cutting out tests because "that one never fails" (until you cut it) and rushing through the tests handwaving issues with "I'm sure we just fat-fingered it". Automated testing takes less time, fewer people, less hardware (normally, but not always), and can be kicked off by anyone in the project team (if you're sane about your project management) at any time. So you can run your suite today and know (worst case on huge projects) by Friday or next week that it's actually passing or failing. If you stick with manual testing, you won't know about failures for weeks, months, and sometimes years. Genius if you're going for government money, moronic if you're spending your own.

logicalmonster 4y ago
> Why anyone would prefer manual testing for anything is beyond me, Formal software testing is great, particularly if you're working on a hard problem with clear goals like building some hypothetical API to do some financial calculations or transactions where an error might cost tens of thousands of dollars or more. But there's some cases where literal manual testing is the best business decision, even if it's not the best software engineering decision. To give one example: have you ever worked in an agency environment? What do clients do when you hand them off any scale of project? Some people within their organization will then go over every inch of the software and see if it works as they expected. They will literally never ask once what your test coverage is. But they will notice if some test data they input themselves doesn't give the right answer, or if some minor UI element they never really thought through doesn't seem to work the way they like. Writing tests in this environment is 99% intellectual masturbation.
- kazinator 4y ago
  > But they will notice if some test data they input themselves doesn't give the right answer, or if some minor UI element they never really thought through doesn't seem to work the way they like. Those people are going to not stop at one; so you will be foolish if you don't write a regression test for the behavior they wanted. When they find the next thing they think is broken, and you try to fix it, you could regress your earlier fix for the first item. In reacting to the user reports, you may be breaking other things that are not tested, but at least you pin down the behavior that the users are reporting. UI element not working right in some way could be a genuine "pass" for testing. I mean, writing some monkey test that feeds events into a UI to check that it's in exactly some expected state could be a big waste of time. Probably if you were writing a reusable widget framework, you might want that sort of testing, because you could make an inadvertant change which makes every instance of some widget behave differently in downstream applications. If you have some complex behavior in your UI that doesn't come from the underlying widgets, and has to be right in certain ways, then that could be worth testing.

poszlem 4y ago

You know what's not given enough credit? The fact that those so-called "patterns" are really just trendy ideas pretending to be "knowledge". It's like fashion - it goes in and out of style. Now we have all these blog posts saying "DRY considered harmful", but back in the day, everyone was all about "DRY considered good". It's just a trendy thing, man. And all this backlash against it? Yup, you guessed it - it's just another trend (because it's not cool to follow the rules anymore). To be real, most of those ideas probably have some truth to them, but like always, it's about weighing the pros and cons. It's all about those tradeoffs.

esailija 4y ago
There is a lot of truth to DRY but it has been distorted by becoming completely obsessed with it as if it was some end to reach, rather than any other property with its own tradeoffs. And its not just developers, but even some companies are also obsessed with removing "waste" by sacrificing everything just to minimize any kind of seemingly duplicated effort without ever considering if it's actually cheaper to do so.. When DRY is shown as high ideal somewhere, how often do they also mention the tradeoffs like tight coupling and premature abstraction?

EGreg 4y ago

I don’t write automated tests and none of my developers do, except the guy building smart contracts. And yet I am not sure I am doing the right thing. We only did it because we couldn’t afford the extra time and money to do it. If a test never fails, is it a good test? If I have to change a test everytime I change the implementation, is it a good test? Writing good tests is really difficult. Yes — just like good interface is not noticed, or how good security which prevents issues isn’t noticed when things go well.

justin_oaks 4y ago
I had a hard time writing tests because nobody else around me did it. Nobody taught me how to implement good tests or implement testable code. I had to figure it out by myself through trial and error, and through reading articles and watching videos. Ultimately, I found writing tests were immensely helpful for the work I was doing. I was writing the API backend for desktop/mobile apps to call. When one of the guys working on the client apps would claim my stuff wasn't working right, I demanded an example HTTP request that I could use to reproduce the problem. Most of the time they never got back to me. Often the problem was with their (untested) code and they wanted to throw the problem over the wall for someone else to investigate. I had the confidence that my code was behaving well because of the tests. So when someone else came over and said "Your stuff is broken" I had the confidence to say "Prove it". With that said, I recognize that there are cases that are lower stakes that may not require such thorough testing. Right now I have some APIs I maintain that I don't write tests for. These APIs are of the type where if there's a bug then the user of the API says "Hey, it's broken" and I say "Ok, I'll fix it in a couple days" and he says "Sure, let me know when it's done". This is quite different than the API that gets called hundreds of times a second and customers are screaming "I want my data now!"

kposehn 4y ago

Conventional Wisdom is just Convenient Assumption in experience. We need to regularly check our assumptions, lest we find ourselves operating at right angles to reality.

kunalgupta 4y ago

Luv manual tests. With everything that ships I bullet point a list of tests that theoretically would be written. Then I test the most important ones by hand. I’ve never had to retest yet, but theoretically if something was wrong i can start debugging by poking at these tests. If something broke at all i could even automate that specific test

acmegeek 4y ago

When it comes to "conventional" wisdom, I think there's simply a confusion between methods and principles. Am sure many here have heard the quote by Harrington Emerson[0] that roughly goes: "Methods are many, but principles are few. Methods always change. Principles never do." So I think that many things, like DRY and KISS, are really simply principles that were distilled after many attempted much and learned a lot. Things go sideways quickly when these are mistaken for methods. Whenever some rule or method doesn't really seem to fit, I wonder if this is really more of a principle to help guide me to the most contextually appropriate method. So, don't necessarily throw out DRY and similar, just treat them as guiding principles that will absolutely make sense in the right situations. The above quote is the most commonly known form, and is often misattributed to Ralph Waldo Emerson. Here is the full original quote: [1] "As to methods there may be a million and then some, but principles are few. The man who grasps principles can successfully select his own methods. The man who tries methods, ignoring principles, is sure to have trouble." [0] https://en.wikiquote.org/wiki/Harrington_Emerson https://en.wikiquote.org/wiki/Harrington_Emerson [1] https://www.goodreads.com/quotes/346365-as-to-methods-there-may-be-a-million-and-then https://www.goodreads.com/quotes/346365-as-to-methods-there-...

alexpetralia 4y ago

You should basically follow conventional wisdom if you are doing conventional things, and follow unconventional wisdom if you are doing unconventional things.

tonymet 4y ago

People are confusing wisdom and rules. It takes wisdom and experience to understand "DRY" , but that doesn't mean people should DRY in every possible case. Following a rule blindly isn't wise. The Wisdom is in the understanding of why the rule matters along with when & how it should be applied.

malcolmstill 4y ago

> If a test never fails, is it a good test? If I have to change a test everytime I change the implementation, is it a good test? Writing good tests is really difficult. If you don't like writing / maintaining tests or don't have the time, let the computer write them for you! [0][1] [0] https://insta.rs/ https://insta.rs/ [1] https://vitest.dev/guide/snapshot.html https://vitest.dev/guide/snapshot.html

GMoromisato 4y ago

You cannot reduce software engineering to an algorithm[1], because if you could, you could (duh) write a program to automate it and not have to worry about it. That's why the job is so hard: you have to think about the specific case and decide the best course of action. Sometimes (maybe often), DRY is the right solution; sometimes not. Same for all the other "rules" of software engineering. I think the OP is just reinforcing this: don't go on autopilot--think! [1] Obligatory disclaimer: Eventually AGI will reduce software engineering to an algorithm, but not today.

0xbadcafebee 4y ago

Ahhhhh. This is what I come to HN for. Pseudointellectual contrarian thinkpieces by bored silicon valley people and the massive upvotes by other pseudointellectual contrarian bored silicon valley people debating the details of a ridiculous premise.

muyuu 4y ago

assuming conventional wisdom to be an anti-pattern is an exceedingly common anti-pattern

DrewADesign 4y ago

When people use meaningful guiding principles dogmatically, the principles aren't the problem. As a chef, I often adopted the "measure twice, cut once" principle from carpentry. Saved big batches of sauce or whatever several times during prep. Working the line on Saturday night, however, doing that would cost someone their job within half an hour. Don't replace problem-solving with rules and always use the right tool for the job. If you realize you aren't, it's usually a lot easier to switch the tool than modify the job.