4 ms·

Exactly one of the reasons I never went down with all the TDD dogma of only writing code to fix broken tests. There is a reason conference talks are always abo

• bob1029 5mo ago
  The biggest flaw I've seen with TDD is the fact that correctness does not compose upward. Every time two units come into contact, you've got an entirely new kind of unit. The tests from constituents do not cover emergent properties of the new things. You will repeat this same exercise the entire way up to the top, and the moment you come into contact with the customer (they want to change everything), the house of cards comes crumbling down and you have to start your agonizingly-slow process all over from the bottom again. The only thing that the business seems to care about is top-down UI testing. This is also convenient because you can leave it until the very end after the customer has already seen several prototypes. I do think TDD makes sense in isolated scopes (prove this specific custom parser works at the edges), but as the general policy for the entire product it's definitely not a viable practice. Much of the time if comes off as an ego trip to see just how cleverly we can mock something so that we can say we technically tested it.
  • pjmlp 5mo ago
    Exactly, the whole system thinking and large scale architecture also fails apart, when writing everything from little working tests.
  • bluGill 5mo ago
    I tell people you should be testing at the level where a change would be so hard you wouldn't do it anyway. Internal helper functions - they are tested only because the code that calls them passes. Interfaces that are used thousands of places - you better test them well because you wouldn't dare change that anyway: it would break too many others. Or to put it differently: a test is an assertion that no matter what, for all time this should never change again. Even if customer requirements change in the future they won't change in such a way as to break this test (this isn't always true, but you should believe it is true). A test is most valuable when it alerts you to a real problem when it fails. If the test fails but there isn't a real problem (either because customer requirements have changed, or it is flaky) it was needless cost to investigate it. If the test passes that gives some hope of correctness, but you can never be sure it is really correct vs a bug in the test (even if you use TDD and so the test failed when you wrote it that doesn't mean a refactoring since didn't make this an always pass test). Part of the problem is if I tell you to write sort() or your new toy language's list type you have an intuitive idea of what it should look like and probably will get them right the first time (other than bugs you want the tests so you catch). These should have tiny micro tests. These things also are really easy to use as examples of how to do TDD - which they are, but they are not representative: this type of code is generally in your standard library already and you are not writing it. Instead you are writing code that isn't well defined with lots of industry experience. It is not clear what the exact interface should be (or more likely it is clear customer requirements will change but you don't know how yet). You have no idea what the best implementation is. You don't know if this will be used in this one place, or if it will become a useful key part that many future projects depend on. You have to make guesses.
  • valvar 5mo ago
    TDD is perfect for bugs; codify a replication first, then fix it.
    • pjmlp 5mo ago
      Example for HLSL graphical glitch?
      • MoreQARespect 5mo ago
        https://hitchdev.com/hitchstory/approach/snapshot-test-driven-development-stdd/ https://hitchdev.com/hitchstory/approach/snapshot-test-drive... set up a rendering profile and preconditions that generates a minimal snippet of images/video using a predefined GPU profile. then test for either a pixel perfect reproduction of the correct behaviour or for the properties you're looking for (if it doesnt reproduce deterministically). this is one way. i also subscribe to the view that if the type system is modified to become stricter in such a way that it can fail reliably in the presence of this type of bug that this is also good enough. some people might argue that these arent "strictly" TDD by some definition but they set out a path to follow red green refactor and confer identical benefits so my view is who gives a duck? I don't have enough domain expertise to know which variant of these approaches is best but I'm enough of a TDD expert to know that what you're implying isnt possible is actually something you would would probably derive a lot of value from if you did it.
        • pjmlp 5mo ago
          Now do that interactive with feed back from design team and user testing.
          • MoreQARespect 5mo ago
            Iterate on the design til the snapshots look the way the design team wants. That's just an extended red where you get feedback from elsewhere.
  • MoreQARespect 5mo ago
    That is a flaw with unit tests written at far too low a level, not with TDD. You would have the same problem if you wrote tests like that after the code. TDD has no opinion about the level at which you wrote your test, it just assumes it's the correct one. This is the number one biggest misconception about TDD which I keep seeing repeated on hacker news. https://news.ycombinator.com/item?id=46810793 https://news.ycombinator.com/item?id=46810793 https://news.ycombinator.com/item?id=45113016 https://news.ycombinator.com/item?id=45113016
    • pjmlp 5mo ago
      TDD for UI effects?
      • MoreQARespect 5mo ago
        snapshot test driven development again. i already wrote a similar answer in response to your other comment. it follows the definition of TDD and it works really well (with some caveats) but again some people get hung up on what their impression of TDD is (e.g. unit tests checking to see if a car object has a steering wheel or whatever...) rather than what it actually is and what about it is that actually works.
        • pjmlp 5mo ago
          How does snapshot do "feels right" from designer point of view?
          • MoreQARespect 5mo ago
            Um, show the snapshot to a designer? When it feels right, lock in the snapshot ("green") and then move on to refactor. Or, probably more likely a group of snapshots.
  • saghm 5mo ago
    I feel this, especially with the crazy lengths people go to mock things sometimes. A couple years back I was having a discussion with a friend/former coworker about testing (I was griping about unnecessary mocks I had to deal with for something at a job causing unnecessary extra work), and he asked how I would approach trying to get full unit test coverage instead. I was taken aback and said that I wouldn't try to get literally everything covered by unit tests in the first place. Most of the teams I've worked on have had the approach that test coverage is good, but it isn't necessarily going to be 100% even when considering all tests; I can't even imagine trying to get 100% coverage for unit tests alone being anywhere close to worth the extra effort, let alone the contortions that the code would need to take to support it.
    • simoncion 5mo ago
      Yeah. Some TDD-obsessed companies will write tests in a way that requires you to spend a half hour understanding the web of mocks in order to update the tests to account for even a minor datastructure change. Coincidentally, your code change would cause those same tests to fail if they weren't mocked out, but they all pass until you make your changes to the mocks. This shreds the "if the tests pass, the change is probably correct" confidence that's most of the reason for having automated tests. I am not a fan of this style of test writing.
• zuzululu 5mo ago
  yup and I find it weird that people still remain so defensive of the Church of TDD even against empirical studies that show its limited benefits https://arxiv.org/abs/2602.07900 https://arxiv.org/abs/2602.07900
  • wolvesechoes 5mo ago
    Many such churches. Substitute static typing for TDD in your comment, and it will remain equally valid statement.
    • mcapodici 5mo ago
      Static typing is very useful and time saving. I can rename something, knowing my IDE can propogate the change. I can call a function and know what it expects, not just "a thing, hope they added a comment so I know what type/shape wont blow it up!". Here I am talking about the basic static typing, and maybe some generics use occasionally, but obviously people also go overboard sometimes with type features and that hinders understanding for newcomers to the codebase.
      • topaz0 5mo ago
        Appreciating that there are some benefits is differwnt from adhering to the church
      • wolvesechoes 4mo ago
        It has benefits, yes. And TDD also has some benefits. In both cases, these benefits are limited, and there are costs associated. And in both cases we have empirical evidence showing that neither is panacea to the problems church members claim they solve.
        • mcapodici 4mo ago
          I don't feel like static typing (the kind you use in Go, C#, Java etc) is a church, it is just a tool. I mean I always want to use it, because it is useful. So that might be like being in the church of say adding an electric motor to my bicycle? I can't show you a study or hard evidence saying having the electric motor is "better". I mean there are people who go nuts with very complicated types/type systems so there is that, and then you have very complicated programs, maybe that is what you mean? Using static typing all the time is just using the tools. Using TDD for everything feels a bit suboptimal to me and so needs some obsession to do that. It only becomes a church then if they keep pestering everyone else to do it.
• e12e 5mo ago
  > TDD dogma of only writing code to fix broken tests. Isn't red-green-refactor pretty ingrained in TDD? Only write code to make a failing test pass; then refactor while making sure the tests still pass? Then write a test that fails, repeat?
  • pjmlp 5mo ago
    Now do a games engine with that approach regarding shaders and the desired visuals.
    • e12e 5mo ago
      I'm not familiar with shaders and game engines, so I'm not sure what you are saying. I had a quick look at godot tests, and seems to me they cover some parts of the shaders? Anyway, I was more wondering who/how people are dogmatic about TDD, and manage to leave out one out of three core concepts from red/green/refactor ?
      • pjmlp 5mo ago
        If I cannot write shader code without broken tests, there is a bunch of yak shaving to make testing possible in first place, and that only covers a small subset of graphics pipeline features. It also takes zero consideration for the interactive nature of games/graphics development.
        • MoreQARespect 5mo ago
          yes, there is some yak shaving necessary to make writing tests possible. There is often a tension between delivering fast and high quality/bug free and what is necessary for medical software or financial calculations might not be necessary for games. The question of whether to write tests at all is not really about TDD though.
        • e12e 4mo ago
          I can't immediately think of what a useful shader test would look like (beyond perhaps, shader doesn't crash program) - if this is something worth discussing, it would probably be useful to see some real world shader code; perhaps especially two versions of the same shader as it evolve. I don't generally test css code to check that a background is now indeed set to "a more mauvey shade of pinky-russet" after a change - but I might want to. I might at least want to run a test with browser automation to check that any text is readable on the background. I could at least find an example of looking at the rendered page for text (as opposed to in the DOM); Google AI had some ideas of how to check the contrast in a screenshot - but no idea if that would actually work as written. https://medium.com/@dzianisv/vibe-engineering-testing-browser-automation-with-ocr-beyond-text-output-validation-717cef1d00d4 https://medium.com/@dzianisv/vibe-engineering-testing-browse... https://share.google/aimode/mW8ClhqGppfpovRrE https://share.google/aimode/mW8ClhqGppfpovRrE