4 ms·

> I suspect best practices for "vibe coding" will end up like test-driven development: a proven method for writing better software that many engineers still cho

> I suspect best practices for "vibe coding" will end up like test-driven development: a proven method for writing better software that many engineers still choose to skip. I’d like to see the proof for TDD; last I heard it slowed development with only minor reliability improvements.

jcmontx 1y ago
I see TDD exactly as the best practice for vibe coding! Context is stored as test coverage, to make sure they don't break things when they hallucinate. YMMV though.
- recursive 1y ago
  Two bad tastes that taste bad together.
RandallBrown 1y ago
Who did you hear the it slowed development from? My personal experience (and I think the experience of many who do it full time) is that it makes things faster.
- chickenzzzzu 1y ago
  I wrote a full fledged 3D exporter in 6 days without a single test case. Used by thousands of people every day. How did I test and debug? Run my code and printf.
  - gotoeleven 1y ago
    [flagged]
  - 0x696C6961 1y ago
    Try working on a project with lots of contributors.
    - chickenzzzzu 1y ago
      The whole point is that it shouldn't happen. You should consume the thing I produce.
rootnod3 1y ago
So, there you have it. TDD is good if applied correctly, and only if you apply it 100% correct. And so it seems for LLM usage. If it doesn't work for you, then you are obviously doing it wrong according to many folks here. TDD is nice to catch refactoring mistakes, LLMs are nice to maybe do some initial refactoring on a small enough code base. And it doesn't mean that one precludes the other. But I haven't seen TDD put engineers out of work and neither should LLMs. Trust either model fully and you are in for a world of hurt.
- MoreQARespect 1y ago
  I would usually measure "TDD correctness" in terms of how closely the test matches a user story vs how closely it mirrors code implementation. The former is desirable, not common. The latter is common, not desirable.
  - rootnod3 1y ago
    And with LLMs you'd have to measure how close the prompt is to a user story then. And how close its output is to the user story.
seadan83 1y ago
The proof for TDD is usually looking at bug detection rates. Similar for code review. OTOH, the "design damage" of TDD is something overlooked by those metrics. What it boils down to: - TDD in the hands of a junior is very good. Drastically reduces bugs, and teaches the junior how to write code that can be tested and is not just a big long single method of spaghetti with every data structure represented as another dimension on some array. - TDD in the hands of a midlevel can be a mixed bag. They've learned how to do TDD well, but have not learned when and why TDD can go bad. This creates design damage, where everything is shoe-horned into TDD and the goal of 90% line coverage is a real consideration. This is maximum correctness but also potentially maximum design damage. - TDD in the hands of a senior is a power tool. The "right" tests are written for the right reasons with the right level of coupling and the tests overall are useful. Every really complicated algorithm I've had to write, TDD was a life saver for getting it landed. Feels a lot like asking someone if they prefer X or Y and they say "X" is the industry best practice. My response universally is now an eye brow raise "oh, is it? For which segments of the industry? Why? How do we know it's actually a best practice? Okay, given our context, why would it be a best practice for US". Juniors don't know the best practices, mid-levels apply them everywhere, seniors evaluate and consider when best practices are not best practices. TDD slows development when tests are written in a blind way with an eye on code coverage and not correctness and design. TDD speeds up development in being a good way to catch errors and is one of the best ways to ensure correctness.
- dogleash 1y ago
  Your comment doesn’t distinguish between having a robust automated test suite and doing TDD. I’ll take your comment as testing is good and constraining your workflow to TDD is worthless.
  - AstroBen 1y ago
    TDD is really commonly misunderstood to be a testing strategy that helps reliability- it's not. It's supposed to guide your software design
    - 1y ago
bsoles 1y ago
TDD doesn't work for the same reasons why Todo lists don't work. I don't have a good explanation why that is so, but I know from countless examples (mine and other people's) that they don't work. In general, doing things work, planning to do things don't.
- jayd16 1y ago
  I guess you could say TDD is the waterfall of coding styles.
juancn 1y ago
I think it works only for people with a certain thought pattern, the ones that like to think everything up-front before doing anything. Most people prefer to play around and make several crappy attempts and combine them until the whole is somewhat solved, then go over and polish it a little, and maybe then add tests and fix the behavior in place. For this last group, TDD it's jarring, unnatural and requires a lot of willpower to follow. It's not bad in itself, it's just not for everyone.
- tincholio 1y ago
  > the ones that like to think everything up-front before doing anything. I don't think that's the case. If they were really thinking up-front, they'd be doing proper req analysis and design work, rather than interactively growing a ball of mud that "does the minimal thing to pass a test". To me, it seems like TDD is sold as this "foolproof" design / dev approach, which is anything but.
- _heitoo 1y ago
  TDD does not require an upfront understanding of the solution. It’s primarily about verifying what your application (or module) does, not how it does it. In other words, tests should be written from the outside in. The common struggle with TDD arises when people reduce it to writing glorified spell checkers. This usually goes hand in hand with the belief that unit tests must always check classes in isolation using excessive mocks—an approach that misses the real purpose of TDD. It has less to do with thought patterns and more with simply misunderstanding the approach entirely because of clinging to the wrong dogmas they've heard somewhere.
corytheboyd 1y ago
TDD works if know what 100% of the inputs/outputs look like, AND you know the implementation will be complex, terse, whatever. Write tests, then twiddle a bunch of regex until the tests pass. This situation rarely comes up in real software jobs, but when it does, yeah use it!
justonceokay 1y ago
I always love to compare Peter Norvig’s sudoku solver implementation [1] versus Ron Jeffries TDD approach [2]. Norvig starts with the theory building and creates a constraint solver in about 50 lines of code. Jeffries starts with TDD, assumes an implementation, has to change that implementation, therefore has to change the tests, and after a series of five blog posts kind of fizzles out on it. To me it just highlights that defining the problem is with so much more than defining the tests, as you can’t write a test for a problem you haven’t defined yet. In this way the tests are an imposition. In short to me it shows that TDD only really works if you already knew how to buold the project to begin with. [1] https://norvig.com/sudoku.html https://norvig.com/sudoku.html [2] https://ronjeffries.com/xprog/articles/oksudoku/ https://ronjeffries.com/xprog/articles/oksudoku/
- draw_down 1y ago
  [dead]

jcmontx 1y ago

I see TDD exactly as the best practice for vibe coding! Context is stored as test coverage, to make sure they don't break things when they hallucinate. YMMV though.

recursive 1y ago
Two bad tastes that taste bad together.

RandallBrown 1y ago

Who did you hear the it slowed development from? My personal experience (and I think the experience of many who do it full time) is that it makes things faster.

chickenzzzzu 1y ago
I wrote a full fledged 3D exporter in 6 days without a single test case. Used by thousands of people every day. How did I test and debug? Run my code and printf.
- gotoeleven 1y ago
  [flagged]
- 0x696C6961 1y ago
  Try working on a project with lots of contributors.
  - chickenzzzzu 1y ago
    The whole point is that it shouldn't happen. You should consume the thing I produce.

rootnod3 1y ago

So, there you have it. TDD is good if applied correctly, and only if you apply it 100% correct. And so it seems for LLM usage. If it doesn't work for you, then you are obviously doing it wrong according to many folks here. TDD is nice to catch refactoring mistakes, LLMs are nice to maybe do some initial refactoring on a small enough code base. And it doesn't mean that one precludes the other. But I haven't seen TDD put engineers out of work and neither should LLMs. Trust either model fully and you are in for a world of hurt.

MoreQARespect 1y ago
I would usually measure "TDD correctness" in terms of how closely the test matches a user story vs how closely it mirrors code implementation. The former is desirable, not common. The latter is common, not desirable.
- rootnod3 1y ago
  And with LLMs you'd have to measure how close the prompt is to a user story then. And how close its output is to the user story.

seadan83 1y ago

The proof for TDD is usually looking at bug detection rates. Similar for code review. OTOH, the "design damage" of TDD is something overlooked by those metrics. What it boils down to: - TDD in the hands of a junior is very good. Drastically reduces bugs, and teaches the junior how to write code that can be tested and is not just a big long single method of spaghetti with every data structure represented as another dimension on some array. - TDD in the hands of a midlevel can be a mixed bag. They've learned how to do TDD well, but have not learned when and why TDD can go bad. This creates design damage, where everything is shoe-horned into TDD and the goal of 90% line coverage is a real consideration. This is maximum correctness but also potentially maximum design damage. - TDD in the hands of a senior is a power tool. The "right" tests are written for the right reasons with the right level of coupling and the tests overall are useful. Every really complicated algorithm I've had to write, TDD was a life saver for getting it landed. Feels a lot like asking someone if they prefer X or Y and they say "X" is the industry best practice. My response universally is now an eye brow raise "oh, is it? For which segments of the industry? Why? How do we know it's actually a best practice? Okay, given our context, why would it be a best practice for US". Juniors don't know the best practices, mid-levels apply them everywhere, seniors evaluate and consider when best practices are not best practices. TDD slows development when tests are written in a blind way with an eye on code coverage and not correctness and design. TDD speeds up development in being a good way to catch errors and is one of the best ways to ensure correctness.

dogleash 1y ago
Your comment doesn’t distinguish between having a robust automated test suite and doing TDD. I’ll take your comment as testing is good and constraining your workflow to TDD is worthless.
- AstroBen 1y ago
  TDD is really commonly misunderstood to be a testing strategy that helps reliability- it's not. It's supposed to guide your software design
  - 1y ago

bsoles 1y ago

TDD doesn't work for the same reasons why Todo lists don't work. I don't have a good explanation why that is so, but I know from countless examples (mine and other people's) that they don't work. In general, doing things work, planning to do things don't.

jayd16 1y ago
I guess you could say TDD is the waterfall of coding styles.

juancn 1y ago

I think it works only for people with a certain thought pattern, the ones that like to think everything up-front before doing anything. Most people prefer to play around and make several crappy attempts and combine them until the whole is somewhat solved, then go over and polish it a little, and maybe then add tests and fix the behavior in place. For this last group, TDD it's jarring, unnatural and requires a lot of willpower to follow. It's not bad in itself, it's just not for everyone.

tincholio 1y ago
> the ones that like to think everything up-front before doing anything. I don't think that's the case. If they were really thinking up-front, they'd be doing proper req analysis and design work, rather than interactively growing a ball of mud that "does the minimal thing to pass a test". To me, it seems like TDD is sold as this "foolproof" design / dev approach, which is anything but.
_heitoo 1y ago
TDD does not require an upfront understanding of the solution. It’s primarily about verifying what your application (or module) does, not how it does it. In other words, tests should be written from the outside in. The common struggle with TDD arises when people reduce it to writing glorified spell checkers. This usually goes hand in hand with the belief that unit tests must always check classes in isolation using excessive mocks—an approach that misses the real purpose of TDD. It has less to do with thought patterns and more with simply misunderstanding the approach entirely because of clinging to the wrong dogmas they've heard somewhere.

corytheboyd 1y ago

TDD works if know what 100% of the inputs/outputs look like, AND you know the implementation will be complex, terse, whatever. Write tests, then twiddle a bunch of regex until the tests pass. This situation rarely comes up in real software jobs, but when it does, yeah use it!

justonceokay 1y ago

I always love to compare Peter Norvig’s sudoku solver implementation [1] versus Ron Jeffries TDD approach [2]. Norvig starts with the theory building and creates a constraint solver in about 50 lines of code. Jeffries starts with TDD, assumes an implementation, has to change that implementation, therefore has to change the tests, and after a series of five blog posts kind of fizzles out on it. To me it just highlights that defining the problem is with so much more than defining the tests, as you can’t write a test for a problem you haven’t defined yet. In this way the tests are an imposition. In short to me it shows that TDD only really works if you already knew how to buold the project to begin with. [1] https://norvig.com/sudoku.html https://norvig.com/sudoku.html [2] https://ronjeffries.com/xprog/articles/oksudoku/ https://ronjeffries.com/xprog/articles/oksudoku/

draw_down 1y ago
[dead]