3 ms·

> That doesn't mean that we cannot use LLMs to build that software, but it does mean that in the end every line of code must be validated to make sure there's n

> That doesn't mean that we cannot use LLMs to build that software, but it does mean that in the end every line of code must be validated to make sure there's no issues injected by the LLM tools that inherently (...) The problem with your assertion is that it fails to understand that today's software, where every single line of code was typed in by real flesh-and-bone humans, already fails to have adequate test coverages, let alone be validated. The main problem with output from LLMs is that they were trained with the code written by humans, and thus they accurately reflect the quality of the code that's found in the wild. Consequently, your line of reasoning actually criticizes LLMs for outputing the same unreliable code that people write. Counterintuitively, LLMs end up generating a better output because at least they are designed to simplify the task of automatically generating tests.

vouwfietsman 1y ago
Right but by your reasoning it would make sense to use LLMs only to augment an incomplete but rigorous testing process, or to otherwise elevate below average code. My issue is not necessarily with the quality of the code, but rather with the intention of the code, which is much more important: a good design without tests is more durable than a bad design with tests.
- motorest 1y ago
  > Right but by your reasoning it would make sense to use LLMs only to augment an incomplete but rigorous testing process, or to otherwise elevate below average code. No. It makes sense to use LLMs to generate tests. Even if their output matches the worst output the average human can write by hand, having any coverage whatsoever already raises the bar from where the average human output is. > My issue is not necessarily with the quality of the code, but rather with the intention of the code (...) That's not the LLM's responsibility. Humans specify what they want and LLMs fill in the blanks. If today's LLMs output bad results, that's a reflection of the prompts. Garbage in, garbage out.
  - vouwfietsman 1y ago
    > No. It makes sense to use LLMs to generate tests. Even if their output matches the worst output the average human can write by hand, having any coverage whatsoever already raises the bar from where the average human output is. Although this is true, it disregards the fact that prompting for tests takes time which may also be spent writing tests, and its not clear if poor quality tests are free, in the sense that further development may cause these tests to fail for the wrong reasons, causing time spent debugging. This is why I used the word "augment": these tests are clearly not the same quality as manual tests, and should be considered separately from manual tests. In other words, they may serve to elevate below average code or augment manual tests, but not more than that. Again, I'm not saying it makes no sense to do this. > That's not the LLM's responsibility. Humans specify what they want and LLMs fill in the blanks. If today's LLMs output bad results, that's a reflection of the prompts. Garbage in, garbage out. This is unlikely to be true, for a couple reasons: 1. Ambiguity makes it impossible to define "garbage", see prompt engineering. In fact, all human natural language output is garbage in the context of programming. 2. As the LLM fills in blanks, it must do so respecting the intention of the code, otherwise the intention of the code erodes, and its design is lost. 3. This would imply that LLMs have reached their peak and only improve by requiring less prompting by a user, this is simply not true as it is trivial to currently find problems an LLM cannot solve, regardless of the amount of prompting.
    - motorest 1y ago
      > Although this is true, it disregards the fact that prompting for tests takes time which may also be spent writing tests (...) No, not today at least. Some services like Copilot provide plugins that implement actions to automatically generate unit tests. This means that the unit test coverage you're describing is a right-click away. https://code.visualstudio.com/docs/copilot/copilot-smart-actions https://code.visualstudio.com/docs/copilot/copilot-smart-act... > (...).and its not clear if poor quality tests are free, in the sense that further development may cause these tests to fail for the wrong reasons, causing time spent debugging. That's not how automated tests work. If you have a green test that turns red when you touch some part of the code, this is the test working as expected, because your code change just introduced unexpected changes that violated an invariant. Also, today's LLMs are able to recreate all your unit tests from scratch. > This is unlikely to be true, for a couple reasons: 1. Ambiguity makes it impossible to define "garbage", see prompt engineering. "Ambiguity" is garbage in this context. > . 2. As the LLM fills in blanks, it must do so respecting the intention of the code, otherwise the intention of the code erodes, and its design is lost. That's the responsibility of the developer, not the LLM. Garbage in, garbage out. > . 3. This would imply that LLMs have reached their peak and only improve by requiring less prompting by a user, this is simply not true as it is trivial to currently find problems an LLM cannot solve, regardless of the amount of prompting. I don't think that point is relevant. The goal of a developer is still to meet the definition of done, not to tie their hands around their back and expect working code to just fall on their lap. Currently the main approach to vibe coding is to set the architecture, and lean on the LLM to progressively go from high level to low level details. Speaking from personal experience in vibecoding, LLMs are quite capable of delivering fully working apps with a single, detailed prompt. However, you get far more satisfactory results (i.e., the app reflects the same errors in judgement you'd make) if you just draft a skeleton and progressively fill in the blanks.
UncleEntity 1y ago
From my testing the robots seem to 'understand' the code more than just learn how do thing X in code from reading code about doing X. I've thrown research papers at them and they just 'get' what needs to be done to take the idea and implement it as a library or whatever. Or, what has become my favorite activity of late, give them some code and ask them how they would make it better -- then take that and split it up into simpler tasks because they get confused it you ask them to do too much at one time. As for debugging, they're not so good at that. Some debugging they can figure out but if they need to do something simple, like counting how far away item A is from item B, then I've found you pretty much have to do that for them. Don't get me wrong, they've found some pretty deep bugs I would have spend a bunch of time tracking down in gdb, so they aren't completely worthless but I have definitely given up on the idea that I can just tell them the problem and they get to work fixing it though. And, yeah, they're good at writing tests. I usually work on python C modules and my typical testing is playing with it in the repl but my current project is getting fully tested at the C level before I have gotten around to the python wrapper code. Overall its been pretty productive using the robots, code is being written I wouldn't have spent the time working on, unit testing is being used to make sure they don't break anything as the project progresses and the codebase is being kept pretty sound because I know enough to see when they're going off the rails as they often do.

vouwfietsman 1y ago

Right but by your reasoning it would make sense to use LLMs only to augment an incomplete but rigorous testing process, or to otherwise elevate below average code. My issue is not necessarily with the quality of the code, but rather with the intention of the code, which is much more important: a good design without tests is more durable than a bad design with tests.

motorest 1y ago
> Right but by your reasoning it would make sense to use LLMs only to augment an incomplete but rigorous testing process, or to otherwise elevate below average code. No. It makes sense to use LLMs to generate tests. Even if their output matches the worst output the average human can write by hand, having any coverage whatsoever already raises the bar from where the average human output is. > My issue is not necessarily with the quality of the code, but rather with the intention of the code (...) That's not the LLM's responsibility. Humans specify what they want and LLMs fill in the blanks. If today's LLMs output bad results, that's a reflection of the prompts. Garbage in, garbage out.
- vouwfietsman 1y ago
  > No. It makes sense to use LLMs to generate tests. Even if their output matches the worst output the average human can write by hand, having any coverage whatsoever already raises the bar from where the average human output is. Although this is true, it disregards the fact that prompting for tests takes time which may also be spent writing tests, and its not clear if poor quality tests are free, in the sense that further development may cause these tests to fail for the wrong reasons, causing time spent debugging. This is why I used the word "augment": these tests are clearly not the same quality as manual tests, and should be considered separately from manual tests. In other words, they may serve to elevate below average code or augment manual tests, but not more than that. Again, I'm not saying it makes no sense to do this. > That's not the LLM's responsibility. Humans specify what they want and LLMs fill in the blanks. If today's LLMs output bad results, that's a reflection of the prompts. Garbage in, garbage out. This is unlikely to be true, for a couple reasons: 1. Ambiguity makes it impossible to define "garbage", see prompt engineering. In fact, all human natural language output is garbage in the context of programming. 2. As the LLM fills in blanks, it must do so respecting the intention of the code, otherwise the intention of the code erodes, and its design is lost. 3. This would imply that LLMs have reached their peak and only improve by requiring less prompting by a user, this is simply not true as it is trivial to currently find problems an LLM cannot solve, regardless of the amount of prompting.
  - motorest 1y ago
    > Although this is true, it disregards the fact that prompting for tests takes time which may also be spent writing tests (...) No, not today at least. Some services like Copilot provide plugins that implement actions to automatically generate unit tests. This means that the unit test coverage you're describing is a right-click away. https://code.visualstudio.com/docs/copilot/copilot-smart-actions https://code.visualstudio.com/docs/copilot/copilot-smart-act... > (...).and its not clear if poor quality tests are free, in the sense that further development may cause these tests to fail for the wrong reasons, causing time spent debugging. That's not how automated tests work. If you have a green test that turns red when you touch some part of the code, this is the test working as expected, because your code change just introduced unexpected changes that violated an invariant. Also, today's LLMs are able to recreate all your unit tests from scratch. > This is unlikely to be true, for a couple reasons: 1. Ambiguity makes it impossible to define "garbage", see prompt engineering. "Ambiguity" is garbage in this context. > . 2. As the LLM fills in blanks, it must do so respecting the intention of the code, otherwise the intention of the code erodes, and its design is lost. That's the responsibility of the developer, not the LLM. Garbage in, garbage out. > . 3. This would imply that LLMs have reached their peak and only improve by requiring less prompting by a user, this is simply not true as it is trivial to currently find problems an LLM cannot solve, regardless of the amount of prompting. I don't think that point is relevant. The goal of a developer is still to meet the definition of done, not to tie their hands around their back and expect working code to just fall on their lap. Currently the main approach to vibe coding is to set the architecture, and lean on the LLM to progressively go from high level to low level details. Speaking from personal experience in vibecoding, LLMs are quite capable of delivering fully working apps with a single, detailed prompt. However, you get far more satisfactory results (i.e., the app reflects the same errors in judgement you'd make) if you just draft a skeleton and progressively fill in the blanks.

UncleEntity 1y ago

From my testing the robots seem to 'understand' the code more than just learn how do thing X in code from reading code about doing X. I've thrown research papers at them and they just 'get' what needs to be done to take the idea and implement it as a library or whatever. Or, what has become my favorite activity of late, give them some code and ask them how they would make it better -- then take that and split it up into simpler tasks because they get confused it you ask them to do too much at one time. As for debugging, they're not so good at that. Some debugging they can figure out but if they need to do something simple, like counting how far away item A is from item B, then I've found you pretty much have to do that for them. Don't get me wrong, they've found some pretty deep bugs I would have spend a bunch of time tracking down in gdb, so they aren't completely worthless but I have definitely given up on the idea that I can just tell them the problem and they get to work fixing it though. And, yeah, they're good at writing tests. I usually work on python C modules and my typical testing is playing with it in the repl but my current project is getting fully tested at the C level before I have gotten around to the python wrapper code. Overall its been pretty productive using the robots, code is being written I wouldn't have spent the time working on, unit testing is being used to make sure they don't break anything as the project progresses and the codebase is being kept pretty sound because I know enough to see when they're going off the rails as they often do.