3 ms·

I couldn’t agree more and I’ve said the same thing many times. I have yet to see an LLM-generated app not collapse under it’s own weight after enough iteration

I couldn’t agree more and I’ve said the same thing many times. I have yet to see an LLM-generated app not collapse under it’s own weight after enough iterations/prompts. It gets stuck in loops (removing and adding the same code/concept), it gets hung up on simple errors, etc. For greenfield it’s amazing, no doubt, but unless you are watching it very closely and approving/reviewing the code along the way it will go off the rails. At a certain point it’s easier to add the new feature or make the modification yourself. Even if the LLM could do it, it would burn tons of money and time. I expect things to get better, this will not always be the state of things, but for now “vibe coding” (specifically not reviewing/writing code yourself) is not sustainable. Most people doing it have a github profile that is a mile wide and a meter deep. LLM’s are amazing and useful, but “vibe coding” with them is not sustainable currently.

RajT88 1y ago
Coding LLM's: Ruby off rails?
bredren 1y ago
The way to get past the removing and adding is to have the prompt include what has already been done and what is left to do. Then specify the need to conclude the work by a deadline. These kinds of things cause the LLM to “finish” tasks and try to move on or say it is done when it is. This won’t let you skip the curation of output along the way, but at least some of the stumbling around can be handled with prompting.
noosphr 1y ago
>I expect things to get better, this will not always be the state of things, but for now “vibe coding” (specifically not reviewing/writing code yourself) is not sustainable. It will not. And I say this as someone whose been building internal LLM tools since 2021. The issue is their context window. If you increase the context window so they can see more code costs skyrocket as n^2 the size of the code base. If you don't then you have all the issues people have in this thread. The reason why I have a job right now is that you can get around this by building tooling for intelligent search that limits the overfill of each context window. This is neither easy, fast, or cheap when done at scale. Worse the problems that you have when doing this are at best very weakly related to the problems the major AI labs are focusing on currently - I've interviewed at two of the top five AI labs and none of the people I talked to cared or really understood what a _real_ agentic system that solves coding should look like.
- quartzeee 1y ago
  > building tooling for intelligent search that limits the overfill of each context window I'm interested to know what you mean by this, in our system we've been trying to compress the context but this is the first I've seen about filtering it down.
  - noosphr 1y ago
    For general text you run some type of vector search against the full-text corpus to see what relevant hits there are and where. Then you feed the first round of results into a ranking/filtering system which does pair wise comparison between each chunk that you've had a good score from the vector search. Contract/expand until you've reach the limit of the context window for your model and run against the original query. For source code, you are even luckier since there are a lot of deterministic tools which provide solid grounding, e.g., etags, and the languages themselves enforce a hierarchical tree-like structure on the source code, viz. block statements. The above means that ranking and chunking strategies are solved already - which is a huge pain for general text. The vector search is then just an enrichment layer on top which brings in documentation and other soft grounding text that keeps the LLM from going berserk. Of course, none of the commercial offerings come even close to letting you do this well. Even the dumb version of search needs to be a self-recursive agent which comes with a good set of vector embeddings and the ability to decide if it's searched enough before it starts answering your questions. If you're interested drop a line on my profile email.
- FLT8 1y ago
  I can't help but wonder whether the solution here is something like building a multi-resolution understanding of the codebase. All the way from an architectural perspective including business context, down to code structure & layout, all the way down to what's happening in specific files and functions. As a human, I don't need to remember the content of every file I work on to be effective, but I do need to understand how to navigate my way around, and enough of how the codebase hangs together to be able to make good decisions about where new code belongs, when and how to refactor etc.. I'm pretty sure I don't have the memory or reading comprehension to match a computer, but I do have the ability to form context maps at different scales and switch 'resolution' depending on what I'm hoping to achieve.
cwsx 1y ago
> For greenfield it’s amazing I'll preface this comment with: I am a recent startup owner (so only dev, which is important) and my entire codebase has been generated via Sonnet (mostly 3.7, now using 4.0). If you actually looked at the work I'm (personally) producing, I guess I'm more of a product-owner/project-manager as I'm really just overseeing the development. > I have yet to see an LLM-generated app not collapse under it’s own weight after enough iterations/prompts. There's a few crucial steps to make an LLM-generated app maintainable (by the LLM): - _have a very, very strong SWE background_; ideally as a "strong" Lead Dev, _this is critical_ - your entire workflow NEEDS to be centered around LLM-development (or even model-specific): - use MCPs wherever possible and make sure they're specifically configured for your project - don't write "human" documentation; use rule + reusable prompt files - you MUST do this in a *very* granular but specialized way; keep rules/prompts very small (like you would when creating tickets) - make sure rules are conditionally applied (using globs); do not auto include anything except your "system rules" - use the LLM to generate said prompts and rules; this forces consistency across prompts, very important - follow a typical agile workflow (creating epics, tickets, backlogs etc) - TESTS TESTS AND MORE TESTS; add automated tools (like linters) EVERYWHERE you can - keep your code VERY modular so the LLM can keep a focused context, rules should provide all key context (like the broader architecture); the goal is for your LLM to only need to read or interact with files related to the strict 'current task' scope - iterating on code is almost always more difficult than writing it from scratch: provided your code is well architected, no single rewrite should be larger than a regular ticket (if the ticket is too large then it needs to be split up) This is off the top of my head so it's pretty broad/messy but I can expand on my points. LLM-coding requires a complete overhaul of your workflow so it is tailored specifically to an LLM, not a human, but this is also a massive learning curve (that take's a lot of time to figure out and optimize). Would I bother doing this if I were still working on a team? Probably not, I don't think it would've saved me much time in a "regular" codebase. As a single developer at a startup? This is the only way I've been able to get "other startup-y" work done while also progressing the codebase - the value of being able to do multiple things at a time, let the LLM and intermittently review the output while you get to work on other things. The biggest tip I can give: LLMs struggle at "coding like a human" and are much better at "bad-practice" workflows (e.g. throwing away large parts of code in favour of a total rewrite) - let the LLM lead the development process, with the rules/prompts as guardrails, and try stay out of it's way while it works (instead of saying "hey X thing didn't work, go fix that now") - hold its hand but let it experiment before jumping in.

RajT88 1y ago

Coding LLM's: Ruby off rails?

bredren 1y ago

The way to get past the removing and adding is to have the prompt include what has already been done and what is left to do. Then specify the need to conclude the work by a deadline. These kinds of things cause the LLM to “finish” tasks and try to move on or say it is done when it is. This won’t let you skip the curation of output along the way, but at least some of the stumbling around can be handled with prompting.

noosphr 1y ago

>I expect things to get better, this will not always be the state of things, but for now “vibe coding” (specifically not reviewing/writing code yourself) is not sustainable. It will not. And I say this as someone whose been building internal LLM tools since 2021. The issue is their context window. If you increase the context window so they can see more code costs skyrocket as n^2 the size of the code base. If you don't then you have all the issues people have in this thread. The reason why I have a job right now is that you can get around this by building tooling for intelligent search that limits the overfill of each context window. This is neither easy, fast, or cheap when done at scale. Worse the problems that you have when doing this are at best very weakly related to the problems the major AI labs are focusing on currently - I've interviewed at two of the top five AI labs and none of the people I talked to cared or really understood what a _real_ agentic system that solves coding should look like.

quartzeee 1y ago
> building tooling for intelligent search that limits the overfill of each context window I'm interested to know what you mean by this, in our system we've been trying to compress the context but this is the first I've seen about filtering it down.
- noosphr 1y ago
  For general text you run some type of vector search against the full-text corpus to see what relevant hits there are and where. Then you feed the first round of results into a ranking/filtering system which does pair wise comparison between each chunk that you've had a good score from the vector search. Contract/expand until you've reach the limit of the context window for your model and run against the original query. For source code, you are even luckier since there are a lot of deterministic tools which provide solid grounding, e.g., etags, and the languages themselves enforce a hierarchical tree-like structure on the source code, viz. block statements. The above means that ranking and chunking strategies are solved already - which is a huge pain for general text. The vector search is then just an enrichment layer on top which brings in documentation and other soft grounding text that keeps the LLM from going berserk. Of course, none of the commercial offerings come even close to letting you do this well. Even the dumb version of search needs to be a self-recursive agent which comes with a good set of vector embeddings and the ability to decide if it's searched enough before it starts answering your questions. If you're interested drop a line on my profile email.
FLT8 1y ago
I can't help but wonder whether the solution here is something like building a multi-resolution understanding of the codebase. All the way from an architectural perspective including business context, down to code structure & layout, all the way down to what's happening in specific files and functions. As a human, I don't need to remember the content of every file I work on to be effective, but I do need to understand how to navigate my way around, and enough of how the codebase hangs together to be able to make good decisions about where new code belongs, when and how to refactor etc.. I'm pretty sure I don't have the memory or reading comprehension to match a computer, but I do have the ability to form context maps at different scales and switch 'resolution' depending on what I'm hoping to achieve.

cwsx 1y ago

> For greenfield it’s amazing I'll preface this comment with: I am a recent startup owner (so only dev, which is important) and my entire codebase has been generated via Sonnet (mostly 3.7, now using 4.0). If you actually looked at the work I'm (personally) producing, I guess I'm more of a product-owner/project-manager as I'm really just overseeing the development. > I have yet to see an LLM-generated app not collapse under it’s own weight after enough iterations/prompts. There's a few crucial steps to make an LLM-generated app maintainable (by the LLM): - _have a very, very strong SWE background_; ideally as a "strong" Lead Dev, _this is critical_ - your entire workflow NEEDS to be centered around LLM-development (or even model-specific): - use MCPs wherever possible and make sure they're specifically configured for your project - don't write "human" documentation; use rule + reusable prompt files - you MUST do this in a *very* granular but specialized way; keep rules/prompts very small (like you would when creating tickets) - make sure rules are conditionally applied (using globs); do not auto include anything except your "system rules" - use the LLM to generate said prompts and rules; this forces consistency across prompts, very important - follow a typical agile workflow (creating epics, tickets, backlogs etc) - TESTS TESTS AND MORE TESTS; add automated tools (like linters) EVERYWHERE you can - keep your code VERY modular so the LLM can keep a focused context, rules should provide all key context (like the broader architecture); the goal is for your LLM to only need to read or interact with files related to the strict 'current task' scope - iterating on code is almost always more difficult than writing it from scratch: provided your code is well architected, no single rewrite should be larger than a regular ticket (if the ticket is too large then it needs to be split up) This is off the top of my head so it's pretty broad/messy but I can expand on my points. LLM-coding requires a complete overhaul of your workflow so it is tailored specifically to an LLM, not a human, but this is also a massive learning curve (that take's a lot of time to figure out and optimize). Would I bother doing this if I were still working on a team? Probably not, I don't think it would've saved me much time in a "regular" codebase. As a single developer at a startup? This is the only way I've been able to get "other startup-y" work done while also progressing the codebase - the value of being able to do multiple things at a time, let the LLM and intermittently review the output while you get to work on other things. The biggest tip I can give: LLMs struggle at "coding like a human" and are much better at "bad-practice" workflows (e.g. throwing away large parts of code in favour of a total rewrite) - let the LLM lead the development process, with the rules/prompts as guardrails, and try stay out of it's way while it works (instead of saying "hey X thing didn't work, go fix that now") - hold its hand but let it experiment before jumping in.