3 ms·

So you do know how the general "writing specification" part is working, you just have the wrong process. Instead of iterating and adding more context on top, re

So you do know how the general "writing specification" part is working, you just have the wrong process. Instead of iterating and adding more context on top, restructure your initial prompt to include the context. DONT DO: First pass: "Create a function [in language $X] with two arguments, a and b, which returns adding those two together" Second pass: "It must take u8 types, not u32 types" INSTEAD DO: First pass: "Create a function [in language $X] with two arguments, a and b, which returns adding those two together" Second pass: "Create a function [in language $X] with two arguments, a and b, both using u8, which returns adding those two together" ---- What you don't want to do, is adding additional messages/context on top of "known bad" context, so instead you should take the clue that the LLM didn't understand correctly as "I need to edit my prompt" not "I need to now after their reply, add more context to correct what was wrong". The goal should be to completely avoid anything bad, not correct it. Together with this, you build up a system/developer prompt you can reuse across projects/scopes, that follows how you code. In that, you add stuff as you discover what's needed to be added, like "Make sure to always handle Exceptions in X way" or similar. > > For just a simple "add two numbers" function, the specification can easily exceed the actual code. So you can probably understand the skepticism when the task is not trivial, and depends on a lot of existing code. Yes, please be skeptical, I am as well, which I guess is why I am seemingly more effective at using LLMs than others who are less skeptical. It's a benefit here to be skeptical, not a drawback. And yes, it isn't trivial to verify work that others have done for you, when you have a concrete idea of how it should be exactly. But as I managed to work with outsourced/contracting developers before, or even collaborate with developers in the same company as me, I also learned to use LLMs in a similar way where you have to review and ensure code follow the architecture/design you intended.

troupo 10mo ago
> INSTEAD DO: > First pass: "Create a function [in language $X] with two arguments, a and b, which returns adding those two together" > Second pass: "Create a function [in language $X] with two arguments, a and b, both using u8, which returns adding those two together" So it will create two different functions (and LLMs do love to ignore anything that came before and create a lot of stuff from scratch again and again). Now what.
- embedding-shape 10mo ago
  What? No, I think you fundamentally misunderstand what workflow I'm suggesting here. You ask: "Do X". The LLM obliges, gives you something you don't want. At this point, don't accept/approve it, so nothing has changed, you still have an empty directory, or whatever. Then you start a brand new context, with iteration on the prompt: "Do X with Y", and the LLM again tries to do it. If something is wrong, repeat until you get what you're happy with, extract what you can into reusable system/developer prompts, then accept/approve the change. Then you end up with one change, and one function, exactly as you specified it. Then if you want, you can re-run the exact same prompt, with the exact same context (nothing!) and you'll get the same results. "LLMs do love to ignore anything that came before" literally cannot happen in this workflow, because there is nothing that "came before".
  - troupo 10mo ago
    > No, I think you fundamentally misunderstand what workflow I'm suggesting here. Ah. Basically meaningless monkey work of baby sitting an eager junior developer. And this is for a simple thing like adding two numbers. See how it doesn't scale at all with anything remotely complex? > "LLMs do love to ignore anything that came before" literally cannot happen in this workflow, because there is nothing that "came before". Of course it can. Because what came before is the project you're working on. Unless of course you end up specifying every single utility function and every single library call in your specs. Which, once again, doesn't scale.
    - embedding-shape 10mo ago
      > See how it doesn't scale at all with anything remotely complex? No, I don't. Does outsourcing not work for you with "anything remotely complex"? Then yeah, LLMs won't help you, because that's a communication issue. Once you figure out how to communicate, using LLMs even for "anything remotely complex" becomes trivial, but requires an open mind. > Because what came before is the project you're working on. Right, if that's what you meant, then yeah, of course they don't ignore the existing code, if there is a function that already does what it needs, it'll use that. If the agent/LLM you use doesn't automatically does this, I suggest you try something better, like Codex or Claude Code. But anyways, you don't really seem like you're looking for improving, but instead try to dismiss better techniques available, so I'm not even sure why I'm trying to help you here. Hopefully at least someone who wants to improve comes across it so this whole conversation wasn't a complete waste of time.

troupo 10mo ago

> INSTEAD DO: > First pass: "Create a function [in language $X] with two arguments, a and b, which returns adding those two together" > Second pass: "Create a function [in language $X] with two arguments, a and b, both using u8, which returns adding those two together" So it will create two different functions (and LLMs do love to ignore anything that came before and create a lot of stuff from scratch again and again). Now what.

embedding-shape 10mo ago
What? No, I think you fundamentally misunderstand what workflow I'm suggesting here. You ask: "Do X". The LLM obliges, gives you something you don't want. At this point, don't accept/approve it, so nothing has changed, you still have an empty directory, or whatever. Then you start a brand new context, with iteration on the prompt: "Do X with Y", and the LLM again tries to do it. If something is wrong, repeat until you get what you're happy with, extract what you can into reusable system/developer prompts, then accept/approve the change. Then you end up with one change, and one function, exactly as you specified it. Then if you want, you can re-run the exact same prompt, with the exact same context (nothing!) and you'll get the same results. "LLMs do love to ignore anything that came before" literally cannot happen in this workflow, because there is nothing that "came before".
- troupo 10mo ago
  > No, I think you fundamentally misunderstand what workflow I'm suggesting here. Ah. Basically meaningless monkey work of baby sitting an eager junior developer. And this is for a simple thing like adding two numbers. See how it doesn't scale at all with anything remotely complex? > "LLMs do love to ignore anything that came before" literally cannot happen in this workflow, because there is nothing that "came before". Of course it can. Because what came before is the project you're working on. Unless of course you end up specifying every single utility function and every single library call in your specs. Which, once again, doesn't scale.
  - embedding-shape 10mo ago
    > See how it doesn't scale at all with anything remotely complex? No, I don't. Does outsourcing not work for you with "anything remotely complex"? Then yeah, LLMs won't help you, because that's a communication issue. Once you figure out how to communicate, using LLMs even for "anything remotely complex" becomes trivial, but requires an open mind. > Because what came before is the project you're working on. Right, if that's what you meant, then yeah, of course they don't ignore the existing code, if there is a function that already does what it needs, it'll use that. If the agent/LLM you use doesn't automatically does this, I suggest you try something better, like Codex or Claude Code. But anyways, you don't really seem like you're looking for improving, but instead try to dismiss better techniques available, so I'm not even sure why I'm trying to help you here. Hopefully at least someone who wants to improve comes across it so this whole conversation wasn't a complete waste of time.