3 ms·

> But often prompting is more complex than programming something. I'd challenge this one; is it more complex, or is all the thinking and decision making concen

> But often prompting is more complex than programming something. I'd challenge this one; is it more complex, or is all the thinking and decision making concentrated into a single sentence or paragraph? For me, programming something is taking a big high over problem and breaking it down into smaller and smaller sections until it's a line of code; the lines of code are relatively low effort / cost little brain power. But in my experience, the problem itself and its nuances are only defined once all code is written. If you have to prompt an AI to write it, you need to define the problem beforehand. It's more design and more thinking upfront, which is something the development community has moved away from in the past ~20 years with the rise of agile development and open source. Techniques like TDD have shifted more of the problem definition forwards as you have to think about your desired outcomes before writing code, but I'm pretty sure (I have no figures) it's only a minority of developers that have the self-discipline to practice test-driven development consistently. (disclaimer: I don't use AI much, and my employer isn't yet looking into or paying for agentic coding, so it's chat style or inline code suggestions)

sksisoakanan 1y ago
The issue with prompting is English (or any other human language) is nowhere near as rigid or strict a language as a programming language. Almost always an idea can be expressed much more succinctly in code than language. Combine that with when you’re reading the code it’s often much easier to develop a prototype solution as you go and you end up with prompting feeling like using 4 men to carry a wheelbarrow instead of having 1 push it.
- michaelfeathers 1y ago
  I think we are going to end up with common design/code specification language that we use for prompting and testing. There's always going to be a need to convey the exact semantics of what we want. If not, for AI then for the humans who have to grapple with what is made.
  - rerdavies 1y ago
    Sounds like "Heavy process". "Specifying exact semantics" has been tried and ended up unimaginably badly.
    - bcrosby95 1y ago
      Nah, imagine a programming language optimized for creating specifications. Feed it to an llm and it implements it. Ideally it can also verify it's solution with your specification code. If LLMs don't gain significantly more general capabilities I could see this happening in the longer term. But it's too early to say. In a sense the llm turns into a compiler.
      - cess11 1y ago
        We've had that for a long, long time. Notably RAD-tooling running on XML. The main lesson has been that it's actually not much of an enabler and the people doing it end up being specialised and rather expensive consultants.
        CamperBob2 1y ago
        RAD before transformers was like trying to build an iPhone before capacitive multitouch: a total waste of time. Things are different now.
        cess11 1y ago
        I'm not so sure. What can you show me that you think would be convincing?
        CamperBob2 1y ago
        I think there are enough examples of genuine AI-facilitated rapid application development out there already, honestly. I wouldn't have anything to add to the pile, since I'm not a RAD kind of guy. Disillusionment seems to spring from expecting the model to be a god or a genie instead of a code generator. Some people are always going to be better at using tools than other people are. I don't see that changing, even though the tools themselves are changing radically.
algorithmsRcool 1y ago
> It's more design and more thinking upfront, which is something the development community has moved away from in the past ~20 years with the rise of agile development and open source. I agree, but even smaller than thinking in agile is just a tight iteration loop when i'm exploring a design. My ADHD makes upfront design a challenge for me and I am personally much more effective starting with a sketch of what needs to be done and then iterating on it until I get a good result. The loop of prompt->study->prompt->study... is disruptive to my inner loop for several reasons, but a big one is that the machine doesn't "think" like i do. So the solutions it scaffolds commonly make me say "huh?" and i have to change my thought process to interpet them and then study them for mistakes. My intution and iteration is, for the time being, more effective than this machine assited loop for the really "interesting" code i have to write. But i will say that AI has been a big time saver for more mundane tasks, especially when I can say "use this example and apply it to the rest of this code/abstraction".
- samsepi01 1y ago
  > "The loop of prompt->study->prompt->study... is disruptive to my inner loop for several reasons, but a big one is that the machine doesn't "think" like i do. So the solutions it scaffolds commonly make me say "huh?" and i have to change my thought process to interpet them and then study them for mistakes. My intution and iteration is, for the time being, more effective than this machine assisted loop..." My thoughts exactly as an ADHD dev. Was having trouble describing my main issue with LLM-assisted development... Thank you for giving me the words!
starlust2 1y ago
A big challenge is that programmers all have unique ever changing personal style and vision that they've never had to communicate before. As well they generally "bikeshed" and add undefined unrequested requirements, because you know someday we might need to support 10000x more users than we have. This is all well and good when the programmer implements something themselves but falls apart when it must be communicated to an LLM. Most projects/systems/orgs don't have the necessary level of detail in their documentation, documentation is fragmented across git/jira/confluence/etc/etc/etc., and it's a hodge podge of technologies without a semblance of consistency. I think we'll find that over the next few years the first really big win will be AI tearing down the mountain of tech & documentation debt. Bringing efficiency to corporate knowledge is likely a key element to AI working within them.
- mlsu 1y ago
  Efficiency to corporate knowledge? Absolutely not, no way. My coworkers are beginning to use AI to write PR descriptions and git commits. I notice, because the amount of text has been increased tenfold while the amount of information has stayed exactly the same. This is a torrent of shit coming down on us, that we are all going to have to deal with it. The vibe coders will be gleefully putting up PRs with 12 paragraphs of "descriptive" text. Thanks no thanks!
  - aloisdg 1y ago
    Use a llm to summarize the PR /j
  - starlust2 1y ago
    Well I'm certainly not saying that AI should generate more corporate spam. That's part of them problem! And also a strawman argument!
bcrosby95 1y ago
I design and think upfront but I don't write it down until I start coding. I can do this for pretty large chunks of code at once. The fastest way I can transcribe a design is with code or pseudocode. Converting it into English can be hard. It reminds me a bit of the discussion of if you have an inner monologue. I don't and turning thoughts into English takes work, especially if you need to be specific with what you want.
- averageRoyalty 1y ago
  I also don't have an inner monologue and can relate somewhat. However I find that natural language (usually) allows me to be more expressive than pseudocode in the same period of time. There's also an intangible benefit of having someone to "bounce off". If I'm using an LLM, I am tweaking the system prompt to slow it down, make it ask questions and bug me before making changes. Even without that, writing out the idea displays quickly potential logic or approach flaws - much fast than writing pseudo in my experience.

sksisoakanan 1y ago

The issue with prompting is English (or any other human language) is nowhere near as rigid or strict a language as a programming language. Almost always an idea can be expressed much more succinctly in code than language. Combine that with when you’re reading the code it’s often much easier to develop a prototype solution as you go and you end up with prompting feeling like using 4 men to carry a wheelbarrow instead of having 1 push it.

michaelfeathers 1y ago
I think we are going to end up with common design/code specification language that we use for prompting and testing. There's always going to be a need to convey the exact semantics of what we want. If not, for AI then for the humans who have to grapple with what is made.
- rerdavies 1y ago
  Sounds like "Heavy process". "Specifying exact semantics" has been tried and ended up unimaginably badly.
  - bcrosby95 1y ago
    Nah, imagine a programming language optimized for creating specifications. Feed it to an llm and it implements it. Ideally it can also verify it's solution with your specification code. If LLMs don't gain significantly more general capabilities I could see this happening in the longer term. But it's too early to say. In a sense the llm turns into a compiler.
    - cess11 1y ago
      We've had that for a long, long time. Notably RAD-tooling running on XML. The main lesson has been that it's actually not much of an enabler and the people doing it end up being specialised and rather expensive consultants.
      - CamperBob2 1y ago
        RAD before transformers was like trying to build an iPhone before capacitive multitouch: a total waste of time. Things are different now.
        cess11 1y ago
        I'm not so sure. What can you show me that you think would be convincing?
        CamperBob2 1y ago
        I think there are enough examples of genuine AI-facilitated rapid application development out there already, honestly. I wouldn't have anything to add to the pile, since I'm not a RAD kind of guy. Disillusionment seems to spring from expecting the model to be a god or a genie instead of a code generator. Some people are always going to be better at using tools than other people are. I don't see that changing, even though the tools themselves are changing radically.

algorithmsRcool 1y ago

> It's more design and more thinking upfront, which is something the development community has moved away from in the past ~20 years with the rise of agile development and open source. I agree, but even smaller than thinking in agile is just a tight iteration loop when i'm exploring a design. My ADHD makes upfront design a challenge for me and I am personally much more effective starting with a sketch of what needs to be done and then iterating on it until I get a good result. The loop of prompt->study->prompt->study... is disruptive to my inner loop for several reasons, but a big one is that the machine doesn't "think" like i do. So the solutions it scaffolds commonly make me say "huh?" and i have to change my thought process to interpet them and then study them for mistakes. My intution and iteration is, for the time being, more effective than this machine assited loop for the really "interesting" code i have to write. But i will say that AI has been a big time saver for more mundane tasks, especially when I can say "use this example and apply it to the rest of this code/abstraction".

samsepi01 1y ago
> "The loop of prompt->study->prompt->study... is disruptive to my inner loop for several reasons, but a big one is that the machine doesn't "think" like i do. So the solutions it scaffolds commonly make me say "huh?" and i have to change my thought process to interpet them and then study them for mistakes. My intution and iteration is, for the time being, more effective than this machine assisted loop..." My thoughts exactly as an ADHD dev. Was having trouble describing my main issue with LLM-assisted development... Thank you for giving me the words!

starlust2 1y ago

A big challenge is that programmers all have unique ever changing personal style and vision that they've never had to communicate before. As well they generally "bikeshed" and add undefined unrequested requirements, because you know someday we might need to support 10000x more users than we have. This is all well and good when the programmer implements something themselves but falls apart when it must be communicated to an LLM. Most projects/systems/orgs don't have the necessary level of detail in their documentation, documentation is fragmented across git/jira/confluence/etc/etc/etc., and it's a hodge podge of technologies without a semblance of consistency. I think we'll find that over the next few years the first really big win will be AI tearing down the mountain of tech & documentation debt. Bringing efficiency to corporate knowledge is likely a key element to AI working within them.

mlsu 1y ago
Efficiency to corporate knowledge? Absolutely not, no way. My coworkers are beginning to use AI to write PR descriptions and git commits. I notice, because the amount of text has been increased tenfold while the amount of information has stayed exactly the same. This is a torrent of shit coming down on us, that we are all going to have to deal with it. The vibe coders will be gleefully putting up PRs with 12 paragraphs of "descriptive" text. Thanks no thanks!
- aloisdg 1y ago
  Use a llm to summarize the PR /j
- starlust2 1y ago
  Well I'm certainly not saying that AI should generate more corporate spam. That's part of them problem! And also a strawman argument!

bcrosby95 1y ago

I design and think upfront but I don't write it down until I start coding. I can do this for pretty large chunks of code at once. The fastest way I can transcribe a design is with code or pseudocode. Converting it into English can be hard. It reminds me a bit of the discussion of if you have an inner monologue. I don't and turning thoughts into English takes work, especially if you need to be specific with what you want.

averageRoyalty 1y ago
I also don't have an inner monologue and can relate somewhat. However I find that natural language (usually) allows me to be more expressive than pseudocode in the same period of time. There's also an intangible benefit of having someone to "bounce off". If I'm using an LLM, I am tweaking the system prompt to slow it down, make it ask questions and bug me before making changes. Even without that, writing out the idea displays quickly potential logic or approach flaws - much fast than writing pseudo in my experience.