3 ms·

I see this "prompting is an art" stuff a lot. I gave Claude a list of 10 <Route> objects and asked it to make an adjustment to all of them. It gave me 9 back. W

I see this "prompting is an art" stuff a lot. I gave Claude a list of 10 <Route> objects and asked it to make an adjustment to all of them. It gave me 9 back. When I asked it to try again it gave me 10 but one didn't work. What's "prompt engineering" there, telling it to try again until it gets it right? I'd rather just do it right the first time.

codr7 1y ago
We used to make fun of and look down on coders who mindlessly copy paste and mash the compile button until the code runs, for good reasons.
- pdimitar 1y ago
  Did you skip the "rigorously verify the LLM code" part of my comment on purpose, just to show contempt?
  - douglasisshiny 1y ago
    I think the point is you could just.... write the code yourself.
    - pdimitar 1y ago
      And I do, most of the time. Sometimes the code to be written is quite verbose and I am on a tighter deadline. And it still takes less time and effort to vet the LLM's code. Otherwise yeah.
tmpz22 1y ago
Prompt engineering is just trying that task on a variety of models and prompt variations until you can better understand the syntax needed to get the desired outcome, if the desired outcome can be gotten. Honestly you’re trying to prove AI is ineffective by telling us it didn’t work with your ineffective protocol. That is not a strong argument.
- only-one1701 1y ago
  What should I have done there? Tell it to make sure that it gives me all 10 objects I give it back? Tell it to not put brackets in the wrong place? This is a real question --- what would you have done?
  - simonw 1y ago
    How long ago was this? I'd be surprised to see Claude 3.7 Sonnet make a mistake of this nature. Either way, when a model starts making dumb mistakes like that these days I start a fresh conversation (to blow away all of the bad tokens in the current one), either with that model or another one. I often switch from Claude 3.7 Sonnet to o3 or o4-mini these days. I paste in the most recent "good" version of the thing we're working on and prompt from there.
    - th0ma5 1y ago
      Lol, "it didn't do it... and if it did it didn't mean it... and if it meant it it surely can't mean it now." This is unserious.
      - gotimo 1y ago
        this is the rhetoric that you will see replied to effectively any negative experience with LLMs in programming.
        th0ma5 1y ago
        Yeah I'm starting to think they aren't aware of the shell game in their own rhetoric. If nothing can ever be wrong then nothing is right either in that worldview.
      - simonw 1y ago
        A full two thirds of the comment you replied to there were me saying "when these things start to make dumb mistakes here are the steps I take to fix the problem".
pdimitar 1y ago
Then don't use it? Nobody is making you. I am also barely using LLMs at the moment. Even 10% of the time would be generous. What I was saying is that I have tried different ways of interacting with LLMs and was happy to discover that the way I describe stuff to another senior dev actually works quite fine with an LLM. So I stuck to that. Again, if an LLM is not up to your task, don't waste your time with it. I am not advocating for "forget everything you knew and just go ask Mr. AI". I am advocating for enabling and productivity-boosting. Some tasks I hate, for some I lack the deeper expertise, others are just verbose and require a ton of typing. If you can prompt the LLM well and vet the code yourself after (something many commenters here deliberately omit so they can happily tear down their straw man) then the LLM will be a net positive. It's one more tool in the box. That's all there is to it really. No idea why people get so polarizing.

codr7 1y ago

We used to make fun of and look down on coders who mindlessly copy paste and mash the compile button until the code runs, for good reasons.

pdimitar 1y ago
Did you skip the "rigorously verify the LLM code" part of my comment on purpose, just to show contempt?
- douglasisshiny 1y ago
  I think the point is you could just.... write the code yourself.
  - pdimitar 1y ago
    And I do, most of the time. Sometimes the code to be written is quite verbose and I am on a tighter deadline. And it still takes less time and effort to vet the LLM's code. Otherwise yeah.

tmpz22 1y ago

Prompt engineering is just trying that task on a variety of models and prompt variations until you can better understand the syntax needed to get the desired outcome, if the desired outcome can be gotten. Honestly you’re trying to prove AI is ineffective by telling us it didn’t work with your ineffective protocol. That is not a strong argument.

only-one1701 1y ago
What should I have done there? Tell it to make sure that it gives me all 10 objects I give it back? Tell it to not put brackets in the wrong place? This is a real question --- what would you have done?
- simonw 1y ago
  How long ago was this? I'd be surprised to see Claude 3.7 Sonnet make a mistake of this nature. Either way, when a model starts making dumb mistakes like that these days I start a fresh conversation (to blow away all of the bad tokens in the current one), either with that model or another one. I often switch from Claude 3.7 Sonnet to o3 or o4-mini these days. I paste in the most recent "good" version of the thing we're working on and prompt from there.
  - th0ma5 1y ago
    Lol, "it didn't do it... and if it did it didn't mean it... and if it meant it it surely can't mean it now." This is unserious.
    - gotimo 1y ago
      this is the rhetoric that you will see replied to effectively any negative experience with LLMs in programming.
      - th0ma5 1y ago
        Yeah I'm starting to think they aren't aware of the shell game in their own rhetoric. If nothing can ever be wrong then nothing is right either in that worldview.
    - simonw 1y ago
      A full two thirds of the comment you replied to there were me saying "when these things start to make dumb mistakes here are the steps I take to fix the problem".

pdimitar 1y ago

Then don't use it? Nobody is making you. I am also barely using LLMs at the moment. Even 10% of the time would be generous. What I was saying is that I have tried different ways of interacting with LLMs and was happy to discover that the way I describe stuff to another senior dev actually works quite fine with an LLM. So I stuck to that. Again, if an LLM is not up to your task, don't waste your time with it. I am not advocating for "forget everything you knew and just go ask Mr. AI". I am advocating for enabling and productivity-boosting. Some tasks I hate, for some I lack the deeper expertise, others are just verbose and require a ton of typing. If you can prompt the LLM well and vet the code yourself after (something many commenters here deliberately omit so they can happily tear down their straw man) then the LLM will be a net positive. It's one more tool in the box. That's all there is to it really. No idea why people get so polarizing.