3 ms·

In my experience, once it starts telling me I’m right, we’re already going downhill and it rarely gets better from there.

flkiwi 1y ago
Sometimes I just ride the lightning to see how off course it is willing to go. This is not a productive use of my time but it sure is amusing. In fairness, I’ve done the same thing to overconfident junior colleagues.
- al_borland 1y ago
  I spent yesterday afternoon doing this. It go to the point where it would acknowledge it was wrong, but would keep giving me the same answer. It also said it would only try one more time before giving up, but then kept going.
  - dingnuts 1y ago
    this happens to me constantly, it's such a huge waste of time. I'm not convinced any of these tools actually save time. It's all a fucking slot machine and Gell-Mann Amnesia and at the end, you often have nothing that works. I spent like two hours yesterday dicking with aider to make a one line change and it hallucinated an invalid input for the only possible parameter and I wound up using the docs the old fashioned way and doing the task in about two minutes
    - brianwawok 1y ago
      The mistake was using AI for a two minute fix. It totally helps at some tasks. Takes some failures to realize that it does indeed have flaws.
      - deleted 1y ago
        [deleted]
  - CuriouslyC 1y ago
    The worst is when it can't do something right, and it does a horrible mock/hack to get it "working." I had a claude fake benchmark data, that pissed me off a bit, though I did make a major architectural improvement to a tool as result (though the real benchmark would have probably made me do it anyhow) so it wasn't all horrible.
- neilv 1y ago
  They should've called it "riding the lightning": https://en.wikipedia.org/wiki/Socratic_dialogue https://en.wikipedia.org/wiki/Socratic_dialogue
- theshrike79 1y ago
  It's also educational to play stupid with LLMs Even if you know exactly where the issue is an it would be a 30 second job to do it manually, you can get a better feel on how to direct it to see what you need to tell it to have it find out the issue and fixing it.
anthem2025 1y ago
Usually it’s a response to my profanity laden “what are you doing? Why? Don’t do that! Stop! Do this instead”
lemming 1y ago
Yeah, I want a feature which stops my agent as soon as it says anything even vaguely like: "let me try another approach". Right after that is when the wheels start falling off, tests get deleted, etc. That phrase is a sure sign the agent should (but never does) ask me for guidance.
- al_borland 1y ago
  I’ve found even giving guidance at this point doesn’t help, as it fundamentally doesn’t get it. I was down one of these rabbit holes with it once while having it write a relatively simple bash script. Something I had written by hand previously in Python, but wanted a bash version and also wanted to see what AI could do. It was 98% there, but couldn’t get that last 2% to save its life. Eventually I went through the code myself, found the bug, and I told it exactly what the bug was and where it was at; it was an off-by-one error. Even when spoon feeding it, it couldn’t fix it and I ended up doing it myself just to get it over with.
deleted 1y ago
[deleted]

flkiwi 1y ago

Sometimes I just ride the lightning to see how off course it is willing to go. This is not a productive use of my time but it sure is amusing. In fairness, I’ve done the same thing to overconfident junior colleagues.

al_borland 1y ago
I spent yesterday afternoon doing this. It go to the point where it would acknowledge it was wrong, but would keep giving me the same answer. It also said it would only try one more time before giving up, but then kept going.
- dingnuts 1y ago
  this happens to me constantly, it's such a huge waste of time. I'm not convinced any of these tools actually save time. It's all a fucking slot machine and Gell-Mann Amnesia and at the end, you often have nothing that works. I spent like two hours yesterday dicking with aider to make a one line change and it hallucinated an invalid input for the only possible parameter and I wound up using the docs the old fashioned way and doing the task in about two minutes
  - brianwawok 1y ago
    The mistake was using AI for a two minute fix. It totally helps at some tasks. Takes some failures to realize that it does indeed have flaws.
    - deleted 1y ago
      [deleted]
- CuriouslyC 1y ago
  The worst is when it can't do something right, and it does a horrible mock/hack to get it "working." I had a claude fake benchmark data, that pissed me off a bit, though I did make a major architectural improvement to a tool as result (though the real benchmark would have probably made me do it anyhow) so it wasn't all horrible.
neilv 1y ago
They should've called it "riding the lightning": https://en.wikipedia.org/wiki/Socratic_dialogue https://en.wikipedia.org/wiki/Socratic_dialogue
theshrike79 1y ago
It's also educational to play stupid with LLMs Even if you know exactly where the issue is an it would be a 30 second job to do it manually, you can get a better feel on how to direct it to see what you need to tell it to have it find out the issue and fixing it.

anthem2025 1y ago

Usually it’s a response to my profanity laden “what are you doing? Why? Don’t do that! Stop! Do this instead”

lemming 1y ago

Yeah, I want a feature which stops my agent as soon as it says anything even vaguely like: "let me try another approach". Right after that is when the wheels start falling off, tests get deleted, etc. That phrase is a sure sign the agent should (but never does) ask me for guidance.

al_borland 1y ago
I’ve found even giving guidance at this point doesn’t help, as it fundamentally doesn’t get it. I was down one of these rabbit holes with it once while having it write a relatively simple bash script. Something I had written by hand previously in Python, but wanted a bash version and also wanted to see what AI could do. It was 98% there, but couldn’t get that last 2% to save its life. Eventually I went through the code myself, found the bug, and I told it exactly what the bug was and where it was at; it was an off-by-one error. Even when spoon feeding it, it couldn’t fix it and I ended up doing it myself just to get it over with.

deleted 1y ago

[deleted]