7 ms·

On a more important level, I found that they still do really badly at even a minorly complex task without extreme babysitting. I wanted it to refactor a parser

On a more important level, I found that they still do really badly at even a minorly complex task without extreme babysitting. I wanted it to refactor a parser in a small project (2.5K lines total) because it'd gotten a bit too interconnected. It made a plan, which looked reasonable, so I told it to do this in stages, with checkpoints. It said it'd done so. I asked it "so is the old architecture also removed?" "No, it has not been removed." "Is the new structured used in place of the old one?" "No, it has not." After it did so, 80% of the test suite failed because nothing it'd written was actually right. Did so three times with increasingly more babysitting, but it failed at the abstract task of "refactor this" no matter what with pretty much the same failure mode. I feel like I have to tell it exactly to make changes X and Y to class Z, remove class A etc etc, at which point I can't let it do stuff unsupervised, which is half of the reason for letting an LLM do this in the first place.

hu3 1y ago
Interesting. What model and tool was used? I have seen similar failure modes in Cursor and VSCode Copilot (using gpt5) where I have to babysit relatively small refactors.
- AllegedAlec 1y ago
  Claude code. Whichever model it started up automatically last weekend, I didn't explicitly check.
  - rglynn 1y ago
    This feels like a classic Sonnet issue. From my experience, Opus or GPT-5-high are less likely to do the "narrow instruction following without making sensible wider decisions based on context" than Sonnet.
    - coldtea 1y ago
      This is "just use another Linux distro" all over again
      - rglynn 1y ago
        Yes and no, it's a fair criticism to some extent. Inasamuch as I would agree that different models of the same type have superficial differences. However, I also think that models which focus on higher reasoning effort in general are better at taking into account the wider context and not missing obvious implications from instructions. Non-reasoning or low-reasoning models serve a purpose, but to suggest they are akin to different flavours misses what is actually quite an important distinction.
deleted 1y ago
[deleted]
habibur 1y ago
Might be related with what the article was talking. AI can't cut-paste. It deletes the code and then regenerates it at another location instead of cut-paste. Obviously generated code drift a little from deleted ones.
jeswin 1y ago
> I wanted it to refactor a parser in a small project This expression tree parser (typescript to sql query builder - https://tinqerjs.org/ https://tinqerjs.org/) has zero lines of hand-written code. It was made with Codex + Claude over two weeks (part-time on the side). Having worked on ORMs previously, it would have taken me 4x-10x the time to get to the same state (which also has 100s of tests, with some repetitions). That's a massive saving in time. I did not have to baby sit the LLMs at all. So the answer is, I think it depends on what you use it for, and how you use it. Like every tool, it takes a really long time to find a process that works for you. In my conversations with other developers who use LLMs extensively, they all have their unique, custom workflows. All of them however do focus on test suites, documentation, and method review processes.
- iLoveOncall 1y ago
  Hum yeah, it shows. Just the fact that the API looks completely different for Postgre and SQLite tells us everything we need to know about the quality of the project here.
  - jeswin 1y ago
    > Just the fact that the API looks completely different for Postgre and SQLite tells us everything we need to know about the quality of the project here. How does the API look completely different for pg and sqlite? Can you share an example? It's an implementation of LINQ's IQueryable. With some bells missing in DotNet's Queryable, like Window functions (RANK queries etc) which I find quite useful. Add: What you've mentioned is largely incorrect. But in any case, it is a query builder. Meaning, an ORM like database abstraction is not the goal. This allows us to support pg's extensions, which aren't applicable to other database.
  - pprotas 1y ago
    I guess the interesting question is whether @jeswin could have created this project at all if AI tools were not involved. And if yes, would the quality even be better?
    - 1y ago
coldtea 1y ago
>I feel like I have to tell it exactly to make changes X and Y to class Z, remove class A etc etc, at which point I can't let it do stuff unsupervised, which is half of the reason for letting an LLM do this in the first place. The reason better turn to "It can do stuff faster than I ever could if I give it step by step high level instructions" instead.
- AllegedAlec 1y ago
  That would be a solution, yes. But currently it feels extremely borked from a UX perspective. It purports to be able to do this, but when you tell it to it breaks in unintuitive ways. I hate this idea of "well you just need to understand all the arcane ways in which to properly use it to its proper effects". It's like a car which has a gear shifter, but that's not fully functional yet, so instead you switch gear by spelling out in morse code the gear you want to go into using L as short and R as long. Furthermore, you shouldn't try to listen to 105-112 on the FM band on the radio, because those frequencies are used to control the brakes and ABS and if you listen to those frequencies the brakes no longer work. We would rightfully stone any engineer who'd design this and then say "well obvious user error" when the user rightfully complains that they crash whenever they listen to Arrow FM.
  - coldtea 1y ago
    >But currently it feels extremely borked from a UX perspective. It purports to be able to do this, but when you tell it to it breaks in unintuitive ways. Thankfully as programmers we know better and don't need to care what the UI pretends to be able to do :) >We would rightfully stone any engineer who'd design this and then say "well obvious user error" when the user rightfully complains that they crash whenever they listen to Arrow FM. We might curse the company and engineer who did it, but we would still use that car and do those workarounds, if doing so allowed us to get to our destination in 1/10 the regular time...
    - AllegedAlec 1y ago
      > >But currently it feels extremely borked from a UX perspective. It purports to be able to do this, but when you tell it to it breaks in unintuitive ways. > Thankfully as programmers we know better and don't need to care what the UI pretends to be able to do :) But we do though. You can't just say "yeah they left all the foot guns in but we ought to know not to use them", especially not when the industry shills tell you those footguns are actually rocket boosters to get you to the fucking moon and back.
jansan 1y ago
I was hoping that LLMs being able to access strict tools, like Gemini using Python libraries, would finally give reliable results. So today I asked Gemini to simplify a mathematical expression with sympy. It did and explained to me how some part of the expression could be simplified wonderfully as a product of two factors. But it was all a lie. Even though I explicitly asked it to use sympy in order to avoid such hallucinations and get results that are actually correct, it used its own flawed reasoning on top and again gave me a completely wrong result. You still cannot trust LLMs. And that is a problem.
- ogogmad 1y ago
  The obvious point has to be made: Generating formal proofs might be a partial fix for this. By contrast, coding is too informal for this to be as effective for it.
deleted 1y ago
[deleted]

hu3 1y ago

Interesting. What model and tool was used? I have seen similar failure modes in Cursor and VSCode Copilot (using gpt5) where I have to babysit relatively small refactors.

AllegedAlec 1y ago
Claude code. Whichever model it started up automatically last weekend, I didn't explicitly check.
- rglynn 1y ago
  This feels like a classic Sonnet issue. From my experience, Opus or GPT-5-high are less likely to do the "narrow instruction following without making sensible wider decisions based on context" than Sonnet.
  - coldtea 1y ago
    This is "just use another Linux distro" all over again
    - rglynn 1y ago
      Yes and no, it's a fair criticism to some extent. Inasamuch as I would agree that different models of the same type have superficial differences. However, I also think that models which focus on higher reasoning effort in general are better at taking into account the wider context and not missing obvious implications from instructions. Non-reasoning or low-reasoning models serve a purpose, but to suggest they are akin to different flavours misses what is actually quite an important distinction.

deleted 1y ago

[deleted]

habibur 1y ago

Might be related with what the article was talking. AI can't cut-paste. It deletes the code and then regenerates it at another location instead of cut-paste. Obviously generated code drift a little from deleted ones.

jeswin 1y ago

> I wanted it to refactor a parser in a small project This expression tree parser (typescript to sql query builder - https://tinqerjs.org/ https://tinqerjs.org/) has zero lines of hand-written code. It was made with Codex + Claude over two weeks (part-time on the side). Having worked on ORMs previously, it would have taken me 4x-10x the time to get to the same state (which also has 100s of tests, with some repetitions). That's a massive saving in time. I did not have to baby sit the LLMs at all. So the answer is, I think it depends on what you use it for, and how you use it. Like every tool, it takes a really long time to find a process that works for you. In my conversations with other developers who use LLMs extensively, they all have their unique, custom workflows. All of them however do focus on test suites, documentation, and method review processes.

iLoveOncall 1y ago
Hum yeah, it shows. Just the fact that the API looks completely different for Postgre and SQLite tells us everything we need to know about the quality of the project here.
- jeswin 1y ago
  > Just the fact that the API looks completely different for Postgre and SQLite tells us everything we need to know about the quality of the project here. How does the API look completely different for pg and sqlite? Can you share an example? It's an implementation of LINQ's IQueryable. With some bells missing in DotNet's Queryable, like Window functions (RANK queries etc) which I find quite useful. Add: What you've mentioned is largely incorrect. But in any case, it is a query builder. Meaning, an ORM like database abstraction is not the goal. This allows us to support pg's extensions, which aren't applicable to other database.
- pprotas 1y ago
  I guess the interesting question is whether @jeswin could have created this project at all if AI tools were not involved. And if yes, would the quality even be better?
  - 1y ago

coldtea 1y ago

>I feel like I have to tell it exactly to make changes X and Y to class Z, remove class A etc etc, at which point I can't let it do stuff unsupervised, which is half of the reason for letting an LLM do this in the first place. The reason better turn to "It can do stuff faster than I ever could if I give it step by step high level instructions" instead.

AllegedAlec 1y ago
That would be a solution, yes. But currently it feels extremely borked from a UX perspective. It purports to be able to do this, but when you tell it to it breaks in unintuitive ways. I hate this idea of "well you just need to understand all the arcane ways in which to properly use it to its proper effects". It's like a car which has a gear shifter, but that's not fully functional yet, so instead you switch gear by spelling out in morse code the gear you want to go into using L as short and R as long. Furthermore, you shouldn't try to listen to 105-112 on the FM band on the radio, because those frequencies are used to control the brakes and ABS and if you listen to those frequencies the brakes no longer work. We would rightfully stone any engineer who'd design this and then say "well obvious user error" when the user rightfully complains that they crash whenever they listen to Arrow FM.
- coldtea 1y ago
  >But currently it feels extremely borked from a UX perspective. It purports to be able to do this, but when you tell it to it breaks in unintuitive ways. Thankfully as programmers we know better and don't need to care what the UI pretends to be able to do :) >We would rightfully stone any engineer who'd design this and then say "well obvious user error" when the user rightfully complains that they crash whenever they listen to Arrow FM. We might curse the company and engineer who did it, but we would still use that car and do those workarounds, if doing so allowed us to get to our destination in 1/10 the regular time...
  - AllegedAlec 1y ago
    > >But currently it feels extremely borked from a UX perspective. It purports to be able to do this, but when you tell it to it breaks in unintuitive ways. > Thankfully as programmers we know better and don't need to care what the UI pretends to be able to do :) But we do though. You can't just say "yeah they left all the foot guns in but we ought to know not to use them", especially not when the industry shills tell you those footguns are actually rocket boosters to get you to the fucking moon and back.

jansan 1y ago

I was hoping that LLMs being able to access strict tools, like Gemini using Python libraries, would finally give reliable results. So today I asked Gemini to simplify a mathematical expression with sympy. It did and explained to me how some part of the expression could be simplified wonderfully as a product of two factors. But it was all a lie. Even though I explicitly asked it to use sympy in order to avoid such hallucinations and get results that are actually correct, it used its own flawed reasoning on top and again gave me a completely wrong result. You still cannot trust LLMs. And that is a problem.

ogogmad 1y ago
The obvious point has to be made: Generating formal proofs might be a partial fix for this. By contrast, coding is too informal for this to be as effective for it.

deleted 1y ago

[deleted]