3 ms·

I agree. I tried something similar: a conversion of a simple PHP library from one system to another. It was only like 500 loc but Gemini 2.5 completely failed a

I agree. I tried something similar: a conversion of a simple PHP library from one system to another. It was only like 500 loc but Gemini 2.5 completely failed around line 300, and even then its output contained straight up hallucinations, half-brained additions, wrong namespaces for dependencies, badly indented code and other PSR style violations. Worse, it also changed working code and broke it.

blensor 2y ago
Did you paste it into the chat or did you use it with a coding agent like Cline? I am majorly impressed with the combination VSCode + Cline + Gemini Today I had it duplicate an esp32 proram from UDP communication to TCP. It first copied the file ( funnily enough by writing it again instead of just straight cp ) Then it started to just change all the headers and declarations Then in a third step it changed one bigger function And in the last step it changed some smaller functions And it reasoned exactly that way "Let's start with this first ... Let's now do this .... " until is was done
- ionwake 2y ago
  I’ve just moved from expensive claudecode to cursor and Gemini - what are you thoughts on cursor vs cline? Thank you
stavros 2y ago
Try asking it to generate a high-level plan of how it's going to do the conversion first, then to generate function definitions for the new functions, then have it generate tests for the new functions, then actually write them, while giving it the output of the tests. It's not like people just one-shot a whole module of code, why would LLMs?
- SpaceNoodled 2y ago
  Only 500 lines? That's miniscule.
- semi-extrinsic 2y ago
  I know many people who can and will one-shot a rewrite of 500 LOC. In my world, 500 LOC is about the length of a single function. I don't understand why we should be talking about generating a high level plan with multiple tests etc. for a single function. And I don't think this is uncommon. Just a random example from Github, this file is 1800 LOC and 4 functions. It implements one very specific thing that's part of a broader library. (I have no affiliation with this code.) https://github.com/elemental/Elemental/blob/master/src/optimization/solvers/SOCP/affine/IPM/Mehrotra.cpp https://github.com/elemental/Elemental/blob/master/src/optim...
  - stavros 2y ago
    > I don't understand why we should be talking about generating a high level plan with multiple tests etc. for a single function. You don't have to, you can write it by hand. I thought we were talking about how we can make computers write code, instead of humans, but it seems that we're trying to prove that LLMs aren't useful instead.
    - SpaceNoodled 2y ago
      No, it's simply being demonstrated that they're not as useful as some claim.
      - stavros 2y ago
        By saying "why do I have to use a specific technique, instead of naively, to get what I want"?
        SpaceNoodled 2y ago
        "Why do I have to put in more work to use this tool vs. not using it?"
        stavros 2y ago
        Which is exactly what I said here: https://news.ycombinator.com/item?id=43537443 https://news.ycombinator.com/item?id=43537443
- chrismorgan 2y ago
  > It's not like people just one-shot a whole module of code, why would LLMs? For conversions between languages or libraries, you often do just one-shot it, writing or modifying code from start to end in order. I remember 15 years ago taking a 10,000 line Java code base and porting it to JavaScript mostly like this, with only a few areas requiring a bit more involved and non-sequential editing.
  - copperx 2y ago
    So, you didn't test it until the end? or did you have to build it in such a way that is was partially testable?
    - chrismorgan 2y ago
      One of the nifty things about the target being JavaScript was that I didn’t have to finish it before I could run it—it was the sort of big library where typical code wouldn’t use most of the functionality. It was audio stuff, so there were a couple of core files that needed more careful porting (from whatever in Java to Mozilla’s Audio Data API, which was a fairly good match), and then the rest was fairly routine that could be done gradually, as I needed them or just when I didn’t have anything better to focus on. Honestly, one of the biggest problems was forgetting to prefix instance properties with `this.`
  - dietr1ch 2y ago
    I think this shows how the approach LLMs take is wrong. For us it's easy because we simply sort of iterate over every function with a simple prompt of doing a translation, but are yet careful enough taking notes of whatever may be relevant to do a higher level change if necessary. Maybe the mistake is mistaking LLMs as capable people instead of a simple, but optimised neuron soup tuned for text.
- fbernier 2y ago
  Isn't that basically the process some "thinking" models try to do for you under the hood? Prompting itself to improve your prompt. I actually have no idea but this is what I guessed it did when using it.
  - stavros 2y ago
    They do some variant of this, but this is more directed. They might not do this on their own, they might follow other lines of reasoning. Maybe more effective, maybe less.

blensor 2y ago

Did you paste it into the chat or did you use it with a coding agent like Cline? I am majorly impressed with the combination VSCode + Cline + Gemini Today I had it duplicate an esp32 proram from UDP communication to TCP. It first copied the file ( funnily enough by writing it again instead of just straight cp ) Then it started to just change all the headers and declarations Then in a third step it changed one bigger function And in the last step it changed some smaller functions And it reasoned exactly that way "Let's start with this first ... Let's now do this .... " until is was done

ionwake 2y ago
I’ve just moved from expensive claudecode to cursor and Gemini - what are you thoughts on cursor vs cline? Thank you

stavros 2y ago

Try asking it to generate a high-level plan of how it's going to do the conversion first, then to generate function definitions for the new functions, then have it generate tests for the new functions, then actually write them, while giving it the output of the tests. It's not like people just one-shot a whole module of code, why would LLMs?

SpaceNoodled 2y ago
Only 500 lines? That's miniscule.
semi-extrinsic 2y ago
I know many people who can and will one-shot a rewrite of 500 LOC. In my world, 500 LOC is about the length of a single function. I don't understand why we should be talking about generating a high level plan with multiple tests etc. for a single function. And I don't think this is uncommon. Just a random example from Github, this file is 1800 LOC and 4 functions. It implements one very specific thing that's part of a broader library. (I have no affiliation with this code.) https://github.com/elemental/Elemental/blob/master/src/optimization/solvers/SOCP/affine/IPM/Mehrotra.cpp https://github.com/elemental/Elemental/blob/master/src/optim...
- stavros 2y ago
  > I don't understand why we should be talking about generating a high level plan with multiple tests etc. for a single function. You don't have to, you can write it by hand. I thought we were talking about how we can make computers write code, instead of humans, but it seems that we're trying to prove that LLMs aren't useful instead.
  - SpaceNoodled 2y ago
    No, it's simply being demonstrated that they're not as useful as some claim.
    - stavros 2y ago
      By saying "why do I have to use a specific technique, instead of naively, to get what I want"?
      - SpaceNoodled 2y ago
        "Why do I have to put in more work to use this tool vs. not using it?"
        stavros 2y ago
        Which is exactly what I said here: https://news.ycombinator.com/item?id=43537443 https://news.ycombinator.com/item?id=43537443
chrismorgan 2y ago
> It's not like people just one-shot a whole module of code, why would LLMs? For conversions between languages or libraries, you often do just one-shot it, writing or modifying code from start to end in order. I remember 15 years ago taking a 10,000 line Java code base and porting it to JavaScript mostly like this, with only a few areas requiring a bit more involved and non-sequential editing.
- copperx 2y ago
  So, you didn't test it until the end? or did you have to build it in such a way that is was partially testable?
  - chrismorgan 2y ago
    One of the nifty things about the target being JavaScript was that I didn’t have to finish it before I could run it—it was the sort of big library where typical code wouldn’t use most of the functionality. It was audio stuff, so there were a couple of core files that needed more careful porting (from whatever in Java to Mozilla’s Audio Data API, which was a fairly good match), and then the rest was fairly routine that could be done gradually, as I needed them or just when I didn’t have anything better to focus on. Honestly, one of the biggest problems was forgetting to prefix instance properties with `this.`
- dietr1ch 2y ago
  I think this shows how the approach LLMs take is wrong. For us it's easy because we simply sort of iterate over every function with a simple prompt of doing a translation, but are yet careful enough taking notes of whatever may be relevant to do a higher level change if necessary. Maybe the mistake is mistaking LLMs as capable people instead of a simple, but optimised neuron soup tuned for text.
fbernier 2y ago
Isn't that basically the process some "thinking" models try to do for you under the hood? Prompting itself to improve your prompt. I actually have no idea but this is what I guessed it did when using it.
- stavros 2y ago
  They do some variant of this, but this is more directed. They might not do this on their own, they might follow other lines of reasoning. Maybe more effective, maybe less.