2 ms·

I've honestly never had that success. LLMs for me more often get in the way than actually help anything. It may be that it's because I deal mostly in very lar

I've honestly never had that success. LLMs for me more often get in the way than actually help anything. It may be that it's because I deal mostly in very large code bases with their own nuances.

eatsyourtacos 2y ago
Counter point though- what if it was trained on your specific code base? Wouldn't it be able to then help with those given nuances? The code base I have I would love to be able to just give some AI free reign and learn the structure since a lot of it is fairly repetitive; I know it would be so easy to say "hey add X just like Y" and it would be able to do it easily.
- throw-qqqqq 2y ago
  The rare times where I have no good idea what to do, it is faster to code with an LLM. The rest of the time, when I know just what I want, it takes longer for me to formulate it, query the LLM and discuss and validate its output. My more experienced/senior colleagues all say roughly the same. It’s great help for our juniors though. They learn a lot and are more capable on their own with the AI assistance. It’s improving all the time though, so I’m not writing it off at all. I am developing an evaluation suite so I can keep watching the progress in a systematic way..
  - taylodl 2y ago
    > I am developing an evaluation suite so I can keep watching the progress in a systematic way.. Sounds like something that should be published on github
    - pona-a 2y ago
      Open benchmarks are vulnerable to saturation. I think benchmarks should have an embargo periodic, until which only 3% of the question-answer pairs is released, with an explicit warning not to use it 3 months after being released.
- AndreasMoeller 2y ago
  I think there are types of problems that AI will be great at solving. If you can pass in a whole codebase then we might have LLMs that can suggest refactoring etc to improve code quality. Code mods like upgrading from python 2 -> 3 could also become possible.
taylodl 2y ago
Works much better on greenfield projects. Brownfield requires the LLM to have ingested your codebase and all the libraries you're using. That's why tools such as DeepSeek are exciting because that potentially opens up the door to much better brownfield development - which is where the real value lies!

I've honestly never had that success. LLMs for me more often get in the way than actually help anything. It may be that it's because I deal mostly in very large code bases with their own nuances.

eatsyourtacos 2y ago

Counter point though- what if it was trained on your specific code base? Wouldn't it be able to then help with those given nuances? The code base I have I would love to be able to just give some AI free reign and learn the structure since a lot of it is fairly repetitive; I know it would be so easy to say "hey add X just like Y" and it would be able to do it easily.

throw-qqqqq 2y ago
The rare times where I have no good idea what to do, it is faster to code with an LLM. The rest of the time, when I know just what I want, it takes longer for me to formulate it, query the LLM and discuss and validate its output. My more experienced/senior colleagues all say roughly the same. It’s great help for our juniors though. They learn a lot and are more capable on their own with the AI assistance. It’s improving all the time though, so I’m not writing it off at all. I am developing an evaluation suite so I can keep watching the progress in a systematic way..
- taylodl 2y ago
  > I am developing an evaluation suite so I can keep watching the progress in a systematic way.. Sounds like something that should be published on github
  - pona-a 2y ago
    Open benchmarks are vulnerable to saturation. I think benchmarks should have an embargo periodic, until which only 3% of the question-answer pairs is released, with an explicit warning not to use it 3 months after being released.
AndreasMoeller 2y ago
I think there are types of problems that AI will be great at solving. If you can pass in a whole codebase then we might have LLMs that can suggest refactoring etc to improve code quality. Code mods like upgrading from python 2 -> 3 could also become possible.

taylodl 2y ago

Works much better on greenfield projects. Brownfield requires the LLM to have ingested your codebase and all the libraries you're using. That's why tools such as DeepSeek are exciting because that potentially opens up the door to much better brownfield development - which is where the real value lies!