3 ms·

It is pretty good at coding; I have 40+ years of hobby and professional experience of coding in dozens of languages, from small projects to millions of lines a

It is pretty good at coding; I have 40+ years of hobby and professional experience of coding in dozens of languages, from small projects to millions of lines architected and (co) written by me. There is software I wrote begin 90s still running production etc. I am a cto now but I still write code for my job about half time and in my free time as well, that is to say, I was until recently; now I have the tooling (made myself as I didn't like what there was at the time) to not code at all and yet I produce more code than ever. We use PRs and reviews and my colleagues are quite surprised about the speed and the quality. So yeah, not bad for sure even in my field of expertise. We are now rolling out my tooling in the company so everyone can forget about the boring stuff and just focus on the business logic. There is resistance as this is going to cost jobs; we don't have infinite work to do and this is much (much) faster.

b20000 2y ago
the last 1% of a project takes as much time as the first 99% of a project
- anonzzzies 2y ago
  And this is related to my comment how?
  - skydhash 2y ago
    The code is the mean, not the end. And you take care in it, because other people will work with it to handle changing requirements. One aspect of seniority is to know the tools to solve a problem, the other is to make the solution practical to maintain and adaptive. And this takes as much time as solving the problem.
    - anonzzzies 2y ago
      I agree, still not really related to a thread that is about llms writing code. The llm and code are tools to address stuff my business deals with; I didn't say it does everything; we were just talking about the code part.
bitwizeshift 2y ago
This hasn’t been my experience at all in the slightest. Been programming since I was in elementary school, and current Copilot, OpenAI and even Gemini models generate code at a very very junior level. It might solve a practical problem, but it can’t write a decent abstraction to save its life unless you repeatedly prompt it to. It also massively struggles to retain coherence when it has more moving parts; if you have different things being mutated, it often just forgets it and will write code that crashes/panics/generates UB/etc. When you are lucky and you get something that vaguely works, the test cases it writes are of negative value. Test cases are either useless cases that don’t cover edge cases, are incorrect entirely and fail, or worse yet — look correct and pass, but are semantically wrong. LLM models have been absolutely hilariously bad at this, where it will generate passing cases for the code as written, but not for the semantics of the code being written. Writing it by hand would catch it quickly, but a junior dev using these tools can easily miss this. Then there is Rust; most models don’t do rust well. In isolation they are kind of okay, but overall it frequently generates borrowing issues that fail to compile.
- anonzzzies 2y ago
  But I guess, and this is dangerous to say I do realize, is that the tooling around the prompts and around the results is key to getting the best results. Just prompts without guards is not how you want to do it.

b20000 2y ago

the last 1% of a project takes as much time as the first 99% of a project

anonzzzies 2y ago
And this is related to my comment how?
- skydhash 2y ago
  The code is the mean, not the end. And you take care in it, because other people will work with it to handle changing requirements. One aspect of seniority is to know the tools to solve a problem, the other is to make the solution practical to maintain and adaptive. And this takes as much time as solving the problem.
  - anonzzzies 2y ago
    I agree, still not really related to a thread that is about llms writing code. The llm and code are tools to address stuff my business deals with; I didn't say it does everything; we were just talking about the code part.

bitwizeshift 2y ago

This hasn’t been my experience at all in the slightest. Been programming since I was in elementary school, and current Copilot, OpenAI and even Gemini models generate code at a very very junior level. It might solve a practical problem, but it can’t write a decent abstraction to save its life unless you repeatedly prompt it to. It also massively struggles to retain coherence when it has more moving parts; if you have different things being mutated, it often just forgets it and will write code that crashes/panics/generates UB/etc. When you are lucky and you get something that vaguely works, the test cases it writes are of negative value. Test cases are either useless cases that don’t cover edge cases, are incorrect entirely and fail, or worse yet — look correct and pass, but are semantically wrong. LLM models have been absolutely hilariously bad at this, where it will generate passing cases for the code as written, but not for the semantics of the code being written. Writing it by hand would catch it quickly, but a junior dev using these tools can easily miss this. Then there is Rust; most models don’t do rust well. In isolation they are kind of okay, but overall it frequently generates borrowing issues that fail to compile.

anonzzzies 2y ago
But I guess, and this is dangerous to say I do realize, is that the tooling around the prompts and around the results is key to getting the best results. Just prompts without guards is not how you want to do it.