5 ms·

Earlier this past week I asked Copilot to generate some Golang tests and it used some obscure assertion library that had a few hundred stars on GitHub. I had to

Earlier this past week I asked Copilot to generate some Golang tests and it used some obscure assertion library that had a few hundred stars on GitHub. I had to explicitly ask it to generate idiomatic tests and even then it still didn't test all of the parameters that it should have. At a previous job I made the mistake of letting it write some repository methods that leveraged SQLAlchemy. Even though I (along with my colleague via PR) reviewed the generated code we ended up with a preprod bug because the LLM used session.flush() instead of session.commit() in exactly one spot for no apparent reason. LLMs are still not ready for prime-time. They churn out code like an overconfident 25-year-old that just downed three lunch beers with a wet burrito at the Mexican place down the street from the office on a rainy Wednesday.

csense 2y ago
> an overconfident 25-year-old that just downed three lunch beers with a wet burrito at the Mexican place down the street from the office on a rainy Wednesday That's...oddly specific.
- alexjplant 2y ago
  I'm only 33 and I've worked with at least two of 'em. They're a type :-D
  - 1attice 2y ago
    No this all tracks. Matches my own experience.
    - deleted 2y ago
      [deleted]
  - blobbers 2y ago
    You were one of them... 8 years ago?
    - eru 2y ago
      (Not the OP.) The LLMs are much more eager to please and to write lots of code. When I was younger, I would get distracted and play computer games (or comment on HN..), rather than churn out mountains of mediocre code all the time.
      - sdesol 2y ago
        > The LLMs are much more eager to please and to write lots of code. My process right now when working LLMs is to do the following: - Create problem and solution statement - Create requirements and user stories - Create architecture - Create skeleton code - Map the skeleton code - Create the full code At every step, where I don't need the full code, the LLM will start coding and I need to stop it and add "Do not generate code. The focus is still design".
        code_for_monkey 2y ago
        one of my biggest issues with the LLM is how it always wants to give me a mountain of code. A lot of the time Im using it for react, and it always gives me a full component no matter how much I specify I just want the method. It will not remember this for more than one message and will go back to giving me as much code as possible.
        sdesol 2y ago
        Yeah, it almost feels like you are talking to somebody with OCD. The frustrating part is, output tokens are usually a lot more expensive than input tokens, so they are wasting energy and money :-). Also, the more they generate, the greater the chance it will create attention issues as the conversation progresses. This is why I built my chat app to let me manipulate LLM responses. If I feel it is not worth knowing, I'll just erase parts of it to ensure the conversation doesn't get side tracked. Or I will go back to the original user message and modify it to say ### IMPORTANT - Do not generate more code than required. The nice thing about LLM conversations are, every time you chat, the LLM treats it as a first time conversation, so this trick will work if the model is smart enough.
- hyperdimension 2y ago
  I thought it was nicely poetic.
DustinBrett 2y ago
I feel like I am taking crazy pills that other devs don't feel this way. How bad are the coders that they think these AI's are giving them super powers. The PR's with AI code are so obvious and when you ask the devs why, they don't even know. They just say, well the AI picked this, as if that means something in and of itself.
- trevor-e 2y ago
  Because there are plenty of devs who take the output, actually read if it makes sense, do a code review, iterate back and forth with it a few times, and then finally check in the result. It's just a tool. Shitty devs will make shitty code regardless of the tool. And good devs are insanely productive with good tools. In your example what's the difference with that dev just copy/pasting from StackOverflow?
  - eru 2y ago
    Agreed! To go off on a tangent: yes, good developers can produce good code faster. And bad developers can produce bad code faster (and perhaps slightly better than before, because the models are mostly a bit smarter than copy-and-paste is). Everyone potentially benefits, but it won't suddenly turn all bad programmers into great programmers.
  - DustinBrett 2y ago
    Because on SO someone real wrote it with a real reason and logic. With AI we still need to double check that what we are giving ever made any sense. And SO also has votes to show the validty. I agree if devs iterated over the results it could be good, but that has not been what I have been seeing. It is not a traditional tool because tools we had in the past had expected results.
- doix 2y ago
  AI gives super powers because it saves you an insane amount of typing. I used to be a vim fanatic, I was very efficient but whenever I changed language there was a period where I had to spend getting efficient. Setup some new snippets for boilerplate, maybe tweak some LSP settings, save some new macros. Now in cursor I just write "extract this block of code into its own function and set up the unit tests" and it does it, with no configuration from my part. Before I'd have a snippet for the unit test boilerplate for that specific project, I'd have to figure out the mocks mysel, etc. Yes, if you use AI to just generate new code blindly and check it in without no understanding, you end up with garbage. But those people were most likely copy pasting from SO before AI, AI just made them faster.
fenomas 2y ago
I don't follow this take. ChatGPT outputted a bug subtle enough to be overlooked by you and your colleague and your test suite, and that means it's not ready for prime time? The day when generative AI might hope to completely handle a coding task isn't here yet - it doesn't know your full requirements, it can't run your integration tests, etc. For now it's a tool, like a linter or a debugger - useful sometimes and not useful other times, but the responsibility to keep bugs out of prod still rests with the coder, not the tools.
- Sammi 2y ago
  Yes and this means it doesn't replace anyone or make someone who isn't able to code able to code. It just means it's a tool for people who already know how to code.
fooker 2y ago
You are using it wrong. Give examples and let it extrapolate.
- lionkor 2y ago
  "You're holding it wrong" doesn't make a small, too light, crooked, and backwards hammer any better.
bboygravity 2y ago
You can't tell us that LLM's aren't ready for prime time in 2025 after you tried Copilot twice last year. New better models are coming out almost daily now and it's almost common knowledge that Copilot was and is one of the worst. Especially right now, it doesn't even come close to what better models have to offer. Also the way to use them is to ask for small chunks of code or questions about code after you gave them tons of context (like in Claude projects for example). "Not ready for prime time" is also just factually incorrect. It is already being used A LOT. To the point that there are rumors that Cursor is buying so much compute from Anthropic that they are making their product unstable, because nvidia can't supply them hardware fast enough.
- KeplerBoy 2y ago
  Copilot isn't a single model. Copilot is merely a brand and uses openAI and anthropics newest models.
- consp 2y ago
  > It is already being used A LOT Which is an argument for quality why? Bad coders are not going to produce better code that way. Just more with less effort.
  - bboygravity 2y ago
    The claim of the comment I replied to was "LLM's are not ready for prime time" and my opinion is that LLM prime time is already here. Using LLM's to code (or to learn how to code) is obviously super popular. Who's talking about quality anyway? Code quality is not and was never the number one most important thing in business, popularity of a product/service or keeping a job. You may find that unfortunate (I agree), but it's just how it is based on my own 15yr+ experience.
- 59nadir 2y ago
  I stopped using AI for code a little over a year ago and at that point I'd used Copilot for 8-12 months. I tried Cursor out a couple of weeks ago for very small autocomplete snippets and it was about the same or slightly worse than Copilot, in my opinion. The integration with the editor was neat but the quality of the suggestions were no different than what I'd had with Copilot much earlier, and the pathological cases where it just spun off into some useless corner of its behavior (recommending code that was already in the very same file, recommending code that didn't make any sense, etc.) seemed to happen more than with Copilot. This was a ridiculously simple project for it to work on, to be clear, just a parser for a language I started working on, and the general structure was already there for it to work with when I started trying Cursor out. From prior experience I know the base is pretty easy to work with for people who aren't even familiar with it (or even parsing in general), so I think given the difficulties that Cursor had even putting together pretty basic things it might be that a user of Cursor would see minimal gains in velocity and end up having less understanding in the medium to long term, at least in this particular case.
Seattle3503 2y ago
> At a previous job I made the mistake of letting it write some repository methods that leveraged SQLAlchemy. Even though I (along with my colleague via PR) reviewed the generated code we ended up with a preprod bug because the LLM used session.flush() instead of session.commit() in exactly one spot for no apparent reason. Ive had ChatGPT do the same thing with code involving SQLAlchemy.
Cthulhu_ 2y ago
Is that the LLM's fault or SQLAlchemy for having that API in the first place? Or was that a gap in your testing strategy, as (if I'm reading it right), flush() doesn't write anything to the database but is only intended as an intermediate step (and commit() calls flush() under water). I think we're in a period similar to self-driving cars, where the LLMs are pretty good, but not perfect; it's those last few percent that break it.
linsomniac 2y ago
>it used some obscure assertion library that had a few hundred stars on GitHub. That sounds like a lot of developers I've worked with.

csense 2y ago

> an overconfident 25-year-old that just downed three lunch beers with a wet burrito at the Mexican place down the street from the office on a rainy Wednesday That's...oddly specific.

alexjplant 2y ago
I'm only 33 and I've worked with at least two of 'em. They're a type :-D
- 1attice 2y ago
  No this all tracks. Matches my own experience.
  - deleted 2y ago
    [deleted]
- blobbers 2y ago
  You were one of them... 8 years ago?
  - eru 2y ago
    (Not the OP.) The LLMs are much more eager to please and to write lots of code. When I was younger, I would get distracted and play computer games (or comment on HN..), rather than churn out mountains of mediocre code all the time.
    - sdesol 2y ago
      > The LLMs are much more eager to please and to write lots of code. My process right now when working LLMs is to do the following: - Create problem and solution statement - Create requirements and user stories - Create architecture - Create skeleton code - Map the skeleton code - Create the full code At every step, where I don't need the full code, the LLM will start coding and I need to stop it and add "Do not generate code. The focus is still design".
      - code_for_monkey 2y ago
        one of my biggest issues with the LLM is how it always wants to give me a mountain of code. A lot of the time Im using it for react, and it always gives me a full component no matter how much I specify I just want the method. It will not remember this for more than one message and will go back to giving me as much code as possible.
        sdesol 2y ago
        Yeah, it almost feels like you are talking to somebody with OCD. The frustrating part is, output tokens are usually a lot more expensive than input tokens, so they are wasting energy and money :-). Also, the more they generate, the greater the chance it will create attention issues as the conversation progresses. This is why I built my chat app to let me manipulate LLM responses. If I feel it is not worth knowing, I'll just erase parts of it to ensure the conversation doesn't get side tracked. Or I will go back to the original user message and modify it to say ### IMPORTANT - Do not generate more code than required. The nice thing about LLM conversations are, every time you chat, the LLM treats it as a first time conversation, so this trick will work if the model is smart enough.
hyperdimension 2y ago
I thought it was nicely poetic.

DustinBrett 2y ago

I feel like I am taking crazy pills that other devs don't feel this way. How bad are the coders that they think these AI's are giving them super powers. The PR's with AI code are so obvious and when you ask the devs why, they don't even know. They just say, well the AI picked this, as if that means something in and of itself.

trevor-e 2y ago
Because there are plenty of devs who take the output, actually read if it makes sense, do a code review, iterate back and forth with it a few times, and then finally check in the result. It's just a tool. Shitty devs will make shitty code regardless of the tool. And good devs are insanely productive with good tools. In your example what's the difference with that dev just copy/pasting from StackOverflow?
- eru 2y ago
  Agreed! To go off on a tangent: yes, good developers can produce good code faster. And bad developers can produce bad code faster (and perhaps slightly better than before, because the models are mostly a bit smarter than copy-and-paste is). Everyone potentially benefits, but it won't suddenly turn all bad programmers into great programmers.
- DustinBrett 2y ago
  Because on SO someone real wrote it with a real reason and logic. With AI we still need to double check that what we are giving ever made any sense. And SO also has votes to show the validty. I agree if devs iterated over the results it could be good, but that has not been what I have been seeing. It is not a traditional tool because tools we had in the past had expected results.
doix 2y ago
AI gives super powers because it saves you an insane amount of typing. I used to be a vim fanatic, I was very efficient but whenever I changed language there was a period where I had to spend getting efficient. Setup some new snippets for boilerplate, maybe tweak some LSP settings, save some new macros. Now in cursor I just write "extract this block of code into its own function and set up the unit tests" and it does it, with no configuration from my part. Before I'd have a snippet for the unit test boilerplate for that specific project, I'd have to figure out the mocks mysel, etc. Yes, if you use AI to just generate new code blindly and check it in without no understanding, you end up with garbage. But those people were most likely copy pasting from SO before AI, AI just made them faster.

fenomas 2y ago

I don't follow this take. ChatGPT outputted a bug subtle enough to be overlooked by you and your colleague and your test suite, and that means it's not ready for prime time? The day when generative AI might hope to completely handle a coding task isn't here yet - it doesn't know your full requirements, it can't run your integration tests, etc. For now it's a tool, like a linter or a debugger - useful sometimes and not useful other times, but the responsibility to keep bugs out of prod still rests with the coder, not the tools.

Sammi 2y ago
Yes and this means it doesn't replace anyone or make someone who isn't able to code able to code. It just means it's a tool for people who already know how to code.

fooker 2y ago

You are using it wrong. Give examples and let it extrapolate.

lionkor 2y ago
"You're holding it wrong" doesn't make a small, too light, crooked, and backwards hammer any better.

bboygravity 2y ago

You can't tell us that LLM's aren't ready for prime time in 2025 after you tried Copilot twice last year. New better models are coming out almost daily now and it's almost common knowledge that Copilot was and is one of the worst. Especially right now, it doesn't even come close to what better models have to offer. Also the way to use them is to ask for small chunks of code or questions about code after you gave them tons of context (like in Claude projects for example). "Not ready for prime time" is also just factually incorrect. It is already being used A LOT. To the point that there are rumors that Cursor is buying so much compute from Anthropic that they are making their product unstable, because nvidia can't supply them hardware fast enough.

KeplerBoy 2y ago
Copilot isn't a single model. Copilot is merely a brand and uses openAI and anthropics newest models.
consp 2y ago
> It is already being used A LOT Which is an argument for quality why? Bad coders are not going to produce better code that way. Just more with less effort.
- bboygravity 2y ago
  The claim of the comment I replied to was "LLM's are not ready for prime time" and my opinion is that LLM prime time is already here. Using LLM's to code (or to learn how to code) is obviously super popular. Who's talking about quality anyway? Code quality is not and was never the number one most important thing in business, popularity of a product/service or keeping a job. You may find that unfortunate (I agree), but it's just how it is based on my own 15yr+ experience.
59nadir 2y ago
I stopped using AI for code a little over a year ago and at that point I'd used Copilot for 8-12 months. I tried Cursor out a couple of weeks ago for very small autocomplete snippets and it was about the same or slightly worse than Copilot, in my opinion. The integration with the editor was neat but the quality of the suggestions were no different than what I'd had with Copilot much earlier, and the pathological cases where it just spun off into some useless corner of its behavior (recommending code that was already in the very same file, recommending code that didn't make any sense, etc.) seemed to happen more than with Copilot. This was a ridiculously simple project for it to work on, to be clear, just a parser for a language I started working on, and the general structure was already there for it to work with when I started trying Cursor out. From prior experience I know the base is pretty easy to work with for people who aren't even familiar with it (or even parsing in general), so I think given the difficulties that Cursor had even putting together pretty basic things it might be that a user of Cursor would see minimal gains in velocity and end up having less understanding in the medium to long term, at least in this particular case.

Seattle3503 2y ago

> At a previous job I made the mistake of letting it write some repository methods that leveraged SQLAlchemy. Even though I (along with my colleague via PR) reviewed the generated code we ended up with a preprod bug because the LLM used session.flush() instead of session.commit() in exactly one spot for no apparent reason. Ive had ChatGPT do the same thing with code involving SQLAlchemy.

Cthulhu_ 2y ago

Is that the LLM's fault or SQLAlchemy for having that API in the first place? Or was that a gap in your testing strategy, as (if I'm reading it right), flush() doesn't write anything to the database but is only intended as an intermediate step (and commit() calls flush() under water). I think we're in a period similar to self-driving cars, where the LLMs are pretty good, but not perfect; it's those last few percent that break it.

linsomniac 2y ago

>it used some obscure assertion library that had a few hundred stars on GitHub. That sounds like a lot of developers I've worked with.