6 ms·

Yet it still codes like a junior developer that memorized all of stack overflow.

by dawnerd 5mo ago

dilap 5mo ago
Personally I don't find this to be true anymore! It's not always great and does still will often tend towards unneeded complexity (especially if not pushed a bit), but I often find GPT 5.5 writing code I would have written myself. This was very much not true with earlier models (who make something that worked, but I'd always have to rewrite to make it "good code").
- dawnerd 5mo ago
  Personally I found 5.5 a massive step back from 5.4. Both of them still use way too many fallbacks and unnecessary checks, especially if you're having it output php. It's fine if you're just one person and checking everything and able to catch and correct. But it's really bad when you have a team all using it, not checking the output and trusting it's output leading to spaghetti code. Technically works, but very messy and will no doubt lead to buggy code. It still writes like a junior dev, in that despite AI being able to get a picture of an entire repo, it's changes are typically confined to the task it's working on and will opt to duplicate logic to keep changes contained. Again, technically works, not ideal.
  - esikich 5mo ago
    Idk man, I think at this point, if you can't get good code out of frontier models, you're doing something wrong. Plenty of resources out there for you to familiarize yourself with the workflows if you can be bothered.
    - _zoltan_ 5mo ago
      100%. this is what I tell people who fail at using these models. the model isn't the problem anymore.
  - dilap 5mo ago
    Yeah, it has a tendency to default to "smallest local hack that will work" and code as defensively as possible. BUT I have had great success using AGENTS.md and becoming better at prompting to get it to not be like this. Basic approach in AGENTS.md: don't code defensively, yada yada, we have a validation layer at X, no need to check for anything behind that layer. Works well. An approach I've found helpful when prompting: What would be the best architecture for this change? If you say "do X" it'll tend to just do the hackiest, shortest path thing. If you say, "what's the best way to do X?" it will think more holistically. That said, who knows, maybe when it's PHP it just really wants to hack ;-) (Also, yes, you still need to review the code -- it will still do stupid things, so you can't just be pure hands off w/o ending up with quality degredations. The same is true of humans too though in my experience...)
raincole 5mo ago
PhDs code like that too. Especially if they're statisticians :)
zulban 5mo ago
Clearly you've never supervised junior developers.
- jldugger 5mo ago
  Or PhDs
- dawnerd 5mo ago
  That's literally my job...
  - Supermancho 5mo ago
    >That's literally my job... Since you’re not in a unique position, I can confidently state that your comparison of LLMs to jr developers seems unfounded. Today, LLMs produce code that is superior to junior developer code by an order of magnitude. Notably, they demonstrate consistent syntax, clear separation of concerns, strong test coverage, organizational rigor, idiomatic API usage, and the ability to generate and maintain documentation, among other measurable qualities. LLMs generally operate at a staff engineer level for a number of languages and ecosystems (including polyglot projects).
    - bigstrat2003 5mo ago
      LLMs absolutely do not exceed the abilities of junior devs. They don't even meet that bar, let alone exceed it. Junior devs are capable of getting syntax right without someone going "hey you messed that up". LLMs are not. Junior devs get basic logic right. LLMs do not. Comparing an LLM to a senior developer is an absolute joke.
      - no-name-here 5mo ago
        1. Which LLM are you using that is “not capable of getting syntax right”? 2. Are you referring to without having a compiler or LSP check it? Although even then, the recent LLMs I've used still frequently get syntax right, whereas I'd expect juniors are often using a LSP or compiler to catch mistakes while writing code?
bdamm 5mo ago
Even if the code was like that (it isn't), the power of the current crop of models to analyze data for patterns and build context out of code is leaps and bounds what it was even a year ago. And any developer will tell you that the hardest part of fixing a bug is knowing where the bug is in the first place. Once you know where it is, fixing it is usually trivial. There is serious magic happening in the construction of model context.
fourseventy 5mo ago
What is the last model you used... lol. Linus Torvalds himself said the newest models are better than him at coding.
- stemchar 5mo ago
  This doesn't sound correct. Source?
  - no-name-here 5mo ago
    In recent months, Linus said it specifically about code for a personal side project of his. The quote was in the commit message. (I’m not the grandparent commenter, and I think grandparent commenter’s claims may be too broad or require context.)
    - yrds96 5mo ago
      There are a missing the context: The vibecoded application was written in python while the main code was written manually in C by Torvalds in this side project. He never ever said that AI produces better code than him in the language where he is proficientI.
      - helloplanets 5mo ago
        https://github.com/torvalds/AudioNoise https://github.com/torvalds/AudioNoise > The python visualizer tool has been basically written by vibe-coding. I know more about analog filters -- and that's not saying much -- than I do about python. It started out as my typical "google and do the monkey-see-monkey-do" kind of programming, but then I cut out the middle-man -- me -- and just used Google Antigravity to do the audio sample visualizer.
_zoltan_ 5mo ago
this is not true at all. I'm using Opus and it's great at very complex problems.
RedCinnabar 5mo ago
Not true anymore since like early 2025 and especially since last December.

Yet it still codes like a junior developer that memorized all of stack overflow.

by dawnerd 5mo ago

Yet it still codes like a junior developer that memorized all of stack overflow.

dilap 5mo ago

Personally I don't find this to be true anymore! It's not always great and does still will often tend towards unneeded complexity (especially if not pushed a bit), but I often find GPT 5.5 writing code I would have written myself. This was very much not true with earlier models (who make something that worked, but I'd always have to rewrite to make it "good code").

dawnerd 5mo ago
Personally I found 5.5 a massive step back from 5.4. Both of them still use way too many fallbacks and unnecessary checks, especially if you're having it output php. It's fine if you're just one person and checking everything and able to catch and correct. But it's really bad when you have a team all using it, not checking the output and trusting it's output leading to spaghetti code. Technically works, but very messy and will no doubt lead to buggy code. It still writes like a junior dev, in that despite AI being able to get a picture of an entire repo, it's changes are typically confined to the task it's working on and will opt to duplicate logic to keep changes contained. Again, technically works, not ideal.
- esikich 5mo ago
  Idk man, I think at this point, if you can't get good code out of frontier models, you're doing something wrong. Plenty of resources out there for you to familiarize yourself with the workflows if you can be bothered.
  - _zoltan_ 5mo ago
    100%. this is what I tell people who fail at using these models. the model isn't the problem anymore.
- dilap 5mo ago
  Yeah, it has a tendency to default to "smallest local hack that will work" and code as defensively as possible. BUT I have had great success using AGENTS.md and becoming better at prompting to get it to not be like this. Basic approach in AGENTS.md: don't code defensively, yada yada, we have a validation layer at X, no need to check for anything behind that layer. Works well. An approach I've found helpful when prompting: What would be the best architecture for this change? If you say "do X" it'll tend to just do the hackiest, shortest path thing. If you say, "what's the best way to do X?" it will think more holistically. That said, who knows, maybe when it's PHP it just really wants to hack ;-) (Also, yes, you still need to review the code -- it will still do stupid things, so you can't just be pure hands off w/o ending up with quality degredations. The same is true of humans too though in my experience...)

raincole 5mo ago

PhDs code like that too. Especially if they're statisticians :)

zulban 5mo ago

Clearly you've never supervised junior developers.

jldugger 5mo ago
Or PhDs
dawnerd 5mo ago
That's literally my job...
- Supermancho 5mo ago
  >That's literally my job... Since you’re not in a unique position, I can confidently state that your comparison of LLMs to jr developers seems unfounded. Today, LLMs produce code that is superior to junior developer code by an order of magnitude. Notably, they demonstrate consistent syntax, clear separation of concerns, strong test coverage, organizational rigor, idiomatic API usage, and the ability to generate and maintain documentation, among other measurable qualities. LLMs generally operate at a staff engineer level for a number of languages and ecosystems (including polyglot projects).
  - bigstrat2003 5mo ago
    LLMs absolutely do not exceed the abilities of junior devs. They don't even meet that bar, let alone exceed it. Junior devs are capable of getting syntax right without someone going "hey you messed that up". LLMs are not. Junior devs get basic logic right. LLMs do not. Comparing an LLM to a senior developer is an absolute joke.
    - no-name-here 5mo ago
      1. Which LLM are you using that is “not capable of getting syntax right”? 2. Are you referring to without having a compiler or LSP check it? Although even then, the recent LLMs I've used still frequently get syntax right, whereas I'd expect juniors are often using a LSP or compiler to catch mistakes while writing code?

bdamm 5mo ago

Even if the code was like that (it isn't), the power of the current crop of models to analyze data for patterns and build context out of code is leaps and bounds what it was even a year ago. And any developer will tell you that the hardest part of fixing a bug is knowing where the bug is in the first place. Once you know where it is, fixing it is usually trivial. There is serious magic happening in the construction of model context.

fourseventy 5mo ago

What is the last model you used... lol. Linus Torvalds himself said the newest models are better than him at coding.

stemchar 5mo ago
This doesn't sound correct. Source?
- no-name-here 5mo ago
  In recent months, Linus said it specifically about code for a personal side project of his. The quote was in the commit message. (I’m not the grandparent commenter, and I think grandparent commenter’s claims may be too broad or require context.)
  - yrds96 5mo ago
    There are a missing the context: The vibecoded application was written in python while the main code was written manually in C by Torvalds in this side project. He never ever said that AI produces better code than him in the language where he is proficientI.
    - helloplanets 5mo ago
      https://github.com/torvalds/AudioNoise https://github.com/torvalds/AudioNoise > The python visualizer tool has been basically written by vibe-coding. I know more about analog filters -- and that's not saying much -- than I do about python. It started out as my typical "google and do the monkey-see-monkey-do" kind of programming, but then I cut out the middle-man -- me -- and just used Google Antigravity to do the audio sample visualizer.

_zoltan_ 5mo ago

this is not true at all. I'm using Opus and it's great at very complex problems.

RedCinnabar 5mo ago

Not true anymore since like early 2025 and especially since last December.