6 ms·

It needs to be said that your opinion on this is well understood by the community, respected, but also far from impartial. You have a clear vested interest in t

It needs to be said that your opinion on this is well understood by the community, respected, but also far from impartial. You have a clear vested interest in the success of _these_ tools. There's a learning curve to any toolset, and it may be that using coding agents effectively is more than a few weeks of upskilling. It may be, and likely will be, that people make their whole careers about being experts on this topic. But it's still a statistical text prediction model, wrapped in fancy gimmicks, sold at a loss by mostly bad faith actors, and very far from its final form. People waiting to get on the bandwagon could well be waiting to pick up the pieces once it collapses.

mattmanser 9mo ago
I have a lot of respect from Simon and read a lot of his articles. But I'm still seeing clear evidence it IS a statistical text prediction model. You ask it the right niche thing and it can only pump out a few variations of the same code, that's clearly someone else's code stolen almost verbatim. And I just use it 2 or 3 times a day. How are SimonW and AntiRez not seeing the same thing? How are they not seeing the propensity for both Claude + ChatGPT to spit out tons of completely pointless error handling code, making what should be a 5 line function a 50 line one? How are they not seeing that you constantly have to nag it to use modern syntax. Typescript, C#, Python, doesn't matter what you're writing in, it will regularly spit out code patterns that are 10 years out of date. And woe betide you using a library that got updated in the last 2 years. It will constantly revert back to old syntax over and over and over again. I've also had to deal with a few of my colleagues using AI code on codebases they don't really understand. Wrong sort, id instead of timestamp. Wrong limit. Wrong json encoding, missing key converters. Wrong timezone on dates. A ton of subtle, not obvious, bugs unless you intimately know the code, but would be things you'd look up if you were writing the code. And that's not even including the bit where the AI obviously decided to edit the wrong search function in a totally different part of the codebase that had nothing to do with what my colleague was doing. But didn't break anything or trigger any tests because it was wrapped in an impossible to hit if clause. And it created a bunch of extra classes to support this phantom code, so hundreds of new lines of code just lurking there, not doing anything but if I hadn't caught it, everyone thinks it does do something.
- jimmaswell 9mo ago
  > You ask it the right niche thing and it can only pump out a few variations of the same code, that's clearly someone else's code stolen almost verbatim. There are only so many ways to express the same idea. Even clean room engineers write incidentally identical code to the source sometimes.
  - mattmanser 9mo ago
    There was an example on here recently where an AI PR to an open source literally had someone else's name in the comments in the code, and included their license. That's the level of tell-tale that's its just stealing code and modifying a couple of variable names. For me personally, the code I've seen might be written in a slightly weird style, or have strange, not applicable to the question, additions. They're so obviously not "clean room" code or incredibly generic, they're the opposite, they're incredibly specific.
- MaybiusStrip 9mo ago
  You can literally go look at some of antirez's PRs described here in this article. They're not seeing it because it's not there? Honestly, what you're describing sounds like the older models. If you are getting these sorts of results with Opus 4.5 or 5.2-codex on high I would be very curious to see your prompts/workflow.
  - suddenlybananas 9mo ago
    People have been saying "Oh use glorp 3.835 and those problems don't happen anymore" for about 3 years at this point. It's always the fact you're not using the latest model that's the problem.
    - pedeypops 9mo ago
      I agree. I've seen people insist moving to a newer model or fine tuning will make the output more clever, "trust me", sometimes without providing any evidence of before and after for the specific use case. One LLM project I saw released was prettymuch useless, but it wasn't the use case or the architectural limitations that were the problem, nope the next thing on the roadmap was "fixing" it by plugging in a better LLM.
johnfn 9mo ago
How does he have a vested interest in the success of these tools? He doesn't work for an AI company. Why must he have some shady ulterior motive rather than just honestly believing the thing they are stated? Yes, he blogs a lot about AI, but don't you have the cart profoundly before the horse if you are asserting that's a "vested interest"? He was free to blog about whatever he wants. Why would he fervently start blogging about AI if he didn't earnestly believe it was an interesting topic to blog about? > But it's still a statistical text prediction model This is reductive to the point of absurdity. What other statistical text prediction model can make tool calls to CLI apps and web searches? It's like saying "a computer is nothing special -- it's just a bunch of wires stuck together"
- Humorist2290 9mo ago
  > Why must he have some shady ulterior motive rather than just honestly believing the thing they are are stated? I wouldn't say it's shady or even untoward. Simon writes prolifically and he seems quite genuinely interested in this. That he has attached his public persona, and what seems like basically all of his time from the last few years, to LLMs and their derivatives is still a vested interest. I wouldn't even say that's bad. Passion about technology is what drives many of us. But it still needs saying. > This is reductive to the point of absurdity. What other statistical text prediction model can make tool calls to CLI apps and web searches? It's just a fact that these things are statistical text prediction models. Sure, they're marvels, but they're not deterministic, nor are they reliable. They are like a slot machine with surprisingly good odds: pull the lever and you're almost guaranteed to get something, maybe a jackpot, maybe you'll lose those tokens. For many people it's cheap enough to just keep pulling the lever until they get what they want, or go bankrupt.

mattmanser 9mo ago

I have a lot of respect from Simon and read a lot of his articles. But I'm still seeing clear evidence it IS a statistical text prediction model. You ask it the right niche thing and it can only pump out a few variations of the same code, that's clearly someone else's code stolen almost verbatim. And I just use it 2 or 3 times a day. How are SimonW and AntiRez not seeing the same thing? How are they not seeing the propensity for both Claude + ChatGPT to spit out tons of completely pointless error handling code, making what should be a 5 line function a 50 line one? How are they not seeing that you constantly have to nag it to use modern syntax. Typescript, C#, Python, doesn't matter what you're writing in, it will regularly spit out code patterns that are 10 years out of date. And woe betide you using a library that got updated in the last 2 years. It will constantly revert back to old syntax over and over and over again. I've also had to deal with a few of my colleagues using AI code on codebases they don't really understand. Wrong sort, id instead of timestamp. Wrong limit. Wrong json encoding, missing key converters. Wrong timezone on dates. A ton of subtle, not obvious, bugs unless you intimately know the code, but would be things you'd look up if you were writing the code. And that's not even including the bit where the AI obviously decided to edit the wrong search function in a totally different part of the codebase that had nothing to do with what my colleague was doing. But didn't break anything or trigger any tests because it was wrapped in an impossible to hit if clause. And it created a bunch of extra classes to support this phantom code, so hundreds of new lines of code just lurking there, not doing anything but if I hadn't caught it, everyone thinks it does do something.

jimmaswell 9mo ago
> You ask it the right niche thing and it can only pump out a few variations of the same code, that's clearly someone else's code stolen almost verbatim. There are only so many ways to express the same idea. Even clean room engineers write incidentally identical code to the source sometimes.
- mattmanser 9mo ago
  There was an example on here recently where an AI PR to an open source literally had someone else's name in the comments in the code, and included their license. That's the level of tell-tale that's its just stealing code and modifying a couple of variable names. For me personally, the code I've seen might be written in a slightly weird style, or have strange, not applicable to the question, additions. They're so obviously not "clean room" code or incredibly generic, they're the opposite, they're incredibly specific.
MaybiusStrip 9mo ago
You can literally go look at some of antirez's PRs described here in this article. They're not seeing it because it's not there? Honestly, what you're describing sounds like the older models. If you are getting these sorts of results with Opus 4.5 or 5.2-codex on high I would be very curious to see your prompts/workflow.
- suddenlybananas 9mo ago
  People have been saying "Oh use glorp 3.835 and those problems don't happen anymore" for about 3 years at this point. It's always the fact you're not using the latest model that's the problem.
  - pedeypops 9mo ago
    I agree. I've seen people insist moving to a newer model or fine tuning will make the output more clever, "trust me", sometimes without providing any evidence of before and after for the specific use case. One LLM project I saw released was prettymuch useless, but it wasn't the use case or the architectural limitations that were the problem, nope the next thing on the roadmap was "fixing" it by plugging in a better LLM.

johnfn 9mo ago

How does he have a vested interest in the success of these tools? He doesn't work for an AI company. Why must he have some shady ulterior motive rather than just honestly believing the thing they are stated? Yes, he blogs a lot about AI, but don't you have the cart profoundly before the horse if you are asserting that's a "vested interest"? He was free to blog about whatever he wants. Why would he fervently start blogging about AI if he didn't earnestly believe it was an interesting topic to blog about? > But it's still a statistical text prediction model This is reductive to the point of absurdity. What other statistical text prediction model can make tool calls to CLI apps and web searches? It's like saying "a computer is nothing special -- it's just a bunch of wires stuck together"

Humorist2290 9mo ago
> Why must he have some shady ulterior motive rather than just honestly believing the thing they are are stated? I wouldn't say it's shady or even untoward. Simon writes prolifically and he seems quite genuinely interested in this. That he has attached his public persona, and what seems like basically all of his time from the last few years, to LLMs and their derivatives is still a vested interest. I wouldn't even say that's bad. Passion about technology is what drives many of us. But it still needs saying. > This is reductive to the point of absurdity. What other statistical text prediction model can make tool calls to CLI apps and web searches? It's just a fact that these things are statistical text prediction models. Sure, they're marvels, but they're not deterministic, nor are they reliable. They are like a slot machine with surprisingly good odds: pull the lever and you're almost guaranteed to get something, maybe a jackpot, maybe you'll lose those tokens. For many people it's cheap enough to just keep pulling the lever until they get what they want, or go bankrupt.