5 ms·

Feels like with every announcement there’s the same comment: “this LLM tool I’m using now is the real deal, the thing I was using previously and spending stupid

by latexr 1y ago

Feels like with every announcement there’s the same comment: “this LLM tool I’m using now is the real deal, the thing I was using previously and spending stupid amounts of money on looked good but failed at XYZ, this new thing is where it’s at”. Rinse and repeat. Which means it wasn’t true any of the previous times, so why would it be true this time? It feels like an endless loop of the “friendship ended” meme with AI companies. https://knowyourmeme.com/editorials/guides/what-is-the-friendship-ended-with-mudasir-meme-and-why-did-the-friendship-end-the-viral-meme-explained https://knowyourmeme.com/editorials/guides/what-is-the-frien... It’s much more likely commenters are still in the honeymoon hype phase and (again) haven’t found the problems because they’re hyper focused on what the new thing is good at that the previous one wasn’t, ignoring the other flaws. I see that a lot with human relationships as well, where people latch on to new partners because they obviously don’t have the big problem that was a strain on the previous relationship. But eventually something else arises. Rinse and repeat.

raducu 1y ago
> “this LLM tool I’m using now is the real deal". GPT-5 is not the final deal, but it's incredibly good as is at coding. Anecdotal, but it's something completely else in terms of capabilities, ignore it at your own peril, but I think it will profoundly change software development.
- hitarpetar 1y ago
  oh it's anecdata? then I don't care. show me evidence please
- latexr 1y ago
  > ignore it at your own peril I’m not arguing for ignoring it, my point is different. > but I think it will profoundly change software development. The point is that this is said every time, together with “the previous thing to which the exact same praise was given, wasn’t it”. So it’s several rounds of “yes yes, the previous time the criticisms were right, but this time it’s different, trust me”. So everyone else is justified in being skeptical. No one wants AI to have the problems it has (technical, ethical, and others). If they didn’t it would be better for everyone. Criticism is a way of surfacing issues so they can be fixed. And sure, I’ll grant that some people want to bash the other side more than they want to arrive at the truth, but those exist in all sides of the argument (probably in roughly equal measure?). So to have a productive conversation we need to go in with the mindset of “we’re on the same side in the goal of not having this suck”.
  - blizdiddy 1y ago
    So what is your thesis? The tools keep getting better, so that’s some kind of gotcha that the emporer has no clothes? Some people prefer the absolute latest and greatest so people on the previous gen were all fakers making Pelican svgs? Maybe the productive thing is actually to ignore naysayers and goalpost movers and use the tools. You aren’t enlightened for not liking a tool. “Oh, hammers? Absolutely a bubble, after all they never fixed the hit-your-thumb issue i blogged about, and nail guns just let you hurt your thumbs faster”
    - latexr 1y ago
      No, that is not my thesis, and nowhere in my post do I talk about a dislike for a particular tool or “being enlightened”. I’ll say it again: > So to have a productive conversation we need to go in with the mindset of “we’re on the same side in the goal of not having this suck”. If you’re unwilling to engage in those terms and steel man the argument, I don’t see the point in engaging in conversation. If what you want is to straw man and throw unsubstantiated jabs at someone, there are other communities better suited for that.
  - raducu 1y ago
    > So it’s several rounds of “yes yes, the previous time the criticisms were right, but this time it’s different, trust me”. So everyone else is justified in being skeptical. True, but even the boy who cried wolf too many times eventually got his sheep eaten by the wolf. I have my own personal anecdotal benchmarks and I never hyped LLMs before GPT-5. Things that simply did not work before GPT-5 no matter how many shots I gave them, GPT-5 breezed through. For me, it would take at least 2 generations of no felt progress in the models to call for diminishing returns, and I'm not seeing them.
iammrpayments 1y ago
It could also be covert advertising like you see in reddit
- jimmydoe 1y ago
  trust your instinct. internet is dead.
  - joenot443 1y ago
    Actually, there are more internet users today than at any point in history. The internet is far from dead. The commenter in question is the CTO of the company which makes Wordfence. My instinct says they're not on the OpenAI payroll and you're looking at a normal comment and not advertisement. I think you should check your priors man; it's worth thinking critically before you toss out accusations like that.
    - mmaunder 1y ago
      Thanks joenot443. I suspect the only way to prove that I’m legit to the doubters is to do something a paid shill or a bot would never ever do. To the commenters who think I’m a shill or bot: fuck every single one of you and the various motherfucking horses you rode in on. I suspect we may be entering a dystopia where vulgarity is proof of life.
      - wayeq 1y ago
        > fuck every single one of you and the various motherfucking horses you rode in on. Maybe you're an LLM trained on 4chan
        mmaunder 1y ago
        lol - touche. Although I suspect that would have been far worse had that been the case.
        iammrpayments 1y ago
zsoltkacsandi 1y ago
From my experience this mostly happened to Antrophic models, and not because of some honeymoon period, but after the introduction of their models, the model quality and the limits are starting to decline. Many people are complaning about this on HN and Reddit. I do not have any proof, but there is a pattern, I suppose Antrophic first attracts customers, then starts to optimize costs/margins.
- latexr 1y ago
  > Many people are complaning about this on HN and Reddit. I do not have any proof, but there is a pattern, I suppose Antrophic first attracts customers, then starts to optimize costs/margins. https://en.wikipedia.org/wiki/Enshittification https://en.wikipedia.org/wiki/Enshittification
tim333 1y ago
I think it's just a function of the models getting better. One is the best and then next month another overtakes it as so on.
igneo676 1y ago
People just aren't used to how LLMs and their tools are developed Depending on the time of the year you can expect fresh updates from any given company on how their new models and tools perform and they'll generally blow the competition out of the water. The trick is to either realize that your current tools will just become magically better in a few months OR lean in and switch companies as their tools and models update.
panarky 1y ago
Results > memes. If we ship more for less because the new agent doesn't tap out, that's not a honeymoon, it's an upgrade.
- palata 1y ago
  "If", indeed. That's the problem of metrics: it depends on what you measure, and sometimes it's hard to get a meaningful answer in the short term. If you ship more for less, but less maintainable or less correct, then it's not necessarily an upgrade. Always the same question: does it benefit the developer? The product? The company? It was already possible, without AI, to look like one is doing a great job ("they are producing so much! Let's promote them!") but actually just building a bad codebase. The art being to get the promotion and move to the next step before the project implodes. Not saying that AI necessarily ends up doing that, but it most certainly help.
Bombthecat 1y ago
Because there is no switching cost, it's easy to switch, just change the API and be done
- sunnybeetroot 1y ago
  With CLI tools like Claude code and codex cli there is some friction due to the change in user experience ie keyboard shortcuts, commands etc
nickstinemates 1y ago
Another way to look at it is it's very cut throat. Switching costs are low for now, by design. The ways to get you locked in haven't been developed.

by latexr 1y ago

raducu 1y ago

> “this LLM tool I’m using now is the real deal". GPT-5 is not the final deal, but it's incredibly good as is at coding. Anecdotal, but it's something completely else in terms of capabilities, ignore it at your own peril, but I think it will profoundly change software development.

hitarpetar 1y ago
oh it's anecdata? then I don't care. show me evidence please
latexr 1y ago
> ignore it at your own peril I’m not arguing for ignoring it, my point is different. > but I think it will profoundly change software development. The point is that this is said every time, together with “the previous thing to which the exact same praise was given, wasn’t it”. So it’s several rounds of “yes yes, the previous time the criticisms were right, but this time it’s different, trust me”. So everyone else is justified in being skeptical. No one wants AI to have the problems it has (technical, ethical, and others). If they didn’t it would be better for everyone. Criticism is a way of surfacing issues so they can be fixed. And sure, I’ll grant that some people want to bash the other side more than they want to arrive at the truth, but those exist in all sides of the argument (probably in roughly equal measure?). So to have a productive conversation we need to go in with the mindset of “we’re on the same side in the goal of not having this suck”.
- blizdiddy 1y ago
  So what is your thesis? The tools keep getting better, so that’s some kind of gotcha that the emporer has no clothes? Some people prefer the absolute latest and greatest so people on the previous gen were all fakers making Pelican svgs? Maybe the productive thing is actually to ignore naysayers and goalpost movers and use the tools. You aren’t enlightened for not liking a tool. “Oh, hammers? Absolutely a bubble, after all they never fixed the hit-your-thumb issue i blogged about, and nail guns just let you hurt your thumbs faster”
  - latexr 1y ago
    No, that is not my thesis, and nowhere in my post do I talk about a dislike for a particular tool or “being enlightened”. I’ll say it again: > So to have a productive conversation we need to go in with the mindset of “we’re on the same side in the goal of not having this suck”. If you’re unwilling to engage in those terms and steel man the argument, I don’t see the point in engaging in conversation. If what you want is to straw man and throw unsubstantiated jabs at someone, there are other communities better suited for that.
- raducu 1y ago
  > So it’s several rounds of “yes yes, the previous time the criticisms were right, but this time it’s different, trust me”. So everyone else is justified in being skeptical. True, but even the boy who cried wolf too many times eventually got his sheep eaten by the wolf. I have my own personal anecdotal benchmarks and I never hyped LLMs before GPT-5. Things that simply did not work before GPT-5 no matter how many shots I gave them, GPT-5 breezed through. For me, it would take at least 2 generations of no felt progress in the models to call for diminishing returns, and I'm not seeing them.

iammrpayments 1y ago

It could also be covert advertising like you see in reddit

jimmydoe 1y ago
trust your instinct. internet is dead.
- joenot443 1y ago
  Actually, there are more internet users today than at any point in history. The internet is far from dead. The commenter in question is the CTO of the company which makes Wordfence. My instinct says they're not on the OpenAI payroll and you're looking at a normal comment and not advertisement. I think you should check your priors man; it's worth thinking critically before you toss out accusations like that.
  - mmaunder 1y ago
    Thanks joenot443. I suspect the only way to prove that I’m legit to the doubters is to do something a paid shill or a bot would never ever do. To the commenters who think I’m a shill or bot: fuck every single one of you and the various motherfucking horses you rode in on. I suspect we may be entering a dystopia where vulgarity is proof of life.
    - wayeq 1y ago
      > fuck every single one of you and the various motherfucking horses you rode in on. Maybe you're an LLM trained on 4chan
      - mmaunder 1y ago
        lol - touche. Although I suspect that would have been far worse had that been the case.
      - iammrpayments 1y ago

zsoltkacsandi 1y ago

From my experience this mostly happened to Antrophic models, and not because of some honeymoon period, but after the introduction of their models, the model quality and the limits are starting to decline. Many people are complaning about this on HN and Reddit. I do not have any proof, but there is a pattern, I suppose Antrophic first attracts customers, then starts to optimize costs/margins.

latexr 1y ago
> Many people are complaning about this on HN and Reddit. I do not have any proof, but there is a pattern, I suppose Antrophic first attracts customers, then starts to optimize costs/margins. https://en.wikipedia.org/wiki/Enshittification https://en.wikipedia.org/wiki/Enshittification

tim333 1y ago

I think it's just a function of the models getting better. One is the best and then next month another overtakes it as so on.

igneo676 1y ago

People just aren't used to how LLMs and their tools are developed Depending on the time of the year you can expect fresh updates from any given company on how their new models and tools perform and they'll generally blow the competition out of the water. The trick is to either realize that your current tools will just become magically better in a few months OR lean in and switch companies as their tools and models update.

panarky 1y ago

Results > memes. If we ship more for less because the new agent doesn't tap out, that's not a honeymoon, it's an upgrade.

palata 1y ago
"If", indeed. That's the problem of metrics: it depends on what you measure, and sometimes it's hard to get a meaningful answer in the short term. If you ship more for less, but less maintainable or less correct, then it's not necessarily an upgrade. Always the same question: does it benefit the developer? The product? The company? It was already possible, without AI, to look like one is doing a great job ("they are producing so much! Let's promote them!") but actually just building a bad codebase. The art being to get the promotion and move to the next step before the project implodes. Not saying that AI necessarily ends up doing that, but it most certainly help.

Bombthecat 1y ago

Because there is no switching cost, it's easy to switch, just change the API and be done

sunnybeetroot 1y ago
With CLI tools like Claude code and codex cli there is some friction due to the change in user experience ie keyboard shortcuts, commands etc

nickstinemates 1y ago

Another way to look at it is it's very cut throat. Switching costs are low for now, by design. The ways to get you locked in haven't been developed.