3 ms·

> The reason this matters is that LLMs are incredibly nifty often useful tools that are not AGI and also seem to be hitting a scaling wall I don't know who ne

> The reason this matters is that LLMs are incredibly nifty often useful tools that are not AGI and also seem to be hitting a scaling wall I don't know who needs to hear this, but the real break through in AI that we have had is not LLMs, but generative AI. LLM is but one specific case. Furthermore, we have hit absolutely no walls. Go download a model from Jan 2024, another from Jan 2025 and one from this year and compare. The difference is exponential in how well they have gotten.

binary132 9mo ago
>go download a model GP was talking about commercially hosted LLMs running in datacenters, not free Chinese models. Local is definitely still improving. That’s another reason the megacenter model (NVDA’s big line up forever plan) is either a financial catastrophe about to happen, or the biggest bailout ever.
- wahnfrieden 9mo ago
  GPT 5.2 is an incredible leap over 5.1 / 5
  - hadlock 9mo ago
    5.2 is great if you ask it engineering questions, or questions an engineer might ask. It is extremely mid, and actually worse than the o3/o4 era models if you start asking it trivia like if the I-80 tunnel on the bay bridge (yerba buena island) is the largest bore in the world. Don't even get me started on whatever model is wired up to the voice chat button. But yes it will write you a flawless, physics accurate flight simulator in rust on the first try. I've proven that. I guess what I'm trying to say is Anthropic was eating their lunch at coding, and OpenAI rose to the challenge, but if you're not doing engineering tasks their current models are arguably worse than older ones.
    - SoftTalker 9mo ago
      My impression is that software developers are the lions share of people actually paying for AI, but perhaps that's just my bubble world view.
      - wahnfrieden 9mo ago
        According to OpenAI it's something like 4.2% of the use. But this data is from before Codex added subscription support and I think only covers ChatGPT (back when most people were using ChatGPT for coding work, before agents got good). https://i.imgur.com/0XG2CKE.jpeg https://i.imgur.com/0XG2CKE.jpeg
        SoftTalker 9mo ago
        I'd believe that but I was commenting on who actually pays for it. My guess is that most individuals using AI in their personal lives are using some sort of free tier.
        wahnfrieden 9mo ago
        Yes 95% are unpaid
        hadlock 9mo ago
        The execs I've talked to, they are paying for it to answer capex questions, as a sounding board for decision making, and perhaps most importantly, crafting/modifying emails for tone/content. In the bay area particularly a lot of execs are foreign with english as their second language and LLMs can cut email generation time in half.
    - magicalhippo 9mo ago
      But how many are willing to fork over $20 or so a month to ask simple trivia questions?
      - hadlock 9mo ago
  - binary132 9mo ago
    how is “GPT 5.2 is good” a response to “downloadable models aren’t relevant”?
raincole 9mo ago
> exponential Is this the second most abused english word (after 'literally')? > a model from Jan 2024, another from Jan 2025 and one from this year You literally can't tell the difference is 'exponential', quadratic, or whatever from three data points. Plus it's not my experience at all. Since Deepseek I haven't found models that one can run on consumer hardware get much better.
- petesergeant 9mo ago
  I’ve heard “orders of magnitude” used more than once to mean 4-5 times
  - galaxyLogic 9mo ago
    In binary 2x is one order of magnitude
    - jonesjohnson 9mo ago
      exactly! I've been wondering about this for quite a while now. Why does everybody automatically assume that I'm using the decimal system when saying "orders of magnitude"?!
      - pilchard123 9mo ago
        Because, as xkcd 169 says, communicating badly and then actung smug when you're misunderstood is not cleverness. "Orders of magnitude" refers to a decimal system in the vast majority of uses (I must admit I have no concrete data on this, but I can find plenty of references to it being base-10 and only a suggestion that it could be sometihng else). Unless you've explicitly stated that you mean something else, people have no reason to think that you mean something else.
      - andruby 9mo ago
        I'd argue that 100% of all humans use the decimal system, most of the time. Maybe 1 to 5% of all humans use another system some of the time. Anyway, there are 10 types of people, those who understand binary and those who don't.
missedthecue 9mo ago
There is a lot of talking past each other when discussing LLM performance. The average person whose typical use case is asking ChatGPT how long they need to boil an egg for hasn't seen improvements for 18 months. Meanwhile if you're super into something like local models for example the tangible improvements are without exaggeration happening almost monthly.
- petesergeant 9mo ago
  > The average person whose typical use case is asking ChatGPT how long they need to boil an egg for hasn't seen improvements for 18 months I don’t think that’s true. I think both my mother and my mother-in-law would start to complain pretty quickly if they got pushed back to 4o. Change may have felt gradual, but I think that’s more a function of growing confidence in what they can expect the machine to do. I also think “ask how long to boil an egg” is missing a lot here. Both use ChatGPT in place of Google for all sorts of shit these days, including plenty of stuff they shouldn’t (like: “will the city be doing garbage collection tomorrow?”). Both are pretty sharp women but neither is remotely technical.
- Foobar8568 9mo ago
  Random trivia are answered much better in my case.
smohare 9mo ago
[dead]
otabdeveloper4 9mo ago
> Go download a model from Jan 2024, another from Jan 2025 and one from this year and compare. I did. The old one is smarter. (The newer ones are more verbose, though. If that impresses you, then you probably think members of parliament are geniuses.)
- moffkalast 9mo ago
  Yeah agreed, there were some minor gains, but new releases are mostly benchmark overfit sycopanthic bullshit that are only better on paper and horrible to use. The more synthetic data they add the less world knowledge the model has and the more useless it becomes. But at least they can almost mimic a basic calculator now /s For api models, OpenAI's releases have regularly not been an improvement for a long while now. Is sonnet 4.5 better than 3.5 outside pretentius agentic workflows it's been trained for? Basically impossible to tell, they make the same braindead mistakes sometimes.

binary132 9mo ago

>go download a model GP was talking about commercially hosted LLMs running in datacenters, not free Chinese models. Local is definitely still improving. That’s another reason the megacenter model (NVDA’s big line up forever plan) is either a financial catastrophe about to happen, or the biggest bailout ever.

wahnfrieden 9mo ago
GPT 5.2 is an incredible leap over 5.1 / 5
- hadlock 9mo ago
  5.2 is great if you ask it engineering questions, or questions an engineer might ask. It is extremely mid, and actually worse than the o3/o4 era models if you start asking it trivia like if the I-80 tunnel on the bay bridge (yerba buena island) is the largest bore in the world. Don't even get me started on whatever model is wired up to the voice chat button. But yes it will write you a flawless, physics accurate flight simulator in rust on the first try. I've proven that. I guess what I'm trying to say is Anthropic was eating their lunch at coding, and OpenAI rose to the challenge, but if you're not doing engineering tasks their current models are arguably worse than older ones.
  - SoftTalker 9mo ago
    My impression is that software developers are the lions share of people actually paying for AI, but perhaps that's just my bubble world view.
    - wahnfrieden 9mo ago
      According to OpenAI it's something like 4.2% of the use. But this data is from before Codex added subscription support and I think only covers ChatGPT (back when most people were using ChatGPT for coding work, before agents got good). https://i.imgur.com/0XG2CKE.jpeg https://i.imgur.com/0XG2CKE.jpeg
      - SoftTalker 9mo ago
        I'd believe that but I was commenting on who actually pays for it. My guess is that most individuals using AI in their personal lives are using some sort of free tier.
        wahnfrieden 9mo ago
        Yes 95% are unpaid
      - hadlock 9mo ago
        The execs I've talked to, they are paying for it to answer capex questions, as a sounding board for decision making, and perhaps most importantly, crafting/modifying emails for tone/content. In the bay area particularly a lot of execs are foreign with english as their second language and LLMs can cut email generation time in half.
  - magicalhippo 9mo ago
    But how many are willing to fork over $20 or so a month to ask simple trivia questions?
    - hadlock 9mo ago
- binary132 9mo ago
  how is “GPT 5.2 is good” a response to “downloadable models aren’t relevant”?

raincole 9mo ago

> exponential Is this the second most abused english word (after 'literally')? > a model from Jan 2024, another from Jan 2025 and one from this year You literally can't tell the difference is 'exponential', quadratic, or whatever from three data points. Plus it's not my experience at all. Since Deepseek I haven't found models that one can run on consumer hardware get much better.

petesergeant 9mo ago
I’ve heard “orders of magnitude” used more than once to mean 4-5 times
- galaxyLogic 9mo ago
  In binary 2x is one order of magnitude
  - jonesjohnson 9mo ago
    exactly! I've been wondering about this for quite a while now. Why does everybody automatically assume that I'm using the decimal system when saying "orders of magnitude"?!
    - pilchard123 9mo ago
      Because, as xkcd 169 says, communicating badly and then actung smug when you're misunderstood is not cleverness. "Orders of magnitude" refers to a decimal system in the vast majority of uses (I must admit I have no concrete data on this, but I can find plenty of references to it being base-10 and only a suggestion that it could be sometihng else). Unless you've explicitly stated that you mean something else, people have no reason to think that you mean something else.
    - andruby 9mo ago
      I'd argue that 100% of all humans use the decimal system, most of the time. Maybe 1 to 5% of all humans use another system some of the time. Anyway, there are 10 types of people, those who understand binary and those who don't.

missedthecue 9mo ago

There is a lot of talking past each other when discussing LLM performance. The average person whose typical use case is asking ChatGPT how long they need to boil an egg for hasn't seen improvements for 18 months. Meanwhile if you're super into something like local models for example the tangible improvements are without exaggeration happening almost monthly.

petesergeant 9mo ago
> The average person whose typical use case is asking ChatGPT how long they need to boil an egg for hasn't seen improvements for 18 months I don’t think that’s true. I think both my mother and my mother-in-law would start to complain pretty quickly if they got pushed back to 4o. Change may have felt gradual, but I think that’s more a function of growing confidence in what they can expect the machine to do. I also think “ask how long to boil an egg” is missing a lot here. Both use ChatGPT in place of Google for all sorts of shit these days, including plenty of stuff they shouldn’t (like: “will the city be doing garbage collection tomorrow?”). Both are pretty sharp women but neither is remotely technical.
Foobar8568 9mo ago
Random trivia are answered much better in my case.

smohare 9mo ago

[dead]

otabdeveloper4 9mo ago

> Go download a model from Jan 2024, another from Jan 2025 and one from this year and compare. I did. The old one is smarter. (The newer ones are more verbose, though. If that impresses you, then you probably think members of parliament are geniuses.)

moffkalast 9mo ago
Yeah agreed, there were some minor gains, but new releases are mostly benchmark overfit sycopanthic bullshit that are only better on paper and horrible to use. The more synthetic data they add the less world knowledge the model has and the more useless it becomes. But at least they can almost mimic a basic calculator now /s For api models, OpenAI's releases have regularly not been an improvement for a long while now. Is sonnet 4.5 better than 3.5 outside pretentius agentic workflows it's been trained for? Basically impossible to tell, they make the same braindead mistakes sometimes.