4 ms·

> I don't see this getting better. We went from 2 + 7 = 11 to "solved a frontier math problem" in 3 years, yet people don't think this will improve?

number6 7mo ago
But can it count the R's in strawberry?
- Aditya_Garg 7mo ago
  yes its ridiculously good at stuff like that now. I dare you to try and trick it.
  - frizlab 7mo ago
    https://news.ycombinator.com/item?id=47495568 https://news.ycombinator.com/item?id=47495568
    - thedatamonger 7mo ago
      what bothers me is not that this issue will certainly disappear now that it has been identified, but that that we have yet to identify the category of these "stupid" bugs ...
      - sigmoid10 7mo ago
        We already know exactly what causes these bugs. They are not a fundamental problem of LLMs, they are a problem of tokenizers. The actual model simply doesn't get to see the same text that you see. It can only infer this stuff from related info it was trained on. It's as if someone asked you how many 1s there are in the binary representation of this text. You'd also need to convert it first to think it through, or use some external tool, even though your computer never saw anything else.
        datsci_est_2015 7mo ago
        Okay but, genuinely not an expert on the latest with LLMs, but isn’t tokenization an inherent part of LLM construction? Kind of like support vectors in SVMs, or nodes in neural networks? Once we remove tokenization from the equation, aren’t we no longer talking about LLMs?
        fenomas 7mo ago
        It's not a side effect of tokenization per se, but of the tokenizers people use in actual practice. If somebody really wanted an LLM that can flawlessly count letters in words, they could train one with a naive tokenizer (like just ascii characters). But the resulting model would be very bad (for its size) at language or reasoning tasks. Basically it's an engineering tradeoff. There is more demand for LLMs that can solve open math problems, but can't count the Rs in strawberry, than there is for models that can count letters but are bad at everything else.
        Measter 7mo ago
        > It's as if someone asked you how many 1s there are in the binary representation of this text. I'm actually kinda pleased with how close I guessed! I estimated 4 set bits per character, which with 491 characters in your post (including spaces) comes to 1964. Then I ran your message through a program to get the actual number, and turns out it has 1800 exactly.
        sigmoid10 7mo ago
        >I estimated 4 set bits per character, which with 491 characters in your post (including spaces) comes to 1964 And that's exactly the kind of reasoning an LLM does when you ask it about characters in a word. It doesn't come from the word, it comes from other heuristics it picked up during training.
      - deleted 7mo ago
        [deleted]
- Paradigma11 7mo ago
  That question is equivalent to asking a human to add the wavelengths of those two colors and divide it by 3.
  - snovv_crash 7mo ago
    Unless you're aware of hyperspectral image adapters for LLMs they aren't capable of that either.
  - szszrk 7mo ago
    Unfair - human beats AI in this comparison, as human will instantly answer "I don't know" instead of yelling a random number. Or at best "I don't know, but maybe I can find out" and proceed to finding out/ But he is unlikely to shout "6" because he heard this number once when someone talked about light.
    - koliber 7mo ago
      > human will instantly answer "I don't know" instead of yelling a random number. Seems that you never worked with Accenture consultants?
      - szszrk 7mo ago
        Fair. Yet this can be filtered with fixed rules, like "output produced by corporate structures is untrusted random data".
  - thegabriele 7mo ago
    Why is that?
    - deleted 7mo ago
      [deleted]
    - Paradigma11 7mo ago
      Because LLMs dont have a textual representation of any text they consume. Its just vectors to them. Which is why they are so good at ignoring typos, the vector distance is so small it makes no difference to them.
saidnooneever 7mo ago
if you let million monkeys bash typewriter. something something book
Validark 7mo ago
I tend to disagree that improvement is inherent. Really I'm just expressing an aesthetic preference when I say this, because I don't disagree that a lot of things improve. But it's not a guarantee, and it does take people doing the work and thinking about the same thing every day for years. In many cases there's only one person uniquely positioned to make a discovery, and it's by no means guaranteed to happen. Of course, in many cases there are a whole bunch of people who seem almost equally capable of solving something first, but I think if you say things like "I'm sure they're going to make it better" you're leaving to chance something you yourself could have an impact on. You can participate in pushing the boundaries or even making a small push on something that accelerates someone else's work. You can also donate money to research you are interested in to help pay people who might come up with breakthroughs. Don't assume other people will build the future, you should do it too! (Not saying you DON'T)
3abiton 7mo ago
The problem class is rather very structured which makes it "easier", yet the results are undeniably impressive
nopinsight 7mo ago
LLMs in some form will likely be a key component in the first AGI system we (help) build. We might still lack something essential. However, people who keep doubting AGI is even possible should learn more about The Church-Turing Thesis. https://plato.stanford.edu/entries/church-turing/ https://plato.stanford.edu/entries/church-turing/
- benterix 7mo ago
  This is a long read on things most people here know at least in some form. Could you pint to a particular fragment or a quote?
- gf000 7mo ago
  AGI is definitely possible - there is nothing fundamentally different in the human brain that would surpass a Turing machine's computational power (unless you believe in some higher powers, etc). We are just meat-computers. But at the same time, there is absolutely no indication or reason to believe that this wave of AI hype is the AGI one and that LLMs can be scaled further. We absolutely don't know almost anything about the nature of human intelligence, so we can't even really claim whether we are close or far.
snemvalts 7mo ago
Scaling law is a power law , requiring orders of magnitude more compute and data for better accuracy from pre-training. Most companies have maxed it out. For RL, we are arriving at a similar point https://www.tobyord.com/writing/how-well-does-rl-scale https://www.tobyord.com/writing/how-well-does-rl-scale Next stop is inference scaling with longer context window and longer reasoning. But instead of it being a one-off training cost, it becomes a running cost. In essence we are chasing ever smaller gains in exchange for exponentially increasing costs. This energy will run out. There needs to be something completely different than LLMs for meaningful further progress.
datsci_est_2015 7mo ago
I’ve seen this style of take so much that I’m dying for someone to name a logical fallacy for it, like “appeal to progress” or something. Step away from LLMs for a second and recognize that “Yesterday it was X, so today it must be X+1” is such a naive take and obviously something that humans so easily fall into a trap of believing (see: flying cars).
- Gareth321 7mo ago
  In finance we say "past performance does not guarantee future returns." Not because we don't believe that, statistically, returns will continue to grow at x rate, but because there is a chance that they won't. The reality bias is actually in favour of these getting better faster, but there is a chance they do not.
  - aspenmartin 7mo ago
    this is true because markets are generally efficient. It's very hard to find predictive signals. That is a completely different space than what we're talking about here. Performance is incredibly predictable through scaling laws that continue to hold even at the largest scales we've built
    - Gareth321 7mo ago
      I agree this is a new space and prediction volatility is much higher. We have evidence going back to at least 2019 that improvements have been exponential (https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com...). The benchmarks are all over the place because improvements don't happen in a straight line. Even composites aren't that useful because the last 10% improvement can require more effort than the first 90%. To be frank, from what I can see, even if all progress stopped right now, it would take 1-2 decades to fully operationalise the existing potential of LLMs. There would be massive economic and social change. But progress is not stopping, and in some measurements, continues to improve exponentially. I really think this is incredibly transformative. Moreso than anything humanity has ever experienced. In the last year, OpenAI and potentially Claude have been working on recursive self-improvement. Meaning these models are designing better versions of themselves. This means we have effectively entered the singularity.
al_borland 7mo ago
[dead]
eamag 7mo ago
Self driving
zeroonetwothree 7mo ago
> We went from 2 + 7 = 11 to "solved a frontier math problem" in 3 years, yet people don't think this will improve? This is disingenuous... I don't think people were impressed by GPT 3.5 because it was bad at math. It's like saying: "We went from being unable to take off and the crew dying in a fire to a moon landing in 2 years, imagine how soon we'll have people on Mars"

number6 7mo ago

But can it count the R's in strawberry?

Aditya_Garg 7mo ago
yes its ridiculously good at stuff like that now. I dare you to try and trick it.
- frizlab 7mo ago
  https://news.ycombinator.com/item?id=47495568 https://news.ycombinator.com/item?id=47495568
  - thedatamonger 7mo ago
    what bothers me is not that this issue will certainly disappear now that it has been identified, but that that we have yet to identify the category of these "stupid" bugs ...
    - sigmoid10 7mo ago
      We already know exactly what causes these bugs. They are not a fundamental problem of LLMs, they are a problem of tokenizers. The actual model simply doesn't get to see the same text that you see. It can only infer this stuff from related info it was trained on. It's as if someone asked you how many 1s there are in the binary representation of this text. You'd also need to convert it first to think it through, or use some external tool, even though your computer never saw anything else.
      - datsci_est_2015 7mo ago
        Okay but, genuinely not an expert on the latest with LLMs, but isn’t tokenization an inherent part of LLM construction? Kind of like support vectors in SVMs, or nodes in neural networks? Once we remove tokenization from the equation, aren’t we no longer talking about LLMs?
        fenomas 7mo ago
        It's not a side effect of tokenization per se, but of the tokenizers people use in actual practice. If somebody really wanted an LLM that can flawlessly count letters in words, they could train one with a naive tokenizer (like just ascii characters). But the resulting model would be very bad (for its size) at language or reasoning tasks. Basically it's an engineering tradeoff. There is more demand for LLMs that can solve open math problems, but can't count the Rs in strawberry, than there is for models that can count letters but are bad at everything else.
      - Measter 7mo ago
        > It's as if someone asked you how many 1s there are in the binary representation of this text. I'm actually kinda pleased with how close I guessed! I estimated 4 set bits per character, which with 491 characters in your post (including spaces) comes to 1964. Then I ran your message through a program to get the actual number, and turns out it has 1800 exactly.
        sigmoid10 7mo ago
        >I estimated 4 set bits per character, which with 491 characters in your post (including spaces) comes to 1964 And that's exactly the kind of reasoning an LLM does when you ask it about characters in a word. It doesn't come from the word, it comes from other heuristics it picked up during training.
    - deleted 7mo ago
      [deleted]
Paradigma11 7mo ago
That question is equivalent to asking a human to add the wavelengths of those two colors and divide it by 3.
- snovv_crash 7mo ago
  Unless you're aware of hyperspectral image adapters for LLMs they aren't capable of that either.
- szszrk 7mo ago
  Unfair - human beats AI in this comparison, as human will instantly answer "I don't know" instead of yelling a random number. Or at best "I don't know, but maybe I can find out" and proceed to finding out/ But he is unlikely to shout "6" because he heard this number once when someone talked about light.
  - koliber 7mo ago
    > human will instantly answer "I don't know" instead of yelling a random number. Seems that you never worked with Accenture consultants?
    - szszrk 7mo ago
      Fair. Yet this can be filtered with fixed rules, like "output produced by corporate structures is untrusted random data".
- thegabriele 7mo ago
  Why is that?
  - deleted 7mo ago
    [deleted]
  - Paradigma11 7mo ago
    Because LLMs dont have a textual representation of any text they consume. Its just vectors to them. Which is why they are so good at ignoring typos, the vector distance is so small it makes no difference to them.

saidnooneever 7mo ago

if you let million monkeys bash typewriter. something something book

Validark 7mo ago

I tend to disagree that improvement is inherent. Really I'm just expressing an aesthetic preference when I say this, because I don't disagree that a lot of things improve. But it's not a guarantee, and it does take people doing the work and thinking about the same thing every day for years. In many cases there's only one person uniquely positioned to make a discovery, and it's by no means guaranteed to happen. Of course, in many cases there are a whole bunch of people who seem almost equally capable of solving something first, but I think if you say things like "I'm sure they're going to make it better" you're leaving to chance something you yourself could have an impact on. You can participate in pushing the boundaries or even making a small push on something that accelerates someone else's work. You can also donate money to research you are interested in to help pay people who might come up with breakthroughs. Don't assume other people will build the future, you should do it too! (Not saying you DON'T)

3abiton 7mo ago

The problem class is rather very structured which makes it "easier", yet the results are undeniably impressive

nopinsight 7mo ago

LLMs in some form will likely be a key component in the first AGI system we (help) build. We might still lack something essential. However, people who keep doubting AGI is even possible should learn more about The Church-Turing Thesis. https://plato.stanford.edu/entries/church-turing/ https://plato.stanford.edu/entries/church-turing/

benterix 7mo ago
This is a long read on things most people here know at least in some form. Could you pint to a particular fragment or a quote?
gf000 7mo ago
AGI is definitely possible - there is nothing fundamentally different in the human brain that would surpass a Turing machine's computational power (unless you believe in some higher powers, etc). We are just meat-computers. But at the same time, there is absolutely no indication or reason to believe that this wave of AI hype is the AGI one and that LLMs can be scaled further. We absolutely don't know almost anything about the nature of human intelligence, so we can't even really claim whether we are close or far.

snemvalts 7mo ago

Scaling law is a power law , requiring orders of magnitude more compute and data for better accuracy from pre-training. Most companies have maxed it out. For RL, we are arriving at a similar point https://www.tobyord.com/writing/how-well-does-rl-scale https://www.tobyord.com/writing/how-well-does-rl-scale Next stop is inference scaling with longer context window and longer reasoning. But instead of it being a one-off training cost, it becomes a running cost. In essence we are chasing ever smaller gains in exchange for exponentially increasing costs. This energy will run out. There needs to be something completely different than LLMs for meaningful further progress.

datsci_est_2015 7mo ago

I’ve seen this style of take so much that I’m dying for someone to name a logical fallacy for it, like “appeal to progress” or something. Step away from LLMs for a second and recognize that “Yesterday it was X, so today it must be X+1” is such a naive take and obviously something that humans so easily fall into a trap of believing (see: flying cars).

Gareth321 7mo ago
In finance we say "past performance does not guarantee future returns." Not because we don't believe that, statistically, returns will continue to grow at x rate, but because there is a chance that they won't. The reality bias is actually in favour of these getting better faster, but there is a chance they do not.
- aspenmartin 7mo ago
  this is true because markets are generally efficient. It's very hard to find predictive signals. That is a completely different space than what we're talking about here. Performance is incredibly predictable through scaling laws that continue to hold even at the largest scales we've built
  - Gareth321 7mo ago
    I agree this is a new space and prediction volatility is much higher. We have evidence going back to at least 2019 that improvements have been exponential (https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com...). The benchmarks are all over the place because improvements don't happen in a straight line. Even composites aren't that useful because the last 10% improvement can require more effort than the first 90%. To be frank, from what I can see, even if all progress stopped right now, it would take 1-2 decades to fully operationalise the existing potential of LLMs. There would be massive economic and social change. But progress is not stopping, and in some measurements, continues to improve exponentially. I really think this is incredibly transformative. Moreso than anything humanity has ever experienced. In the last year, OpenAI and potentially Claude have been working on recursive self-improvement. Meaning these models are designing better versions of themselves. This means we have effectively entered the singularity.

al_borland 7mo ago

[dead]

eamag 7mo ago

Self driving

zeroonetwothree 7mo ago

> We went from 2 + 7 = 11 to "solved a frontier math problem" in 3 years, yet people don't think this will improve? This is disingenuous... I don't think people were impressed by GPT 3.5 because it was bad at math. It's like saying: "We went from being unable to take off and the crew dying in a fire to a moon landing in 2 years, imagine how soon we'll have people on Mars"