11 ms·

LLMs of today advance in incremental improvements. There is a finite amount of incremental improvements left between the performance of today's LLMs and the li

LLMs of today advance in incremental improvements. There is a finite amount of incremental improvements left between the performance of today's LLMs and the limits of human performance. This alone should give you second thoughts on "AI doomerism".

latexr 9mo ago
That is not necessarily true. That would be like arguing there is a finite number of improvements between the rockets of today and Star Trek ships. To get warp technology you can’t simply improve combustion engines, eventually you need to switch to something else. That could also apply to LLMs, that there would be a hard wall that the current approach can’t breach.
- ACCount37 9mo ago
  If that's the case, then, what's the wall? The "walls" that stopped AI decades ago stand no more. NLP and CSR were thought to be the "final bosses" of AI by many - until they fell to LLMs. There's no replacement. The closest thing to a "hard wall" LLMs have is probably online learning? And even that isn't really a hard wall. Because LLMs are good at in-context learning, which does many of the same things, and can do things like set up fine-tuning runs on themselves using CLI.
  - latexr 9mo ago
    > If that's the case, then, what's the wall? I didn’t say that is the case, I said it could be. Do you understand the difference? And if it is the case, it doesn’t immediately follow that we would know right now what exactly the wall would be. Often you have to hit it first. There are quite a few possible candidates.
    - ACCount37 9mo ago
      And there could be a teapot in an orbit around the Sun. Do we have any evidence for that being the case though? So far, there's a distinct lack of "wall" to be seen - and a lot of the proposed "fundamental" limitations of LLMs were discovered to be bogus with interpretability techniques, or surpassed with better scaffolding and better training.
      - latexr 9mo ago
        > And there could be a teapot in an orbit around the Sun. I think you’re confused. You are the one making the extraordinary claim, the burden of proof is on you. You asserted LLMs have a finite number of steps to go to reach (overcome?) human limits. You don’t know that. It hasn’t happened. You can’t prove it. I, on the other hand, merely pointed out that is not a certainty. Your teapot argument works against you.
        ACCount37 9mo ago
        The case against "hard wall": every "hard wall" that was predicted so far was bypassed, and the measured performance keeps going up steadily. The case for "hard wall": wishful thinking.
        Yossarrian22 9mo ago
        He’s not saying there is a hard wall he’s saying there’s a point where we’ll need new techniques or technologies not just refine the current one. Less of a hard barrier like the speed of light than an innovative one like creating artificial ammonia to make industrial amounts of fertilizer to support increasing crop amounts
  - myrmidon 9mo ago
    Agree completely with your position. I do think though that lack of online learning is a bigger drawback than a lot of people believe, because it can often be hidden/obfuscated by training for the benchmarks, basically. This becomes very visible when you compare performance on more specialized tasks that LLMs were not trained for specifically, e.g. playing games like Pokemon or Factorio: General purpose LLMs are lagging behind a lot in those compared to humans. But it's only a matter of time until we solve this IMO.
    - ACCount37 9mo ago
      By now, I subscribe to "you're just training them wrong". Pre-training a base model on text datasets teaches that model a lot, but it doesn't teach it to be good at agentic tasks and long horizon tasks. Which is why there's a capability gap there - the gap companies have to overcome "in post" with things like RLVR.
  - kubb 9mo ago
    The wall is training data. Yes, we can make more and more of post training examples. No, we can never make enough. And there are diminishing returns to that process.
  - glenneroo 9mo ago
    Hallucinations are IMO a hard wall. They have gotten slightly better over the years but you still get random results that may or may not be true, or rather, are in a range between 0-100% true, depending on which part of the answer you look at.
    - ACCount37 9mo ago
      Are they now? OpenAI's o3 was SOTA, and valued by its users for its high performance on hard tasks - while also being an absolute hallucination monster due to one of OpenAI's RLVR oopsies. You'd never know whether it's brilliant or completely full of shit at any given moment in time. People still used o3 because it was well worth it. So clearly, hallucinations do not stop AI usage - or even necessarily undermine AI performance. And if the bar you have to clear is "human performance", rather than something like "SQL database", then the bar isn't that high. See: the notorious unreliability of eyewitness testimonies. Humans avoid hallucinations better than LLMs do - not because they're fundamentally superior, but because they get a lot of meta-knowledge "for free" as a part of their training process. LLMs get very little meta-knowledge in pre-training, and little skill in using what they have. Doesn't mean you can't train them to be more reliable - there are pipelines for that already. It just makes it hard.
falloutx 9mo ago
AI doomerism was sold by the AI companies as some sort of "learn it or you'll fall behind". But they didnt think it through, now that AI is widely seen as a bad thing by general public (except programmers who think they can deliver slop faster). Who would be buying $200/month sub when they get laid off, I am not sure the strategy of spreading fear was worth it. I also don't think this tech can ever be profitable. I hope it burns more money at this rate.
- pmontra 9mo ago
  The employer buys the AI subscription, not the employee. An employee that sends company code to an external AI is somebody looking for troubles. In the case of contractors, the contractors buy the subscription but they need authorization to give access to the code. That's obvious if the property of the code is of the customer but there might be NDAs even if the contractor owns the code.
  - falloutx 9mo ago
    If companies have very little no of employees, AI companies are expecting regular people to pay for AI access. Then who would be buying $200/month for a thing that took their job? By cutting employees strat, the AI companies also lose much more in revenue.
112233 9mo ago
pole-vaulting records improve incrementally too. and there is finite distance left to the moon. without deep understanding and experience and numbers to back up the opinion, any progress seems about to reach arbitrary goals.
immibis 9mo ago
You can't get to the moon by learning to climb taller and taller trees.

latexr 9mo ago

That is not necessarily true. That would be like arguing there is a finite number of improvements between the rockets of today and Star Trek ships. To get warp technology you can’t simply improve combustion engines, eventually you need to switch to something else. That could also apply to LLMs, that there would be a hard wall that the current approach can’t breach.

ACCount37 9mo ago
If that's the case, then, what's the wall? The "walls" that stopped AI decades ago stand no more. NLP and CSR were thought to be the "final bosses" of AI by many - until they fell to LLMs. There's no replacement. The closest thing to a "hard wall" LLMs have is probably online learning? And even that isn't really a hard wall. Because LLMs are good at in-context learning, which does many of the same things, and can do things like set up fine-tuning runs on themselves using CLI.
- latexr 9mo ago
  > If that's the case, then, what's the wall? I didn’t say that is the case, I said it could be. Do you understand the difference? And if it is the case, it doesn’t immediately follow that we would know right now what exactly the wall would be. Often you have to hit it first. There are quite a few possible candidates.
  - ACCount37 9mo ago
    And there could be a teapot in an orbit around the Sun. Do we have any evidence for that being the case though? So far, there's a distinct lack of "wall" to be seen - and a lot of the proposed "fundamental" limitations of LLMs were discovered to be bogus with interpretability techniques, or surpassed with better scaffolding and better training.
    - latexr 9mo ago
      > And there could be a teapot in an orbit around the Sun. I think you’re confused. You are the one making the extraordinary claim, the burden of proof is on you. You asserted LLMs have a finite number of steps to go to reach (overcome?) human limits. You don’t know that. It hasn’t happened. You can’t prove it. I, on the other hand, merely pointed out that is not a certainty. Your teapot argument works against you.
      - ACCount37 9mo ago
        The case against "hard wall": every "hard wall" that was predicted so far was bypassed, and the measured performance keeps going up steadily. The case for "hard wall": wishful thinking.
        Yossarrian22 9mo ago
        He’s not saying there is a hard wall he’s saying there’s a point where we’ll need new techniques or technologies not just refine the current one. Less of a hard barrier like the speed of light than an innovative one like creating artificial ammonia to make industrial amounts of fertilizer to support increasing crop amounts
- myrmidon 9mo ago
  Agree completely with your position. I do think though that lack of online learning is a bigger drawback than a lot of people believe, because it can often be hidden/obfuscated by training for the benchmarks, basically. This becomes very visible when you compare performance on more specialized tasks that LLMs were not trained for specifically, e.g. playing games like Pokemon or Factorio: General purpose LLMs are lagging behind a lot in those compared to humans. But it's only a matter of time until we solve this IMO.
  - ACCount37 9mo ago
    By now, I subscribe to "you're just training them wrong". Pre-training a base model on text datasets teaches that model a lot, but it doesn't teach it to be good at agentic tasks and long horizon tasks. Which is why there's a capability gap there - the gap companies have to overcome "in post" with things like RLVR.
- kubb 9mo ago
  The wall is training data. Yes, we can make more and more of post training examples. No, we can never make enough. And there are diminishing returns to that process.
- glenneroo 9mo ago
  Hallucinations are IMO a hard wall. They have gotten slightly better over the years but you still get random results that may or may not be true, or rather, are in a range between 0-100% true, depending on which part of the answer you look at.
  - ACCount37 9mo ago
    Are they now? OpenAI's o3 was SOTA, and valued by its users for its high performance on hard tasks - while also being an absolute hallucination monster due to one of OpenAI's RLVR oopsies. You'd never know whether it's brilliant or completely full of shit at any given moment in time. People still used o3 because it was well worth it. So clearly, hallucinations do not stop AI usage - or even necessarily undermine AI performance. And if the bar you have to clear is "human performance", rather than something like "SQL database", then the bar isn't that high. See: the notorious unreliability of eyewitness testimonies. Humans avoid hallucinations better than LLMs do - not because they're fundamentally superior, but because they get a lot of meta-knowledge "for free" as a part of their training process. LLMs get very little meta-knowledge in pre-training, and little skill in using what they have. Doesn't mean you can't train them to be more reliable - there are pipelines for that already. It just makes it hard.

falloutx 9mo ago

AI doomerism was sold by the AI companies as some sort of "learn it or you'll fall behind". But they didnt think it through, now that AI is widely seen as a bad thing by general public (except programmers who think they can deliver slop faster). Who would be buying $200/month sub when they get laid off, I am not sure the strategy of spreading fear was worth it. I also don't think this tech can ever be profitable. I hope it burns more money at this rate.

pmontra 9mo ago
The employer buys the AI subscription, not the employee. An employee that sends company code to an external AI is somebody looking for troubles. In the case of contractors, the contractors buy the subscription but they need authorization to give access to the code. That's obvious if the property of the code is of the customer but there might be NDAs even if the contractor owns the code.
- falloutx 9mo ago
  If companies have very little no of employees, AI companies are expecting regular people to pay for AI access. Then who would be buying $200/month for a thing that took their job? By cutting employees strat, the AI companies also lose much more in revenue.

112233 9mo ago

pole-vaulting records improve incrementally too. and there is finite distance left to the moon. without deep understanding and experience and numbers to back up the opinion, any progress seems about to reach arbitrary goals.

immibis 9mo ago

You can't get to the moon by learning to climb taller and taller trees.