6 ms·

Extrapolation of past progress isn't evidence.

by StrLght 2y ago

cubefox 2y ago
Past progress is evidence for future progress.
- moe_sc 2y ago
  Might be an indicator, but it isn't evidence.
- StrLght 2y ago
  That's probably what every self-driving car company thought ~10 years ago or so, everything was moving so fast for them back then. Now it doesn't seem like we're getting close to solution for this. Surely this time it's going to be different, AGI is just around a corner. /s
  - johnthewise 2y ago
    Would you have predicted in summer of 2022 that gpt4 level conversational agent is a possibility in the next 5 years? People have tried to do it in the past 60 years and failed. How is this time not different? On a side note, I find this type of critique of what future of tech might look like the most uninteresting one. Since tech by nature inspiries people about the future, all tech get hyped up. all you gotta do then is pick any tech, point out people have been wrong, and ask how likely is it that this time it is different.
    - StrLght 2y ago
      Unfortunately, I don't see any relevance in that argument, if you consider GPT-4 to be a breakthrough -- then sure, single breakthroughs happen, I am not arguing with that. Actually, same thing happened with self-driving: I don't think many people expected Tesla to drop FSD publicly back then. Now, chain of breakthroughs happening in a small timeframe? Good luck with that.
      - cubefox 2y ago
        We have seen multiple massive AI breakthroughs in the last few years.
        Jensson 2y ago
        They are the same breakthrough applied to different domains, I don't see them as different. We will need a new breakthrough, not applying the same solution to new things.
        StrLght 2y ago
        Which ones are you referring to? Just to make it clear, I see only 1 breakthrough [0]. Everything that happened afterwards is just application of this breakthrough with different training sets / to different domains / etc. [0]: https://en.wikipedia.org/wiki/Attention_Is_All_You_Need https://en.wikipedia.org/wiki/Attention_Is_All_You_Need
        cubefox 2y ago
        Autoregressive language models, the discovery of the Chinchilla scaling law, MoEs, supervised fine-tuning, RLHF, whatever was used to create OpenAI o1, diffusion models, AlphaGo, AlphaFold, AlphaGeometry, AlphaProof.
  - mitthrowaway2 2y ago
    If you wake up from a coma and see the headline "Today Waymo has rolled out a nationwide robotaxi service", what year do you infer that it is?
- nitwit005 2y ago
  Not exactly. If you focus in on a single technology, you tend to see rapid improvement, followed by slower progress. Sometimes this is masked by people spending more due to the industry becoming more important, but it tends to be obvious over the longer term.
mitthrowaway2 2y ago
You don't have to extrapolate. There's a frenzy of talent being applied to this problem, it's drawing more brainpower the more progress that is made. Young people see this as one of the most interesting, prestigious, and best-paying fields to work in. A lot of these researchers are really talented, and are doing more than just scaling up. They're pushing at the frontiers in every direction, and finding methods that work. The progress is broadening; it's not just LLMs, it's diffusion models, it's SLAM, it's computer vision, it's inverse problems, it's locomotion. The tooling is constantly improving and being shared, lowering the barrier to entry. And classic "hard problems" are yielding in the process. It's getting hard to even find hard problems any more. I'm not saying this as someone cheering this on; I'm alarmed by it. But I can't pretend that it's running out of steam. It's possible it will run out of money, but even if so, only for a while.
- leptons 2y ago
  The AI bubble is already starting to burst. They Sam Altmans' of the world over-sold their product and over-played their hand by suggesting AGI is coming. It's not. What they have is far, far, far from AGI. "AI" is not going to be as important as you think it is in the near future, it's just the current tech-buzz and there will be something else that takes its place, just like when "web 2.0" was the new hotness.
  - kranuck 2y ago
    It's gonna be massive because companies love to replace humans at any opportunity and they don't care at all about quality in a lot of places. For example, why hire any call center workers? They already outsourced the jobs to the lowest bidder and their customers absolutely hate it. Fire those people and get some AI in there so it can provide shitty service for even cheaper. In other words, it will just make things a bit worse for everyone but those at the very top. usual shit.
- mvdtnz 2y ago
  > There's a frenzy of talent being applied to this problem, it's drawing more brainpower the more progress that is made. Young people see this as one of the most interesting, prestigious, and best-paying fields to work in. A lot of these researchers are really talented, and are doing more than just scaling up. They're pushing at the frontiers in every direction, and finding methods that work. You could have seen this exact kind of thing written 5 years ago in a thread about blockchains.
coryfklein 2y ago
Do you expect the hockeystick graph of technological development since the industrial evolution to slow? Or that it will proceed, only without significant advances in AI? Seems like the base case here is for the exponential growth to continue, and you'd need a convincing argument to say otherwise.
- StrLght 2y ago
  Which chart are you referencing exactly? How does it define technological development? It's nearly impossible for me to discuss a chart without knowing what axis refer. Without specifics all I can say is that I don't acknowledge any measurable benefits of AI (in its' current state) in real world applications. So I'd say I am leaning towards latter.
- kranuck 2y ago
  That's no guarantee that AI continues advancing at the same pace, and no one has been arguing against overall technological progress slowing Refining technology is easier than the original breakthrough, but it doesn't usually lead to a great leap forward. LLMs were the result of breakthroughs, but refining them isn't guaranteed to lead to AGI. It's not guaranteed (or likely) to improve at an exponential rate.

Extrapolation of past progress isn't evidence.

cubefox 2y ago

Past progress is evidence for future progress.

moe_sc 2y ago
Might be an indicator, but it isn't evidence.
StrLght 2y ago
That's probably what every self-driving car company thought ~10 years ago or so, everything was moving so fast for them back then. Now it doesn't seem like we're getting close to solution for this. Surely this time it's going to be different, AGI is just around a corner. /s
- johnthewise 2y ago
  Would you have predicted in summer of 2022 that gpt4 level conversational agent is a possibility in the next 5 years? People have tried to do it in the past 60 years and failed. How is this time not different? On a side note, I find this type of critique of what future of tech might look like the most uninteresting one. Since tech by nature inspiries people about the future, all tech get hyped up. all you gotta do then is pick any tech, point out people have been wrong, and ask how likely is it that this time it is different.
  - StrLght 2y ago
    Unfortunately, I don't see any relevance in that argument, if you consider GPT-4 to be a breakthrough -- then sure, single breakthroughs happen, I am not arguing with that. Actually, same thing happened with self-driving: I don't think many people expected Tesla to drop FSD publicly back then. Now, chain of breakthroughs happening in a small timeframe? Good luck with that.
    - cubefox 2y ago
      We have seen multiple massive AI breakthroughs in the last few years.
      - Jensson 2y ago
        They are the same breakthrough applied to different domains, I don't see them as different. We will need a new breakthrough, not applying the same solution to new things.
      - StrLght 2y ago
        Which ones are you referring to? Just to make it clear, I see only 1 breakthrough [0]. Everything that happened afterwards is just application of this breakthrough with different training sets / to different domains / etc. [0]: https://en.wikipedia.org/wiki/Attention_Is_All_You_Need https://en.wikipedia.org/wiki/Attention_Is_All_You_Need
        cubefox 2y ago
        Autoregressive language models, the discovery of the Chinchilla scaling law, MoEs, supervised fine-tuning, RLHF, whatever was used to create OpenAI o1, diffusion models, AlphaGo, AlphaFold, AlphaGeometry, AlphaProof.
- mitthrowaway2 2y ago
  If you wake up from a coma and see the headline "Today Waymo has rolled out a nationwide robotaxi service", what year do you infer that it is?
nitwit005 2y ago
Not exactly. If you focus in on a single technology, you tend to see rapid improvement, followed by slower progress. Sometimes this is masked by people spending more due to the industry becoming more important, but it tends to be obvious over the longer term.

mitthrowaway2 2y ago

You don't have to extrapolate. There's a frenzy of talent being applied to this problem, it's drawing more brainpower the more progress that is made. Young people see this as one of the most interesting, prestigious, and best-paying fields to work in. A lot of these researchers are really talented, and are doing more than just scaling up. They're pushing at the frontiers in every direction, and finding methods that work. The progress is broadening; it's not just LLMs, it's diffusion models, it's SLAM, it's computer vision, it's inverse problems, it's locomotion. The tooling is constantly improving and being shared, lowering the barrier to entry. And classic "hard problems" are yielding in the process. It's getting hard to even find hard problems any more. I'm not saying this as someone cheering this on; I'm alarmed by it. But I can't pretend that it's running out of steam. It's possible it will run out of money, but even if so, only for a while.

leptons 2y ago
The AI bubble is already starting to burst. They Sam Altmans' of the world over-sold their product and over-played their hand by suggesting AGI is coming. It's not. What they have is far, far, far from AGI. "AI" is not going to be as important as you think it is in the near future, it's just the current tech-buzz and there will be something else that takes its place, just like when "web 2.0" was the new hotness.
- kranuck 2y ago
  It's gonna be massive because companies love to replace humans at any opportunity and they don't care at all about quality in a lot of places. For example, why hire any call center workers? They already outsourced the jobs to the lowest bidder and their customers absolutely hate it. Fire those people and get some AI in there so it can provide shitty service for even cheaper. In other words, it will just make things a bit worse for everyone but those at the very top. usual shit.
mvdtnz 2y ago
> There's a frenzy of talent being applied to this problem, it's drawing more brainpower the more progress that is made. Young people see this as one of the most interesting, prestigious, and best-paying fields to work in. A lot of these researchers are really talented, and are doing more than just scaling up. They're pushing at the frontiers in every direction, and finding methods that work. You could have seen this exact kind of thing written 5 years ago in a thread about blockchains.

coryfklein 2y ago

Do you expect the hockeystick graph of technological development since the industrial evolution to slow? Or that it will proceed, only without significant advances in AI? Seems like the base case here is for the exponential growth to continue, and you'd need a convincing argument to say otherwise.

StrLght 2y ago
Which chart are you referencing exactly? How does it define technological development? It's nearly impossible for me to discuss a chart without knowing what axis refer. Without specifics all I can say is that I don't acknowledge any measurable benefits of AI (in its' current state) in real world applications. So I'd say I am leaning towards latter.
kranuck 2y ago
That's no guarantee that AI continues advancing at the same pace, and no one has been arguing against overall technological progress slowing Refining technology is easier than the original breakthrough, but it doesn't usually lead to a great leap forward. LLMs were the result of breakthroughs, but refining them isn't guaranteed to lead to AGI. It's not guaranteed (or likely) to improve at an exponential rate.