4 ms·

The thing I never understood is: why would it go vertical? It would at best be an exponential curve, and I have doubts about that. I admit looking at the 100 y

by mcv 3y ago

The thing I never understood is: why would it go vertical? It would at best be an exponential curve, and I have doubts about that. I admit looking at the 100 years before 1993, it looks like innovation is constantly speeding up, but even then there's not going to be a moment that we suddenly have infinite knowledge. There's no such thing as infinite knowledge; it's still bound by physical limits. It still takes time and resources to actually do something with it. And if you look at the past 30 years, it doesn't really look like innovation is speeding up at all. There is plenty of innovation, but is it happening at an ever faster pace? I don't see it. Not to mention that much of it is hype and fashion, and not really fundamentally new. Even AI progress is driven mostly by faster hardware and more data, and not really fundamentally new technologies. And that's not even getting into the science crisis: lots of science is not really reproducible. And while LLMs are certainly an exciting new technology, it's not at all clear that they're really more than a glorified autocorrect. So I'm extremely skeptical about those singularity ideas. It's an exciting SciFi idea, but I don't think it's true. And certainly not within the next 30 years.

bratbag 3y ago
Its infinite from the perspective of our side of the curve. It's another application of advanced technology appearing to be magic, but imagine transitioning into it in a matter of hours, then with that advanced tech transitioning further into damn-near godhood within minutes. Then imagine what happens in the second after that. It may be operating within the boundaries of physics, but they would be physical rules well beyond our understanding and may even be infinite by our own limited definition of physics. That's the curve.
- sfn42 3y ago
  I think it's completely ridiculous how you people just casually imply that a current computer can somehow achieve "damn-near godhood". Hardware is a bottleneck. Better hardware can't be designed, manufactured and installed in minutes. You can't just infinitely improve software sitting on the same hardware. At some point the software is as efficient as it can possibly be. And realistically a complex piece of software won't be close to optimal. You people love dreaming up possibilities but you never stop to actually think about the limitations. You just assume everything will work out magically and satisfy your fantasy.
  - abss 3y ago
    Science inherently concerns itself with understanding and defining limits. However, when it comes to intelligence, there appears to be a significant gap in our grasp of these boundaries. Reflecting on my own thoughts from 35 years ago, I recall envisioning the singularity at the age of 12 or 13. Looking back, I realize that many of my conceptions from that age were either incorrect or grossly exaggerated. This realization leads me to speculate that the notion of the singularity might fall into a similar category, although concrete evidence to support this is currently lacking. While I find myself partially aligning with your points regarding magical thinking in this context, I am curious about how one might construct a robust, scientifically grounded argument in this domain.
  - kridsdale3 3y ago
    I'm glad to see people like you that understand that computers are ultimately machines that we put electric current (limited by our infrastructure) in to, and even with optimal efficiency (which we are no where close to), some math comes out. This year we all learned that a lot of what we thought was human thought can be expressed as math. Cool. But we can't scale that any higher than the physical input to the computer, and the physical heat dissipation that we can handle either. A 100% occupancy computing device is effectively a radiator. So singularity people think 2 fallacies: 1. That it is possible to write an exponential function that somehow scales to asymptotic 2. That it is possible to write software that is so energy efficient that its output somehow exceeds the power draw 1 can be redefined to say "effectively" vertical. Ok. Cool motive, not a singularity. 2 can be handled by pumping more and more power in to the machine and building new nuclear plants to do so. This is what Microsoft is doing today.
  - wait_a_minute 3y ago
    > Hardware is a bottleneck. Better hardware can't be designed, manufactured and installed in minutes. Yes you can. The Cloud.
    - mcv 3y ago
      Even the cloud has its limits, though. It's not infinite, and it does require money.
- wait_a_minute 3y ago
  Like the Rapture and other concepts like that. How fascinating. Within a split second, in the time it takes someone to blink and hear a trumpet's blast, everything could change. Amazing! I can't wait!
  - mcv 3y ago
    As both a Christian and an AI-educated software engineer who subscribes to neither the rapture nor the singularity, I really like this comparison. They both feel like a kind of escapist thinking that in the near future, this magical event will happen that will make everything else irrelevant.
    - wait_a_minute 3y ago
      The Singularity people and the Rapture people are both correct. Can already see the event starting to take shape, just gotta know where to look for all the signs. Everything is rapidly heading in that direction.
jncfhnb 3y ago
It wouldn’t. It would be a logistic curve. Pretty much everything people call exponential should actually be logistic
- diffeomorphism 3y ago
  That just agrees with the post. Nothing becomes vertical or infinite, at best exponential for a limited time, i.e. logistic.
  - jncfhnb 3y ago
    I’m aware that it agrees with the post
  - deleted 3y ago
    [deleted]
sjamaan 3y ago
I think these progressions of technology are more likely to be like Moore's law: it might be true for a while but eventually it'll peter out. AI itself doesn't understand anything, and there's a limit to human understanding, so technological progression will eventually be self-limiting.
AngaraliTurk 3y ago
> And while LLMs are certainly an exciting new technology, it's not at all clear that they're really more than a glorified autocorrect. Are we sure things like biology, or heck, even the universe as a whole and its parts, aren't "glorified x thing"? Can't we apply this argument to just about anything?
- rf15 3y ago
  I feel like comparing it to biological systems glances over like half the points the parent makes. Also of course it's a sophisticated autocorrect, there's no system for agency, which is what we also desire from a proper AGI.
  - AngaraliTurk 3y ago
    I wanted to put the focus on the overused "glorified x thing" sentence, which to me seems to be applicable to just about anything. I didn't want to liken/compare AI to biological systems per se.
  - HDThoreaun 3y ago
    Free will is an illusion. no one has agency. we're all just following our baked in incentives.
    - mcv 3y ago
      I'm able to take initiative and act on my own internal thought processes, though. I'm not limited by whether someone prompts me to do something.
      - AngaraliTurk 3y ago
        You and me think we are but we can't be sure, and many before us have raised a doubt. As for the prompt(s), to use such a limiting term, they could as well come from a self-reinforcing loop that starts when we're born and is influenced by external stimuli.
antonvs 3y ago
> it's not at all clear that they're really more than a glorified autocorrect. I use LLMs regularly in my job - many times a day - and I suspect you haven't used them much if you think this. They're relevant to the singularity discussion though, because they already give a taste of what superhuman intelligence could look like. ChatGPT, for example, is objectively superhuman in many ways, despite its significant limitations. Once systems like this are more integrated with the outside world and able to learn more directly from feedback, we'll get an even bigger leap forward. Dismissing this as "glorified autocorrect" is extremely far off base.
- mcv 3y ago
  > ChatGPT, for example, is objectively superhuman in many ways But so is a car. ChatGPT is mostly superhuman in its ability to draw upon enormous numbers of existing sources. It's not superhuman, or even human, in terms of logic, reasoning, or inventing something truly new.

by mcv 3y ago

bratbag 3y ago

Its infinite from the perspective of our side of the curve. It's another application of advanced technology appearing to be magic, but imagine transitioning into it in a matter of hours, then with that advanced tech transitioning further into damn-near godhood within minutes. Then imagine what happens in the second after that. It may be operating within the boundaries of physics, but they would be physical rules well beyond our understanding and may even be infinite by our own limited definition of physics. That's the curve.

sfn42 3y ago
I think it's completely ridiculous how you people just casually imply that a current computer can somehow achieve "damn-near godhood". Hardware is a bottleneck. Better hardware can't be designed, manufactured and installed in minutes. You can't just infinitely improve software sitting on the same hardware. At some point the software is as efficient as it can possibly be. And realistically a complex piece of software won't be close to optimal. You people love dreaming up possibilities but you never stop to actually think about the limitations. You just assume everything will work out magically and satisfy your fantasy.
- abss 3y ago
  Science inherently concerns itself with understanding and defining limits. However, when it comes to intelligence, there appears to be a significant gap in our grasp of these boundaries. Reflecting on my own thoughts from 35 years ago, I recall envisioning the singularity at the age of 12 or 13. Looking back, I realize that many of my conceptions from that age were either incorrect or grossly exaggerated. This realization leads me to speculate that the notion of the singularity might fall into a similar category, although concrete evidence to support this is currently lacking. While I find myself partially aligning with your points regarding magical thinking in this context, I am curious about how one might construct a robust, scientifically grounded argument in this domain.
- kridsdale3 3y ago
  I'm glad to see people like you that understand that computers are ultimately machines that we put electric current (limited by our infrastructure) in to, and even with optimal efficiency (which we are no where close to), some math comes out. This year we all learned that a lot of what we thought was human thought can be expressed as math. Cool. But we can't scale that any higher than the physical input to the computer, and the physical heat dissipation that we can handle either. A 100% occupancy computing device is effectively a radiator. So singularity people think 2 fallacies: 1. That it is possible to write an exponential function that somehow scales to asymptotic 2. That it is possible to write software that is so energy efficient that its output somehow exceeds the power draw 1 can be redefined to say "effectively" vertical. Ok. Cool motive, not a singularity. 2 can be handled by pumping more and more power in to the machine and building new nuclear plants to do so. This is what Microsoft is doing today.
- wait_a_minute 3y ago
  > Hardware is a bottleneck. Better hardware can't be designed, manufactured and installed in minutes. Yes you can. The Cloud.
  - mcv 3y ago
    Even the cloud has its limits, though. It's not infinite, and it does require money.
wait_a_minute 3y ago
Like the Rapture and other concepts like that. How fascinating. Within a split second, in the time it takes someone to blink and hear a trumpet's blast, everything could change. Amazing! I can't wait!
- mcv 3y ago
  As both a Christian and an AI-educated software engineer who subscribes to neither the rapture nor the singularity, I really like this comparison. They both feel like a kind of escapist thinking that in the near future, this magical event will happen that will make everything else irrelevant.
  - wait_a_minute 3y ago
    The Singularity people and the Rapture people are both correct. Can already see the event starting to take shape, just gotta know where to look for all the signs. Everything is rapidly heading in that direction.

jncfhnb 3y ago

It wouldn’t. It would be a logistic curve. Pretty much everything people call exponential should actually be logistic

diffeomorphism 3y ago
That just agrees with the post. Nothing becomes vertical or infinite, at best exponential for a limited time, i.e. logistic.
- jncfhnb 3y ago
  I’m aware that it agrees with the post
- deleted 3y ago
  [deleted]

sjamaan 3y ago

I think these progressions of technology are more likely to be like Moore's law: it might be true for a while but eventually it'll peter out. AI itself doesn't understand anything, and there's a limit to human understanding, so technological progression will eventually be self-limiting.

AngaraliTurk 3y ago

> And while LLMs are certainly an exciting new technology, it's not at all clear that they're really more than a glorified autocorrect. Are we sure things like biology, or heck, even the universe as a whole and its parts, aren't "glorified x thing"? Can't we apply this argument to just about anything?

rf15 3y ago
I feel like comparing it to biological systems glances over like half the points the parent makes. Also of course it's a sophisticated autocorrect, there's no system for agency, which is what we also desire from a proper AGI.
- AngaraliTurk 3y ago
  I wanted to put the focus on the overused "glorified x thing" sentence, which to me seems to be applicable to just about anything. I didn't want to liken/compare AI to biological systems per se.
- HDThoreaun 3y ago
  Free will is an illusion. no one has agency. we're all just following our baked in incentives.
  - mcv 3y ago
    I'm able to take initiative and act on my own internal thought processes, though. I'm not limited by whether someone prompts me to do something.
    - AngaraliTurk 3y ago
      You and me think we are but we can't be sure, and many before us have raised a doubt. As for the prompt(s), to use such a limiting term, they could as well come from a self-reinforcing loop that starts when we're born and is influenced by external stimuli.

antonvs 3y ago

> it's not at all clear that they're really more than a glorified autocorrect. I use LLMs regularly in my job - many times a day - and I suspect you haven't used them much if you think this. They're relevant to the singularity discussion though, because they already give a taste of what superhuman intelligence could look like. ChatGPT, for example, is objectively superhuman in many ways, despite its significant limitations. Once systems like this are more integrated with the outside world and able to learn more directly from feedback, we'll get an even bigger leap forward. Dismissing this as "glorified autocorrect" is extremely far off base.

mcv 3y ago
> ChatGPT, for example, is objectively superhuman in many ways But so is a car. ChatGPT is mostly superhuman in its ability to draw upon enormous numbers of existing sources. It's not superhuman, or even human, in terms of logic, reasoning, or inventing something truly new.