5 ms·

Singularity Missed

35 points by telotortium 2y ago

Quinner 2y ago
What this is really saying is that hardware is a necessary precondition for, but not determinative of, intelligence. If you have a human brain worth of neurons firing randomly you don't get consciousness. The structure and instruction set is crucial. And we are perhaps 20 years away from figuring out how to structure the compute power we have to get the consciousness of a human adult.
throwuxiytayq 2y ago
> So we shouldn't expect human like intelligence before late 2040's. That is of course if we actually get an insect brain this year. Which we won't. Was this written pre-ChatGPT? I am amazed that the author decided that this is an insightful take he'd like to share on the internet. They managed to confuse computational capacity and actual capability, while remaining completely blind to the fact that nobody expects AI development to happen along this sort of curve, or to resemble a walk up the list of intelligent animal species. We didn't expect it 10 years ago, we especially don't expect it now that it's super prominently not happening.
- telotortium 2y ago
  It does show that Kurzweil's model of AI development is significantly deficient. Kurzweil isn't considered widely discredited in the AGI space, so it's a useful argument.
  - timdiggerm 2y ago
    > Kurzweil isn't considered widely discredited in the AGI space Thank you for helping discredit the AGI space
andrewla 2y ago
I'm of the school of thought that says that the Singularity Theory is just fundamentally unsound. In particular the "futurism" school which tries to extrapolate existing trends to the future. To me it seems pretty clear that the path of technological development does not follow a predictable roadmap, and the things that seem important now end up being supplanted by unanticipated things, and the things that will seem important in the future will be important in ways that we can't predict. So far I feel this has been borne out as each concrete prediction of people like Kurzweil has been ruled out or seen to be irrelevant or uninteresting, but it seems like the school still exists because it can always offer a post hoc adjustment towards the same end. I have a question for people who believe in Kurzweil's theory -- what would it take to disprove it? To ultimately say "yeah, this was just an incorrect model of the future"?
- marssaxman 2y ago
  I was able to get more excited about these exponential curve models when I was younger, but these days there is always a skeptical voice in the back of my head predicting that a logistic function will prove to be a better fit, considering that we live on a finite planet.
- observationist 2y ago
  You've fundamentally misunderstood what the singularity is. It's when technological progress outpaces the capacity for any person to control or understand what's happening. We get a taste of this, with the internet, for example - no country or person or entity can control what gets published online. We see an enormous amount of bots posting random slop on platforms, spam emails, or outright spam websites being published. We see a huge number of invalid scientific papers published to "legitimate" journals. That information far exceeds the capacity for any person to read the whole, let alone understand or control it. The singularity refers to a point in time specifically when technology is being iterated and developed so rapidly that nobody can know what tomorrow will bring. Irving J Good said it best: "Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make." The singularity is an inevitability. It's a recognition of a fact about the universe. There will come a point in time where humans aren't the top of the food chain anymore. We won't be in control of technological development. It's not any specific set of technologies or a particular threshold of research hours per day or anything which can be easily quantified like that. It's the collective output being greater than the sum of its parts, in ways that are fundamentally unpredictable, with unpredictable consequences. Kurzweil's predictions have been irritatingly accurate. This blog post is irritatingly ignorant. Brains are fleshy computers in bone vats. You simulate your very own Matrix every single conscious moment of your existence. Computation is what brains do. Disputing this is putting ignorance of either brains or computers on display. Computation is sufficient to explaining anything and everything brains do. It's the simplest explanation, it's scientifically coherent, and there is a total absence of any evidence of any phenomena or feature that is unexplained by brains-as-computers. The more we implement functions of intelligence in computer software, the faster technological development occurs, and more of the intellectual development starts to occur in artificial minds.
hyeonwho4 2y ago
I must be missing something. The article claims that $1000 of compute (hardware?) was supposed to surpass an insect brain about 23 years ago, and we hasn't achieved that benchmark yet. But $1000 of time on Claude Opus will buy 13 million tokens of output, or about 52,000 human hours of output content at typical human author writing rates. The content will be well-formatted, logical, and if well-prompted, virtually indistinguishable from that of an above-average human writer. $1000 on DALLE-3 will generate 8300 images, some fraction of which will pass an artistic turing test. And $1000 on AlphaFold will do things that no human can do. So it seems Kurtweil was right on target, and AI did surpass human capabilities around 2023?
- aquilaFiera 2y ago
  In both the article and Kurzweil's case, it just depends on how you want to set up the goal posts (which the article somewhat alludes to). If you want to measure flops and compare that way (which is similar to what you're suggesting), sure, we've surpassed human-level intelligence. If you want to measure capabilities like autonomous navigation (like the article does) there is still a ways to go before we have animal-level capabilities. Both discussions have merit. But it's a question of measurements and goal posts.
- aabhay 2y ago
  Umm.. this is a severe recency bias. What about all the things that humans can do that robots/AI haven’t yet shown promise in at all. Like basically any sculptural art, glass blowing, knot tying, or almost any athletic sport at competitive level like soccer. While you might say “that’s just a hardware problem not an intelligence problem”, you ignore the fact that robotics researchers have been working on this problem for decades and are nowhere near this goal. Even in simulation, we don’t have a compelling mechanism for autonomous completion of a task like driving cross country if it involves fixing a tire. I personally consider the physically embodied tasks extremely challenging for AI because it involves continuous real time sensory integration and extensive nuanced tool use
  - robotresearcher 2y ago
    In the small league of RoboCup, games progress faster than humans can really follow, let alone participate in. Those robots are low mass and sensing is done off-board. But it’s real time. Boy, is it.
natch 2y ago
More like "point missed." The singularity is about brains, not bodies. Of course physical instantiations of embodied agents will lag a very short bit when we first hit AGI.
deleted 2y ago
[deleted]
FrameworkFred 2y ago
I disagree that we're greatly off-target. We can certainly build a grenade-dropper that sometimes picks the wrong target and gets intercepted by hostile actors. We have LLMs that aren't doing some things that we might task a single human brain with. It's really a matter of product, market, fit. When our army of bug-smart drones wipes out civilians and heritage sites, does something cruel or otherwise distasteful, and/or requires a deployment of a million drones to get the job done, folks realize they actually wanted something with human-level capabilities...like remote-controlled drones. When we build an AI chat bot that's often wrong and can't do math all that well, folks complain that they didn't really want any old brain, they want something better. I'm not sure I buy everything Kurzweil's selling TBH, but I don't think this article is making a great argument.
cen4 2y ago
I have been trying to kill a mosquito for about 20 minutes now. Can't believe what those speck of dust brains are doing.
falcor84 2y ago
> So we should expect to be able to have e.g. a combat drone that can take a grenade, navigate for miles to enemy territory, pick on an enemy target, drop the grenade, and navigate back to the base. Call me naive, but I would like to believe that at least part of the reason we don't have these autonomous murder machines is that we choose not to develop them.
uoaei 2y ago
This article highlights a gripe I've had for a while surrounding this conversation: Analog and digital computing may be mathematically equivalent, but the devil is in the implementation, basically destroying any sense of equivalence in "processing power". The modes and methods of processing are so distinct (even if the quantities of "information flow" are equal) that reducing everything to FLOPS is not just irrelevant, it's horribly misleading.

35 points by telotortium 2y ago

What this is really saying is that hardware is a necessary precondition for, but not determinative of, intelligence. If you have a human brain worth of neurons firing randomly you don't get consciousness. The structure and instruction set is crucial. And we are perhaps 20 years away from figuring out how to structure the compute power we have to get the consciousness of a human adult.

throwuxiytayq 2y ago

> So we shouldn't expect human like intelligence before late 2040's. That is of course if we actually get an insect brain this year. Which we won't. Was this written pre-ChatGPT? I am amazed that the author decided that this is an insightful take he'd like to share on the internet. They managed to confuse computational capacity and actual capability, while remaining completely blind to the fact that nobody expects AI development to happen along this sort of curve, or to resemble a walk up the list of intelligent animal species. We didn't expect it 10 years ago, we especially don't expect it now that it's super prominently not happening.

telotortium 2y ago
It does show that Kurzweil's model of AI development is significantly deficient. Kurzweil isn't considered widely discredited in the AGI space, so it's a useful argument.
- timdiggerm 2y ago
  > Kurzweil isn't considered widely discredited in the AGI space Thank you for helping discredit the AGI space

andrewla 2y ago

I'm of the school of thought that says that the Singularity Theory is just fundamentally unsound. In particular the "futurism" school which tries to extrapolate existing trends to the future. To me it seems pretty clear that the path of technological development does not follow a predictable roadmap, and the things that seem important now end up being supplanted by unanticipated things, and the things that will seem important in the future will be important in ways that we can't predict. So far I feel this has been borne out as each concrete prediction of people like Kurzweil has been ruled out or seen to be irrelevant or uninteresting, but it seems like the school still exists because it can always offer a post hoc adjustment towards the same end. I have a question for people who believe in Kurzweil's theory -- what would it take to disprove it? To ultimately say "yeah, this was just an incorrect model of the future"?

marssaxman 2y ago
I was able to get more excited about these exponential curve models when I was younger, but these days there is always a skeptical voice in the back of my head predicting that a logistic function will prove to be a better fit, considering that we live on a finite planet.
observationist 2y ago
You've fundamentally misunderstood what the singularity is. It's when technological progress outpaces the capacity for any person to control or understand what's happening. We get a taste of this, with the internet, for example - no country or person or entity can control what gets published online. We see an enormous amount of bots posting random slop on platforms, spam emails, or outright spam websites being published. We see a huge number of invalid scientific papers published to "legitimate" journals. That information far exceeds the capacity for any person to read the whole, let alone understand or control it. The singularity refers to a point in time specifically when technology is being iterated and developed so rapidly that nobody can know what tomorrow will bring. Irving J Good said it best: "Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make." The singularity is an inevitability. It's a recognition of a fact about the universe. There will come a point in time where humans aren't the top of the food chain anymore. We won't be in control of technological development. It's not any specific set of technologies or a particular threshold of research hours per day or anything which can be easily quantified like that. It's the collective output being greater than the sum of its parts, in ways that are fundamentally unpredictable, with unpredictable consequences. Kurzweil's predictions have been irritatingly accurate. This blog post is irritatingly ignorant. Brains are fleshy computers in bone vats. You simulate your very own Matrix every single conscious moment of your existence. Computation is what brains do. Disputing this is putting ignorance of either brains or computers on display. Computation is sufficient to explaining anything and everything brains do. It's the simplest explanation, it's scientifically coherent, and there is a total absence of any evidence of any phenomena or feature that is unexplained by brains-as-computers. The more we implement functions of intelligence in computer software, the faster technological development occurs, and more of the intellectual development starts to occur in artificial minds.

hyeonwho4 2y ago

I must be missing something. The article claims that $1000 of compute (hardware?) was supposed to surpass an insect brain about 23 years ago, and we hasn't achieved that benchmark yet. But $1000 of time on Claude Opus will buy 13 million tokens of output, or about 52,000 human hours of output content at typical human author writing rates. The content will be well-formatted, logical, and if well-prompted, virtually indistinguishable from that of an above-average human writer. $1000 on DALLE-3 will generate 8300 images, some fraction of which will pass an artistic turing test. And $1000 on AlphaFold will do things that no human can do. So it seems Kurtweil was right on target, and AI did surpass human capabilities around 2023?

aquilaFiera 2y ago
In both the article and Kurzweil's case, it just depends on how you want to set up the goal posts (which the article somewhat alludes to). If you want to measure flops and compare that way (which is similar to what you're suggesting), sure, we've surpassed human-level intelligence. If you want to measure capabilities like autonomous navigation (like the article does) there is still a ways to go before we have animal-level capabilities. Both discussions have merit. But it's a question of measurements and goal posts.
aabhay 2y ago
Umm.. this is a severe recency bias. What about all the things that humans can do that robots/AI haven’t yet shown promise in at all. Like basically any sculptural art, glass blowing, knot tying, or almost any athletic sport at competitive level like soccer. While you might say “that’s just a hardware problem not an intelligence problem”, you ignore the fact that robotics researchers have been working on this problem for decades and are nowhere near this goal. Even in simulation, we don’t have a compelling mechanism for autonomous completion of a task like driving cross country if it involves fixing a tire. I personally consider the physically embodied tasks extremely challenging for AI because it involves continuous real time sensory integration and extensive nuanced tool use
- robotresearcher 2y ago
  In the small league of RoboCup, games progress faster than humans can really follow, let alone participate in. Those robots are low mass and sensing is done off-board. But it’s real time. Boy, is it.

natch 2y ago

More like "point missed." The singularity is about brains, not bodies. Of course physical instantiations of embodied agents will lag a very short bit when we first hit AGI.

deleted 2y ago

[deleted]

FrameworkFred 2y ago

I disagree that we're greatly off-target. We can certainly build a grenade-dropper that sometimes picks the wrong target and gets intercepted by hostile actors. We have LLMs that aren't doing some things that we might task a single human brain with. It's really a matter of product, market, fit. When our army of bug-smart drones wipes out civilians and heritage sites, does something cruel or otherwise distasteful, and/or requires a deployment of a million drones to get the job done, folks realize they actually wanted something with human-level capabilities...like remote-controlled drones. When we build an AI chat bot that's often wrong and can't do math all that well, folks complain that they didn't really want any old brain, they want something better. I'm not sure I buy everything Kurzweil's selling TBH, but I don't think this article is making a great argument.

cen4 2y ago

I have been trying to kill a mosquito for about 20 minutes now. Can't believe what those speck of dust brains are doing.

falcor84 2y ago

> So we should expect to be able to have e.g. a combat drone that can take a grenade, navigate for miles to enemy territory, pick on an enemy target, drop the grenade, and navigate back to the base. Call me naive, but I would like to believe that at least part of the reason we don't have these autonomous murder machines is that we choose not to develop them.

uoaei 2y ago

This article highlights a gripe I've had for a while surrounding this conversation: Analog and digital computing may be mathematically equivalent, but the devil is in the implementation, basically destroying any sense of equivalence in "processing power". The modes and methods of processing are so distinct (even if the quantities of "information flow" are equal) that reducing everything to FLOPS is not just irrelevant, it's horribly misleading.