5 ms·

As someone almost completely without knowledge of AI and ML, these are some signs why I'm skeptical of this kind of claims: - Most of the imminent AGI / Singul

by the_af 5y ago

As someone almost completely without knowledge of AI and ML, these are some signs why I'm skeptical of this kind of claims: - Most of the imminent AGI / Singularity / Robot Apocalypse stuff seems to come, with few exceptions, not from practitioners or computer scientists specialized in AI, but from "visionaries" (in the best case), internet celebrities, people with unrelated areas of expertise, or downright cranks (who are self-proclaimed experts) such as Yudkowsky. - The assertion that "a lot of effort/investment is going into this, so it will happen" begs the question that "this" is at all possible. If something is a dead end, no amount of investment and attention is going to bring it into existence. Quoting the article, "with this much distributed attention fixed on the problem, AGI will be solved" is not at all a given. - Where are all the AI/ML practitioners, i.e. people who don't make a living out of predicting The End of the World, and with actual subject-matter achievements, predicting the Singularity and the Robot Apocalypse?

mountainriver 5y ago
The trajectory of progress gives evidence that it is in fact possible and likely. Any venture into the unknown could be unsuccessful but if you see progress you can start to make estimates. And yes most of the “robots will kill us” comes from people who aren’t building the algorithms. This could be biased in people not thinking their work is harmful but is most likely that once you see how the sausage is made you are less worried about it.
- hn_throwaway_99 5y ago
  > The trajectory of progress gives evidence that it is in fact possible and likely 100% disagree. In fact, I'd argue that the opposite is often true, where you see initially a fast rate of progress that results in diminishing returns over time. It's like estimating that a kid who was 3 feet tall at age 5 and 4 feet tall at age 10 will be 10 feet tall at age 40. I have very strong skepticism of any sort of hand-wavy "Look, we've made some progress, so it's highly likely we'll eventually cross some threshold that results in infinite progress."
  - thirdwhrldPzz 5y ago
    Pareto principle; we get 80 percent there and that last 20 becomes the new 100. We keep diving into one infinitely big little number pattern fractal after another, chasing poetry to alleviate the existential dread of biological death. The idea we can fundamentally hang the churn the universe given the vastness of its mass and unseen churn is pretty funny to me. Information may be forever but without the right sorting method you can’t reconstruct it once scattered. Ah, our delusions of permanence.
mkaic 5y ago
Hi! Author here. I think you raise some great points! I'll address them each: -- I am a professional AI practitioner. I work in the field of medical deep learning and love the field. I am strongly considering starting a few experimental forays into some of the concepts I mentioned in my post as side projects, especially self-modifying model architectures. -- Yes, you're totally right! I am making the fundamental assumption that it is possible. My reasoning is based on my belief that human behavior is simply a function of the many sensory and environmental inputs we experience. Since neural networks can approximate any function, I believe they can approximate general human behavior. -- This is fair. The topic of the singularity is often used for traffic/fame (I mean, I'm guilty of this myself with this very post, though I hope I still managed to spark some productive discourse) and so there are always conflicts of interest to take into account. I can't name any examples off the top of my head that perfectly fit your criteria, but depending on how much you trust him, Elon Musk seems to be genuinely concerned about the potential for a malevolent singularity. Thank you so much for your comment! I really appreciate your feedback. Have a wonderful day.
- jonny_eh 5y ago
  ^ To be clear, mkaic is the original author of the article.
  - mkaic 5y ago
    Oh, whoops, I'll edit the comment to make it more clear. Thanks for reminding me.
- soVeryTired 5y ago
  Cubic splines can approximate any function too, so the universality argument is a little weak IMO. Even if one buys into the idea that human behaviour is a 'function' of sensory and environmental 'inputs', that's a long way from showing a neural net a million different texts and asking it to generalise.
  - mkaic 5y ago
    I think the first AGI to pass a Turing test will probably be a simple language model. I don't think it will look like any of the GPTs, but I think text completion is a great starting point. I'm not sure how other inputs will be added into the mix, but I am sure that they will be -- heck, maybe once we train a general language model, it may very well just tell us how to incorporate things like video, audio, haptics, gyro data, etc into its architecture.
hyperpallium2 5y ago
Vernor Vinge coined the term, and he was a computer scientist (though more famous as a science fiction writer, a profession which I guess TBF makes money from visions...). An exponential looks the same whereever you are on it, so arguably we are in the singularity now, and have been for quite some time... "Singularity" is a terrible term for an exponential. It's meant to convey that we can't predict what's next... which has always been the case. The problem of predicting an exponentialling expanding search space of arbitrary conplexity is that it gets big fast. It also means each little 'bit' of information allows you to see a little further, sometimes revealing things you could never have imagined (because you couldn't see that far before).
- jasonwatkinspdx 5y ago
  But the above comment's entire point is there's zero reason for us to assume we're on an unbounded exponential vs a sigmoid.
  - unoti 5y ago
    > But the above comment's entire point is there's zero reason for us to assume we're on an unbounded exponential vs a sigmoid. Something about that which is discussed at length in "The Singularity is Near" is the idea that the unbounded exponential is actually comprised of many smaller sigmoids. Technological progress looks like sigmoids locally-- for example you have the industrial revolution, the cotton gin, steam power, etc. all looking like a sigmoids of the exploitation of automation. At some point you get all those benefits and progress starts to level off like a sigmoids. Then another sigmoid comes along when you get electricity, and a new sigmoid starts. Then later we get computers, and that starts to level off, then networking comes along and we get a new sigmoid. Then deep learning... The AI winters were the levelling off of sigmoids by one way of thinking. And maybe we're tapering off on our current round of what we can do with existing architectures.
    - ImaCake 5y ago
      I’m no expert, just a humble biostatistics student, generally when you sum together a lot of random variables following a specific distribution you end up with basically the same distribution (scaled by the N random vars). So a lot of sigmoids put together (e.g. covid spread) will still eventually be a sigmoid. Biology seems to run on sigmoids that at first look like exponentials.
gorwell 5y ago
Yudkowsky doesn't strike me as a crank. Why do you say that?
- ForHackernews 5y ago
  What has he ever done in AI except talk about it? At best he's an AI promoter, and hype-men are often cranks or scammers (see also: VR, web3, cryptocurrencies, MLM)
  - marvin 5y ago
    Yudkowsky is a philosopher, in the sense of someone who thinks a lot about things that haven't been achieved yet. Lots of otherwise smart people (wrongly!) discount the value of philosophy, but it's close by every time there's a paradigm shift in humanity's knowledge. Philosophers can be scientists and vice versa. If anything, I'm surprised that this philosophy isn't mentioned more in a thread where the author gleefully talks about ML being used to create better AI, layer by layer until the thing is even more opaque than what we're currently working with. This is terrifying, as we currently have only very loose ideas about how to reliably ensure that a powerful reinforcement learning system doesn't accidentally optimize for something we don't want. The current paradigm is "turn it off", which works well for now but seems like a fragile shield long-term.
    - the_af 5y ago
      > Yudkowsky is a philosopher At least inasmuch as anyone who thinks about stuff can be considered a philosopher. But he strikes me much more as a self-appointed expert on matters where he shows no achievements. He writes fanfiction about AI rather than actually doing stuff with AI.
      - psyc 5y ago
        Most people who think about stuff don’t leave a mountain of highly organized and entertaining essays for posterity.
hither_shores 5y ago
> Where are all the AI/ML practitioners, i.e. people who don't make a living out of predicting The End of the World, and with actual subject-matter achievements, predicting the Singularity and the Robot Apocalypse? The answer is in the question: they're spending most of their time doing AI research or ML work, whereas the internet celebrities who write most of what you read spend most of their time getting you to read what they write.
0xBABAD00C 5y ago
> Most of the imminent AGI / Singularity / Robot Apocalypse stuff seems to come, with few exceptions, not from practitioners or computer scientists specialized in AI I'm a practitioner at a top company for many years now, and I think we're getting close to a point of no return. I think best case scenario is some form of unrecognizably transformed "humanity", worst case is too horrible to even spell it out. The reason why this is not the consensus among experts has to do with a combination of blind spots and biases. Humans are especially bad at (a) evaluating / reflecting on themselves, and (b) extrapolating, especially with non-linear processes.
- mensetmanusman 5y ago
  That’s what experts in fusion have been saying for 50 years though… now that Moore’s law has ended, it might be a long while…
- breuleux 5y ago
  I have seldom seen predictions about technology come to fruition at the predicted date. If they come at all (they usually don't), it's usually much later. Sure, some naysayers might have said "we will never solve chess/go/etc." but I think they were overall fewer than the people who thought it was right around the corner (and it was more of a philosophical argument than a scientific one). Totally unpredicted advances coming out of the blue, yeah, that happens often. But as far as AGI goes, it's been predicted pretty much every year since the 70s. Being too conservative about extrapolation doesn't seem to be our problem. If anything, I think we're overeager.
lalaithion 5y ago
In 2019, 32 AI experts participated in a survey on AGI timing: 45% of respondents predict a date before 2060 34% of all participants predicted a date after 2060 21% of participants predicted that singularity will never occur. https://research.aimultiple.com/artificial-general-intelligence-singularity-timing/ https://research.aimultiple.com/artificial-general-intellige...
- throwawaygh 5y ago
  Why 2060? I'd LOVE to see the ages of the respondents in each group. (I have a phd in CS and am, by most reasonable definitions, an "AI Expert". Whatever the hell that means. I've been a respondent in very similar surveys run by PIs at fancy universities and so on. These responses are always wild ass guesses and should be totally ignored. I've even left a comment to this effect on one such survey.)
- 7373737373 5y ago
  The prediction contest here has a peak at around 2042: https://www.metaculus.com/questions/3479/when-will-the-first-artificial-general-intelligence-system-be-devised-tested-and-publicly-known-of/ https://www.metaculus.com/questions/3479/when-will-the-first...
Tepix 5y ago
Ray Kurzweil works on AI at Google and he wrote The Book about the coming singularity.
- the_af 5y ago
  I consider Kurzweil unreliable on this subject, as well as on the similarly kooky topic of immortality. He belongs to the "internet celebrity"/"guru" cadre rather than to the scientific community doing serious research on AI. In fact, his actual subject matter expertise isn't in AI. He is a "futurist", not a real researcher. I was definitely thinking of Kurzweil, not only Yudkowsky, when mentioning internet cranks. Working for Google is not enough. It's like when someone who's an accomplished physicist decides to give their opinions on life, biology, evolution, etc: not their area of expertise, so we don't need to hold their opinions in particularly high regard.

by the_af 5y ago

mountainriver 5y ago

The trajectory of progress gives evidence that it is in fact possible and likely. Any venture into the unknown could be unsuccessful but if you see progress you can start to make estimates. And yes most of the “robots will kill us” comes from people who aren’t building the algorithms. This could be biased in people not thinking their work is harmful but is most likely that once you see how the sausage is made you are less worried about it.

hn_throwaway_99 5y ago
> The trajectory of progress gives evidence that it is in fact possible and likely 100% disagree. In fact, I'd argue that the opposite is often true, where you see initially a fast rate of progress that results in diminishing returns over time. It's like estimating that a kid who was 3 feet tall at age 5 and 4 feet tall at age 10 will be 10 feet tall at age 40. I have very strong skepticism of any sort of hand-wavy "Look, we've made some progress, so it's highly likely we'll eventually cross some threshold that results in infinite progress."
- thirdwhrldPzz 5y ago
  Pareto principle; we get 80 percent there and that last 20 becomes the new 100. We keep diving into one infinitely big little number pattern fractal after another, chasing poetry to alleviate the existential dread of biological death. The idea we can fundamentally hang the churn the universe given the vastness of its mass and unseen churn is pretty funny to me. Information may be forever but without the right sorting method you can’t reconstruct it once scattered. Ah, our delusions of permanence.

mkaic 5y ago

Hi! Author here. I think you raise some great points! I'll address them each: -- I am a professional AI practitioner. I work in the field of medical deep learning and love the field. I am strongly considering starting a few experimental forays into some of the concepts I mentioned in my post as side projects, especially self-modifying model architectures. -- Yes, you're totally right! I am making the fundamental assumption that it is possible. My reasoning is based on my belief that human behavior is simply a function of the many sensory and environmental inputs we experience. Since neural networks can approximate any function, I believe they can approximate general human behavior. -- This is fair. The topic of the singularity is often used for traffic/fame (I mean, I'm guilty of this myself with this very post, though I hope I still managed to spark some productive discourse) and so there are always conflicts of interest to take into account. I can't name any examples off the top of my head that perfectly fit your criteria, but depending on how much you trust him, Elon Musk seems to be genuinely concerned about the potential for a malevolent singularity. Thank you so much for your comment! I really appreciate your feedback. Have a wonderful day.

jonny_eh 5y ago
^ To be clear, mkaic is the original author of the article.
- mkaic 5y ago
  Oh, whoops, I'll edit the comment to make it more clear. Thanks for reminding me.
soVeryTired 5y ago
Cubic splines can approximate any function too, so the universality argument is a little weak IMO. Even if one buys into the idea that human behaviour is a 'function' of sensory and environmental 'inputs', that's a long way from showing a neural net a million different texts and asking it to generalise.
- mkaic 5y ago
  I think the first AGI to pass a Turing test will probably be a simple language model. I don't think it will look like any of the GPTs, but I think text completion is a great starting point. I'm not sure how other inputs will be added into the mix, but I am sure that they will be -- heck, maybe once we train a general language model, it may very well just tell us how to incorporate things like video, audio, haptics, gyro data, etc into its architecture.

hyperpallium2 5y ago

Vernor Vinge coined the term, and he was a computer scientist (though more famous as a science fiction writer, a profession which I guess TBF makes money from visions...). An exponential looks the same whereever you are on it, so arguably we are in the singularity now, and have been for quite some time... "Singularity" is a terrible term for an exponential. It's meant to convey that we can't predict what's next... which has always been the case. The problem of predicting an exponentialling expanding search space of arbitrary conplexity is that it gets big fast. It also means each little 'bit' of information allows you to see a little further, sometimes revealing things you could never have imagined (because you couldn't see that far before).

jasonwatkinspdx 5y ago
But the above comment's entire point is there's zero reason for us to assume we're on an unbounded exponential vs a sigmoid.
- unoti 5y ago
  > But the above comment's entire point is there's zero reason for us to assume we're on an unbounded exponential vs a sigmoid. Something about that which is discussed at length in "The Singularity is Near" is the idea that the unbounded exponential is actually comprised of many smaller sigmoids. Technological progress looks like sigmoids locally-- for example you have the industrial revolution, the cotton gin, steam power, etc. all looking like a sigmoids of the exploitation of automation. At some point you get all those benefits and progress starts to level off like a sigmoids. Then another sigmoid comes along when you get electricity, and a new sigmoid starts. Then later we get computers, and that starts to level off, then networking comes along and we get a new sigmoid. Then deep learning... The AI winters were the levelling off of sigmoids by one way of thinking. And maybe we're tapering off on our current round of what we can do with existing architectures.
  - ImaCake 5y ago
    I’m no expert, just a humble biostatistics student, generally when you sum together a lot of random variables following a specific distribution you end up with basically the same distribution (scaled by the N random vars). So a lot of sigmoids put together (e.g. covid spread) will still eventually be a sigmoid. Biology seems to run on sigmoids that at first look like exponentials.

gorwell 5y ago

Yudkowsky doesn't strike me as a crank. Why do you say that?

ForHackernews 5y ago
What has he ever done in AI except talk about it? At best he's an AI promoter, and hype-men are often cranks or scammers (see also: VR, web3, cryptocurrencies, MLM)
- marvin 5y ago
  Yudkowsky is a philosopher, in the sense of someone who thinks a lot about things that haven't been achieved yet. Lots of otherwise smart people (wrongly!) discount the value of philosophy, but it's close by every time there's a paradigm shift in humanity's knowledge. Philosophers can be scientists and vice versa. If anything, I'm surprised that this philosophy isn't mentioned more in a thread where the author gleefully talks about ML being used to create better AI, layer by layer until the thing is even more opaque than what we're currently working with. This is terrifying, as we currently have only very loose ideas about how to reliably ensure that a powerful reinforcement learning system doesn't accidentally optimize for something we don't want. The current paradigm is "turn it off", which works well for now but seems like a fragile shield long-term.
  - the_af 5y ago
    > Yudkowsky is a philosopher At least inasmuch as anyone who thinks about stuff can be considered a philosopher. But he strikes me much more as a self-appointed expert on matters where he shows no achievements. He writes fanfiction about AI rather than actually doing stuff with AI.
    - psyc 5y ago
      Most people who think about stuff don’t leave a mountain of highly organized and entertaining essays for posterity.

hither_shores 5y ago

> Where are all the AI/ML practitioners, i.e. people who don't make a living out of predicting The End of the World, and with actual subject-matter achievements, predicting the Singularity and the Robot Apocalypse? The answer is in the question: they're spending most of their time doing AI research or ML work, whereas the internet celebrities who write most of what you read spend most of their time getting you to read what they write.

0xBABAD00C 5y ago

> Most of the imminent AGI / Singularity / Robot Apocalypse stuff seems to come, with few exceptions, not from practitioners or computer scientists specialized in AI I'm a practitioner at a top company for many years now, and I think we're getting close to a point of no return. I think best case scenario is some form of unrecognizably transformed "humanity", worst case is too horrible to even spell it out. The reason why this is not the consensus among experts has to do with a combination of blind spots and biases. Humans are especially bad at (a) evaluating / reflecting on themselves, and (b) extrapolating, especially with non-linear processes.

mensetmanusman 5y ago
That’s what experts in fusion have been saying for 50 years though… now that Moore’s law has ended, it might be a long while…
breuleux 5y ago
I have seldom seen predictions about technology come to fruition at the predicted date. If they come at all (they usually don't), it's usually much later. Sure, some naysayers might have said "we will never solve chess/go/etc." but I think they were overall fewer than the people who thought it was right around the corner (and it was more of a philosophical argument than a scientific one). Totally unpredicted advances coming out of the blue, yeah, that happens often. But as far as AGI goes, it's been predicted pretty much every year since the 70s. Being too conservative about extrapolation doesn't seem to be our problem. If anything, I think we're overeager.

lalaithion 5y ago

In 2019, 32 AI experts participated in a survey on AGI timing: 45% of respondents predict a date before 2060 34% of all participants predicted a date after 2060 21% of participants predicted that singularity will never occur. https://research.aimultiple.com/artificial-general-intelligence-singularity-timing/ https://research.aimultiple.com/artificial-general-intellige...

throwawaygh 5y ago
Why 2060? I'd LOVE to see the ages of the respondents in each group. (I have a phd in CS and am, by most reasonable definitions, an "AI Expert". Whatever the hell that means. I've been a respondent in very similar surveys run by PIs at fancy universities and so on. These responses are always wild ass guesses and should be totally ignored. I've even left a comment to this effect on one such survey.)
7373737373 5y ago
The prediction contest here has a peak at around 2042: https://www.metaculus.com/questions/3479/when-will-the-first-artificial-general-intelligence-system-be-devised-tested-and-publicly-known-of/ https://www.metaculus.com/questions/3479/when-will-the-first...

Tepix 5y ago

Ray Kurzweil works on AI at Google and he wrote The Book about the coming singularity.

the_af 5y ago
I consider Kurzweil unreliable on this subject, as well as on the similarly kooky topic of immortality. He belongs to the "internet celebrity"/"guru" cadre rather than to the scientific community doing serious research on AI. In fact, his actual subject matter expertise isn't in AI. He is a "futurist", not a real researcher. I was definitely thinking of Kurzweil, not only Yudkowsky, when mentioning internet cranks. Working for Google is not enough. It's like when someone who's an accomplished physicist decides to give their opinions on life, biology, evolution, etc: not their area of expertise, so we don't need to hold their opinions in particularly high regard.