4 ms·

Despite the cynicism and his black and white predictions, I think his rhetoric still has valuable contributions. It forces others to take a true account of what

Despite the cynicism and his black and white predictions, I think his rhetoric still has valuable contributions. It forces others to take a true account of what intelligence is and what kind of intelligence is AI capable of in the medium term (evidence: this thread). For some reason, this doesn't get enough attention and we have people like Elon and Stephen Hawking making dire predictions all over the place.

ForHackernews 8y ago
I can't believe there are people genuinely afraid of a hypothetical powerful malevolent AI, yet seemingly not that concerned by actual climate change.
- lern_too_spel 8y ago
  Who says you can only worry about one thing?
  - coldtea 8y ago
    Anybody who understands opportunity costs...
    - lern_too_spel 8y ago
      That's reductionist to the point of absurdity. You might be able to only focus on one thing at a time, but a day is long, and you need to worry about multiple things in a day to merely survive. In your free time, it is possible to worry about poverty, climate change, superintelligence, and many other things. The reason that rich people worry about superintelligence is that it could bring the same uncaring devastation to the rich as climate change brings to the poor.
      - coldtea 8y ago
        >The reason that rich people worry about superintelligence is that it could bring the same uncaring devastation to the rich as climate change brings to the poor. The problem with this is that I believe one is a genuine threat, the other is a fad.
        lern_too_spel 8y ago
        > the other is a fad. In what way? Do you not believe that superintelligence is possible, or do you believe that any superintelligence will automatically care about the well-being of humans? Both beliefs seem naive to me and to many luminaries in the field: https://people.eecs.berkeley.edu/~russell/research/future/ https://people.eecs.berkeley.edu/~russell/research/future/.
        coldtea 8y ago
        >Do you not believe that super-intelligence is possible I don't believe super-intelligence is possible. I don't believe we're anywhere near modeling intelligence, and even if we did I don't believe intelligence will "exponentially increase" given more computing power (the same way there's a limit to speeding up barely- or non-parallelizable programs).
        lern_too_spel 8y ago
        > I don't believe super-intelligence is possible. The fact that organizations outperform individuals at many tasks shows that superintelligence is possible. If you can dramatically increase the communication bandwidth of an organization through computerization, you will trivially achieve superintelligence over organizations. Exponential increasing intelligence is not necessary for bad outcomes.
        coldtea 8y ago
        >The fact that organizations outperform individuals at many tasks shows that superintelligence is possible. That's a little hand wavy example. Besides, organizations lose out to individuals all the time where intelligence matters -- e.g. that's why the stupidity of bureaucracy, or the army, and "design by committee" is a thing. Also teams of 5-10 do often do better than teams of 100 or 200 (even in programming), except of course in labor intensive tasks (of course an army of 1000 will defeat 10 people, except if among the ten is Chuck Norris).
  - deleted 8y ago
    [deleted]
- jimbokun 8y ago
  Who are the people who both predict super human intelligent AI and do not believe in human caused climate change?
  - ForHackernews 8y ago
    I didn't say they didn't believe in it, I said they were less concerned about it. https://www.effectivealtruism.org/articles/introduction-to-effective-altruism/ https://www.effectivealtruism.org/articles/introduction-to-e... > Climate change and nuclear war are well-known threats to the long-term survival of our species. Many researchers believe that risks from emerging technologies, such as advanced artificial intelligence and designed pathogens, may be even more worrying. ... > First, you need to consider which problem you should focus on. Some of the most promising problems appear to be safely developing artificial intelligence, improving biosecurity policy, or working toward the end of factory farming.
- sampo 8y ago
  > I can't believe there are people genuinely afraid of a hypothetical powerful malevolent AI I don't think even the AI doomsayers, deep down, actually believe what they preach. It's just a way to signal that one is clever and informed of new tech. If they actually believed what they say, they'd be worried of being targeted by violent protestors, like drug testing companies and crop breeding companies have to be.
dmreedy 8y ago
I disagree. I think his rhetoric feeds the 'we already understand it' mythology around the nature of AI technologies, and worse the state of our own understanding of the system they claim to model, and directly contributes to the harmful and frustrating boom-and-bust cycles that AI goes through. I'd lump him in the same category as Elon Musk and Stephen Hawking in this space; sellers of fantasy. I admire his optimism, but I think it's irresponsible to sell it like he does.
- nyrulez 8y ago
  Marvin Minsky ("Father" of AI at MIT) made the bold claim in 70s that we will be marrying robots by 2000. After it didn't come to fruition, it led most AI researchers to take stock of the situation and realize these goals aren't that trivial. Similarly, if we don't have bold predictions like these that we can actually measure within our lifetimes, we fall prey to fantasies that cannot be measured. Once this prediction fails miserably (I think) it helps many others to re-calibrate all their BS.
  - dmreedy 8y ago
    Perhaps something operational like, "We will marry robots" is measurable. But "outpace human intelligence"? We're arguably incapable of measuring human intelligence right now, much less artificial intelligence, much less comparing between the two. We don't even really have a good operational definition for what the word 'intelligence' means. The closest we have is the Turing Test, which, while pragmatic, does not answer the question, "is this computer smarter than a human", if it answers any question at all. I'd argue that Kurzweil is selling precisely these kinds of fantasies that cannot be measured.
    - doublekill 8y ago
      For a more recent working definition of intelligence, see the work of Marcus Hutter and Shane Legg on Universal Intelligence. That we don't understand or can't define intelligence is a popular trope not grounded in reality. There are entire scientific and well-established fields that study digital and biological intelligence.

ForHackernews 8y ago

I can't believe there are people genuinely afraid of a hypothetical powerful malevolent AI, yet seemingly not that concerned by actual climate change.

lern_too_spel 8y ago
Who says you can only worry about one thing?
- coldtea 8y ago
  Anybody who understands opportunity costs...
  - lern_too_spel 8y ago
    That's reductionist to the point of absurdity. You might be able to only focus on one thing at a time, but a day is long, and you need to worry about multiple things in a day to merely survive. In your free time, it is possible to worry about poverty, climate change, superintelligence, and many other things. The reason that rich people worry about superintelligence is that it could bring the same uncaring devastation to the rich as climate change brings to the poor.
    - coldtea 8y ago
      >The reason that rich people worry about superintelligence is that it could bring the same uncaring devastation to the rich as climate change brings to the poor. The problem with this is that I believe one is a genuine threat, the other is a fad.
      - lern_too_spel 8y ago
        > the other is a fad. In what way? Do you not believe that superintelligence is possible, or do you believe that any superintelligence will automatically care about the well-being of humans? Both beliefs seem naive to me and to many luminaries in the field: https://people.eecs.berkeley.edu/~russell/research/future/ https://people.eecs.berkeley.edu/~russell/research/future/.
        coldtea 8y ago
        >Do you not believe that super-intelligence is possible I don't believe super-intelligence is possible. I don't believe we're anywhere near modeling intelligence, and even if we did I don't believe intelligence will "exponentially increase" given more computing power (the same way there's a limit to speeding up barely- or non-parallelizable programs).
        lern_too_spel 8y ago
        > I don't believe super-intelligence is possible. The fact that organizations outperform individuals at many tasks shows that superintelligence is possible. If you can dramatically increase the communication bandwidth of an organization through computerization, you will trivially achieve superintelligence over organizations. Exponential increasing intelligence is not necessary for bad outcomes.
        coldtea 8y ago
        >The fact that organizations outperform individuals at many tasks shows that superintelligence is possible. That's a little hand wavy example. Besides, organizations lose out to individuals all the time where intelligence matters -- e.g. that's why the stupidity of bureaucracy, or the army, and "design by committee" is a thing. Also teams of 5-10 do often do better than teams of 100 or 200 (even in programming), except of course in labor intensive tasks (of course an army of 1000 will defeat 10 people, except if among the ten is Chuck Norris).
- deleted 8y ago
  [deleted]
jimbokun 8y ago
Who are the people who both predict super human intelligent AI and do not believe in human caused climate change?
- ForHackernews 8y ago
  I didn't say they didn't believe in it, I said they were less concerned about it. https://www.effectivealtruism.org/articles/introduction-to-effective-altruism/ https://www.effectivealtruism.org/articles/introduction-to-e... > Climate change and nuclear war are well-known threats to the long-term survival of our species. Many researchers believe that risks from emerging technologies, such as advanced artificial intelligence and designed pathogens, may be even more worrying. ... > First, you need to consider which problem you should focus on. Some of the most promising problems appear to be safely developing artificial intelligence, improving biosecurity policy, or working toward the end of factory farming.
sampo 8y ago
> I can't believe there are people genuinely afraid of a hypothetical powerful malevolent AI I don't think even the AI doomsayers, deep down, actually believe what they preach. It's just a way to signal that one is clever and informed of new tech. If they actually believed what they say, they'd be worried of being targeted by violent protestors, like drug testing companies and crop breeding companies have to be.

dmreedy 8y ago

I disagree. I think his rhetoric feeds the 'we already understand it' mythology around the nature of AI technologies, and worse the state of our own understanding of the system they claim to model, and directly contributes to the harmful and frustrating boom-and-bust cycles that AI goes through. I'd lump him in the same category as Elon Musk and Stephen Hawking in this space; sellers of fantasy. I admire his optimism, but I think it's irresponsible to sell it like he does.

nyrulez 8y ago
Marvin Minsky ("Father" of AI at MIT) made the bold claim in 70s that we will be marrying robots by 2000. After it didn't come to fruition, it led most AI researchers to take stock of the situation and realize these goals aren't that trivial. Similarly, if we don't have bold predictions like these that we can actually measure within our lifetimes, we fall prey to fantasies that cannot be measured. Once this prediction fails miserably (I think) it helps many others to re-calibrate all their BS.
- dmreedy 8y ago
  Perhaps something operational like, "We will marry robots" is measurable. But "outpace human intelligence"? We're arguably incapable of measuring human intelligence right now, much less artificial intelligence, much less comparing between the two. We don't even really have a good operational definition for what the word 'intelligence' means. The closest we have is the Turing Test, which, while pragmatic, does not answer the question, "is this computer smarter than a human", if it answers any question at all. I'd argue that Kurzweil is selling precisely these kinds of fantasies that cannot be measured.
  - doublekill 8y ago
    For a more recent working definition of intelligence, see the work of Marcus Hutter and Shane Legg on Universal Intelligence. That we don't understand or can't define intelligence is a popular trope not grounded in reality. There are entire scientific and well-established fields that study digital and biological intelligence.