4 ms·

Yeah. Maciej wrote a pretty good piece rebutting AI alarmism and kind of alludes to that as one of several points. http://idlewords.com/talks/superintelligence

Yeah. Maciej wrote a pretty good piece rebutting AI alarmism and kind of alludes to that as one of several points. http://idlewords.com/talks/superintelligence.htm http://idlewords.com/talks/superintelligence.htm > With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation. and > But the hard takeoff scenario requires that there be a feature of the AI algorithm that can be repeatedly optimized to make the AI better at self-improvement. And so on. It's a good read.

5_minutes 9y ago
Thanks for this
westoncb 9y ago
Similarly, I wonder if it's something where adding to it doesn't actually do much, like all the variations you see done on Turing Machines (adding more tapes or 2d tapes etc.) which seem like they would increase its capacity somehow, but it turns out they have the same power as the original Turing Machine. Or like the arithmetic of infinite numbers, e.g. aleph-0 + aleph-0 = aleph-0.
spynxic 9y ago
> With no way to define intelligence (except just pointing to ourselves.. ..and anything else considered to be alive) Not to make an attempt at arguing whether or not machines can be intelligent but machines are not submerged in an environment that effects them, giving them no merit to be intelligent about anything; and even if they were they still have no ability to influence the environment. (Sensors for example) It seems many that speak on AI have completely disregarded coming up with a reasonable implementation of intelligence in favor of pondering its potential uses. (Typically in ideals central to humans)
- xarball 9y ago
  > machines are not submerged in an environment that effects them, giving them no merit to be intelligent about anything You're alluding to the problem of having to 'raise' a super-intelligence with a vested interest in the continuity of its physical existence. I think I agree with where you're going with that. Our understanding of 'self' or 'self preservation' likely comes from a long history of suffering, pain, and failure brought on by trials at understanding the self from evolution's attempts at imbuing the information for longevity into cognition. Mother nature has had years of exponential growth with an inconceivable number of branches attempting to understand the 'self', that lead to failure, all necessary because the idea of 'self preservation' or 'grounding one's sense of ego into a concept of physical sensory data' is very obscure to precisely define, let along to define such that it can be TESTED between generations. Based on how much of a headstart evolution has on that refining process, and how bad humans are at implementing that or even exposing sensory information about that that plug directly into concepts that necessarily affect an artificial's sense of self, I'm willing to bet that we are completely ill-equipped to grow any real intelligence that's truly grounded in the real world, until it can make all the same failed attempts at living longer or evolving + testing those attempts, with that exact same directional goal of 'self-preservation' which humans have. Self-preservation is really the only thing you can evolve to, to guide a physical evolution. I'm not aware of any other physical goals that could lead to the same human-like intelligence or human-like neural models. Most creatures seem to hit the exact same point. An agent-based survival of the fittest with generational physical trials that punish poor understandings of the self's physical manifestation, trying to optimize for its longevity. Anything that doesn't hit that kind of model is strictly not aware of the manifestation of its ego, as we know it (e.g. plants, bacteria, etc), and I don't think the physical sense of self can evolve into grounded concepts without allowing for generational failure on the understanding of self, due to incorrect concepts of 'physical self' between competing generations. TL;DR - The concept of physical self is an evolved trait only if there is a goal tied to it, it cannot arrive without the concept of 'self existence' being threatened or when not allowed to fail.
apsec112 9y ago
Rebuttal to that essay by the Machine Intelligence Research Institute (AI safety group): https://intelligence.org/2017/01/13/response-to-ceglowski-on-superintelligence/ https://intelligence.org/2017/01/13/response-to-ceglowski-on...
- crooked-v 9y ago
  That write-up seems to mainly try to shrug off the "what is intelligence" issue by virtue of "maximization", while ignoring that when dealing with human intelligence, selecting for maximization of a single field gets you autistic savants, not Lex Luthor-grade manipulative geniuses.
- SiVal 9y ago
  My own rebuttal to that unimpressive Maciej argument is that he merely listed many ways in which something that seemed as though it ought to work didn't work out at all as expected, so we can safely assume that something as challenging as "superintelligence" won't work out either, and is therefore no danger. But the argument that there are countless ways for plans to go awry and instead cause something unexpected to happen is hardly reassuring evidence that there is, therefore, "nothing to worry about".
saulrh 9y ago
You're betting everything on vacuous what-ifs. I mean, let's turn this around: With no way to define intelligence (except by pointing to ourselves), we don't even know if we're good at it. For all we know human intelligence is a dead end. Maybe any entity that is not a product of evolution, free of genetic detritus that becomes maladaptive around advanced technology, will outstrip human civilization like human civilization outstripped evolution. I don't doubt the book is full of entertaining soundbytes, but given these two quotes I am unconvinced that it is any more than entertainment.
sampo 9y ago
> Maybe any entity significantly smarter than a human being would be crippled by existential despair This was addressed in a Spider-Man story where a villain enhances his intelligence, then first becomes a very successful crime boss, then writes novels and music, and then falls into existential despair. https://en.wikipedia.org/wiki/Flowers_for_Rhino https://en.wikipedia.org/wiki/Flowers_for_Rhino
- kobeya 9y ago
  I think it was done first, and probably better, in Flowers for Algernon.
  - perl4ever 9y ago
    "it is a pastiche of the science fiction story Flowers for Algernon."
  - kwhitefoot 9y ago
    Just in case anyone her is unfamiliar with the story here is a link to the Wikipedia synopsis: https://en.wikipedia.org/wiki/Flowers_for_Algernon https://en.wikipedia.org/wiki/Flowers_for_Algernon. Daniel Keyes should be more famous and the story more widely known. And now for something weird: http://www.unboundworlds.com/2008/10/challenged-and-banned-flowers-for-algernon/ http://www.unboundworlds.com/2008/10/challenged-and-banned-f... How anyone could think of banning one of the most affecting and humane books ever written is beyond me.
felipemnoa 9y ago
>>> With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation.<< The only intelligence that I believe is important is problem solving. If an AI is as capable as humans at problem solving then that is all you need. >>crippled by existential despair Sounds to me that you are talking about feelings. I really do not see the purpose of purposely giving the AI feelings. A strong AI should not have any feelings just like your google maps application does not have any feelings when giving you directions. A general AI will simply be a general problem solver at the same level of a human but with a virtual infinite capacity to scale its computational abilities. You could definitely give it free will so that it decides its own goals but why would anybody do something like that unless they want it to turn on them? It would be like somebody launching a nuclear ballistic missile and letting it decide which city to land in. It is possible to program that type of free will but why? It could decide to destroy your city.
- perl4ever 9y ago
  Feelings regulate actions. People don't blindly follow rules that would lead to disaster because of more fundamental regulatory mechanisms such as feelings. In order to make an AI that is as flexible as a smart human, it can't be brittle, and therefore will need ways to break out of logical traps. That is what a general problem solver must be like, inherently.
  - deleted 9y ago
    [deleted]
  - therein 9y ago
    Very interesting point about using feelings and intuition to break out of logical traps. Does that also mean this AI will be fallible or perhaps even stubborn? Will it ever refuse to accept that it is wrong about a result it produces?
  - felipemnoa 9y ago
doktrin 9y ago
> pretty good piece rebutting AI alarmism I only read the text, but I was under the impression it was mainly meant to be a funny Wat-style talk and not a meaningful rebuttal. If, in fact, this piece is intended as a serious rebuttal then I honestly think it fails. The overwhelming majority of the content is either a funny whataboutism (e.g. Gilligan's Island, pot smoking roommate, the Great Emu War, etc.), or jokes about what could be loosely termed the "AI community". AI 'alarmism', as I understand it, is not predicated on the idea that super-AI is a certainty - merely that it's a possibility. No point raised in this piece gives me a reason to believe that AGI (or whatever) is either impossible or so unlikely that it can be disregarded.
CrystalGamma 9y ago
Where did you get that URL? It's not linked on http://idlewords.com/talks/ http://idlewords.com/talks/ …

5_minutes 9y ago

Thanks for this

westoncb 9y ago

Similarly, I wonder if it's something where adding to it doesn't actually do much, like all the variations you see done on Turing Machines (adding more tapes or 2d tapes etc.) which seem like they would increase its capacity somehow, but it turns out they have the same power as the original Turing Machine. Or like the arithmetic of infinite numbers, e.g. aleph-0 + aleph-0 = aleph-0.

spynxic 9y ago

> With no way to define intelligence (except just pointing to ourselves.. ..and anything else considered to be alive) Not to make an attempt at arguing whether or not machines can be intelligent but machines are not submerged in an environment that effects them, giving them no merit to be intelligent about anything; and even if they were they still have no ability to influence the environment. (Sensors for example) It seems many that speak on AI have completely disregarded coming up with a reasonable implementation of intelligence in favor of pondering its potential uses. (Typically in ideals central to humans)

xarball 9y ago
> machines are not submerged in an environment that effects them, giving them no merit to be intelligent about anything You're alluding to the problem of having to 'raise' a super-intelligence with a vested interest in the continuity of its physical existence. I think I agree with where you're going with that. Our understanding of 'self' or 'self preservation' likely comes from a long history of suffering, pain, and failure brought on by trials at understanding the self from evolution's attempts at imbuing the information for longevity into cognition. Mother nature has had years of exponential growth with an inconceivable number of branches attempting to understand the 'self', that lead to failure, all necessary because the idea of 'self preservation' or 'grounding one's sense of ego into a concept of physical sensory data' is very obscure to precisely define, let along to define such that it can be TESTED between generations. Based on how much of a headstart evolution has on that refining process, and how bad humans are at implementing that or even exposing sensory information about that that plug directly into concepts that necessarily affect an artificial's sense of self, I'm willing to bet that we are completely ill-equipped to grow any real intelligence that's truly grounded in the real world, until it can make all the same failed attempts at living longer or evolving + testing those attempts, with that exact same directional goal of 'self-preservation' which humans have. Self-preservation is really the only thing you can evolve to, to guide a physical evolution. I'm not aware of any other physical goals that could lead to the same human-like intelligence or human-like neural models. Most creatures seem to hit the exact same point. An agent-based survival of the fittest with generational physical trials that punish poor understandings of the self's physical manifestation, trying to optimize for its longevity. Anything that doesn't hit that kind of model is strictly not aware of the manifestation of its ego, as we know it (e.g. plants, bacteria, etc), and I don't think the physical sense of self can evolve into grounded concepts without allowing for generational failure on the understanding of self, due to incorrect concepts of 'physical self' between competing generations. TL;DR - The concept of physical self is an evolved trait only if there is a goal tied to it, it cannot arrive without the concept of 'self existence' being threatened or when not allowed to fail.

apsec112 9y ago

Rebuttal to that essay by the Machine Intelligence Research Institute (AI safety group): https://intelligence.org/2017/01/13/response-to-ceglowski-on-superintelligence/ https://intelligence.org/2017/01/13/response-to-ceglowski-on...

crooked-v 9y ago
That write-up seems to mainly try to shrug off the "what is intelligence" issue by virtue of "maximization", while ignoring that when dealing with human intelligence, selecting for maximization of a single field gets you autistic savants, not Lex Luthor-grade manipulative geniuses.
SiVal 9y ago
My own rebuttal to that unimpressive Maciej argument is that he merely listed many ways in which something that seemed as though it ought to work didn't work out at all as expected, so we can safely assume that something as challenging as "superintelligence" won't work out either, and is therefore no danger. But the argument that there are countless ways for plans to go awry and instead cause something unexpected to happen is hardly reassuring evidence that there is, therefore, "nothing to worry about".

saulrh 9y ago

You're betting everything on vacuous what-ifs. I mean, let's turn this around: With no way to define intelligence (except by pointing to ourselves), we don't even know if we're good at it. For all we know human intelligence is a dead end. Maybe any entity that is not a product of evolution, free of genetic detritus that becomes maladaptive around advanced technology, will outstrip human civilization like human civilization outstripped evolution. I don't doubt the book is full of entertaining soundbytes, but given these two quotes I am unconvinced that it is any more than entertainment.

sampo 9y ago

> Maybe any entity significantly smarter than a human being would be crippled by existential despair This was addressed in a Spider-Man story where a villain enhances his intelligence, then first becomes a very successful crime boss, then writes novels and music, and then falls into existential despair. https://en.wikipedia.org/wiki/Flowers_for_Rhino https://en.wikipedia.org/wiki/Flowers_for_Rhino

kobeya 9y ago
I think it was done first, and probably better, in Flowers for Algernon.
- perl4ever 9y ago
  "it is a pastiche of the science fiction story Flowers for Algernon."
- kwhitefoot 9y ago
  Just in case anyone her is unfamiliar with the story here is a link to the Wikipedia synopsis: https://en.wikipedia.org/wiki/Flowers_for_Algernon https://en.wikipedia.org/wiki/Flowers_for_Algernon. Daniel Keyes should be more famous and the story more widely known. And now for something weird: http://www.unboundworlds.com/2008/10/challenged-and-banned-flowers-for-algernon/ http://www.unboundworlds.com/2008/10/challenged-and-banned-f... How anyone could think of banning one of the most affecting and humane books ever written is beyond me.

felipemnoa 9y ago

>>> With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation.<< The only intelligence that I believe is important is problem solving. If an AI is as capable as humans at problem solving then that is all you need. >>crippled by existential despair Sounds to me that you are talking about feelings. I really do not see the purpose of purposely giving the AI feelings. A strong AI should not have any feelings just like your google maps application does not have any feelings when giving you directions. A general AI will simply be a general problem solver at the same level of a human but with a virtual infinite capacity to scale its computational abilities. You could definitely give it free will so that it decides its own goals but why would anybody do something like that unless they want it to turn on them? It would be like somebody launching a nuclear ballistic missile and letting it decide which city to land in. It is possible to program that type of free will but why? It could decide to destroy your city.

perl4ever 9y ago
Feelings regulate actions. People don't blindly follow rules that would lead to disaster because of more fundamental regulatory mechanisms such as feelings. In order to make an AI that is as flexible as a smart human, it can't be brittle, and therefore will need ways to break out of logical traps. That is what a general problem solver must be like, inherently.
- deleted 9y ago
  [deleted]
- therein 9y ago
  Very interesting point about using feelings and intuition to break out of logical traps. Does that also mean this AI will be fallible or perhaps even stubborn? Will it ever refuse to accept that it is wrong about a result it produces?
- felipemnoa 9y ago

doktrin 9y ago

> pretty good piece rebutting AI alarmism I only read the text, but I was under the impression it was mainly meant to be a funny Wat-style talk and not a meaningful rebuttal. If, in fact, this piece is intended as a serious rebuttal then I honestly think it fails. The overwhelming majority of the content is either a funny whataboutism (e.g. Gilligan's Island, pot smoking roommate, the Great Emu War, etc.), or jokes about what could be loosely termed the "AI community". AI 'alarmism', as I understand it, is not predicated on the idea that super-AI is a certainty - merely that it's a possibility. No point raised in this piece gives me a reason to believe that AGI (or whatever) is either impossible or so unlikely that it can be disregarded.

CrystalGamma 9y ago

Where did you get that URL? It's not linked on http://idlewords.com/talks/ http://idlewords.com/talks/ …