4 ms·

Ugh. This is a pretty bad article. And don't get me wrong... I'm no "AI Alarmist." But to me, this article is equally as useless as the various pronouncements i

Ugh. This is a pretty bad article. And don't get me wrong... I'm no "AI Alarmist." But to me, this article is equally as useless as the various pronouncements it seems to be trying to refute. There's a lot of verbiage here, but no cohesive, coherent argument to support the thesis that "Superhuman AI is just a myth" (which is to say, "Superhuman AI is not possible"). The reality is, there are a lot of unknowns in this space, and to dismiss the possibility of an "evil super-intelligence" out-of-hand is, IMO, every bit as silly as buying into the "OMG, AI is unleashing the demon" fear-mongering. I just think it's too early to know for sure the answers to some of these questions, like: 1. Is AGI possible at all (for whatever definition of "AGI" you want to use)? 2. If the answer to (1) is "yes", then what about ASI (Artificial Super Intelligence) - is that possible? 3. If (1) and (2) are both "yes", then is it possible to create an ASI that will either be benevolent, or controllable, or both? And so on... we just don't know yet. And that in and of itself arguably suggests the exercise of a certain measure of caution.

enord 5y ago
You're making an argument that is already in TFA. It's in addition -and only partially related to- the main point which is: The arguments that are commonly used to demonstrate the possibility or threat of "ASI" are made from claims presented _without evidence_. This will invalidate fantastical accelerationist woo until such evidence is presented, and -in the current state of the art research within relevant fields- it's not even certain we can know what would constitute such evidence _in principle_.
- mindcrime 5y ago
  I assume you're referring to this part: If the expectation of a superhuman AI takeover is built on five key assumptions that have no basis in evidence, then this idea is more akin to a religious belief — a myth My problem is, even this is wrong. I do not find the "five key assumptions" he cites to be the foundation of beliefs that a dangerous "Superhuman AI" could emerge. So to me, this article simply fails right from the jump. The thing he is arguing for and/or against just don't, collectively, constitute a well-developed argument for the overall thesis. Not in my opinion anyway. To go through them one by one: Artificial intelligence is already getting smarter than us, at an exponential rate. I don't know anybody making this claim, nor is it apparent that belief in this is necessary to think that dangerous "Superhuman AI" could be a problem in the future. I guess it comes down to what you think Kelly meant by "soon" in the preceding sentence. We’ll make AIs into a general purpose intelligence, like our own. It's the "like our own" part of this that I have a problem with. Yes, I think many people believe that we will create some manner of "general intelligence", but I also believe that a lot of researchers acknowledge that it may not be 100% "human like". But this may be my own bias showing, as I personally harp a lot on the importance of the distinction between "human level" intelligence and "human like" intelligence. We can make human intelligence in silicon. This is probably the most sound one of all, but here's the thing... saying "there is no evidence for this" is simply wrong. To hold to this is to completely ignore all progress made building AI in silicon to date. And while we don't yet have "general intelligence", I don't see how one can just ignore incremental progress completely and say "there's nothing there". I would argue that the null hypothesis should probably be that general intelligence on silicon is possible that that the burden of proof is on the people arguing the opposite. Also, not everybody thinking about / working on AI is focused on the "in silicon" approach. The majority? Yes. But I'm sure there are a handful of people working on ideas related to DNA computing, chemical computing, growing synthetic brain matter, etc. Intelligence can be expanded without limit. To me, this may or not not be true, but it's completely irrelevant to the thesis at hand. The question isn't "can machine intelligence be unbounded?", but rather "can machine intelligence exceed human intelligence?" Once we have exploding superintelligence it can solve most of our problems. Again, this strikes me as mostly irrelevant to the question at hand. So yes, this article raises some interesting points, but I don't see it accomplishing what it seems to purport to do.
  - roywiggins 5y ago
    Yeah. All you need is an intelligence that's human level but faster, and you have something that is, to some extent, superhuman. If you can get to human intelligence levels you can just crank up the clock speed a bit and give it a better working memory, and now you have something like a super-Einstein in a box. It could be that there's no way to do that in silica without outstripping plausible bounds on hardware size or power use, I don't know...
    - goatlover 5y ago
      But Einstein was no ordinary human, he was a genius. So you would need the AGI to be more than just your average human. Would the average Joe sped up be some form of superintelligence? They would be able to perform tasks faster, but that doesn't make them smarter.
  - enord 5y ago
    "Som fanden leser bibelen" - Norwegian proverb, lit.: "Like the devil reads the bible" I sense that your arguments are in concordance with TFA, broadly speaking. Also, I suspect you read it rather uncharitably for reasons having to do with tone or stated thesis. Either that, or my reading comprehension is failing me. Maybe I want it to be more convincing that it really is, as i already laid bare my sentiments towards accelerationist woo.
    - mindcrime 5y ago
      Norwegian proverb, lit.: "Like the devil reads the bible" I'll take that as a compliment! All joking aside, I'm not saying every word of the article is wrong or anything. Maybe I'm just being too literal or pedantic or something. It just seems to me that the article sets out to make one very specific claim, and I just don't think it succeeds in making a strong case for that specific claim. Is there some interesting discussion there? Yes. Are there things that Kelly says that I agree with? Yes, absolutely. So maybe we're not too far apart on this after all.
idatum 5y ago
Maybe the article was AI generated? /s
eutectic 5y ago
Humanity is an existence proof for human level AI, and it seems arrogant to assume that our brains are at the limit of what is possible. Reproductive fitness requires many attributes which obviously compete with intelligence, and evolution is a blind hill-climbing search which cannot easily escape deep local minima. To me it's obvious that the development of superhuman AI is inevitable assuming we don't wipe ourselves out before we have time to develop it.
- dragontamer 5y ago
  But Google can read, injest, and search for information far faster than I can. Stockfish players chess far better than I can. AlphaGo plays Go far better than I can. Random Sudoku bot built on top of minizinc / z3 / whatever SAT solver can solve NP-complete problems better than any human. -------- Here's the thing: what is "strong AI" ?? As far as I can tell, we're already at superhuman AI in most intelligence tasks (collecting data, collating it together, analyzing it, etc. etc.). Its called a computer. What if "superhuman AI" is just the stuff we have today at our fingertips? It wasnt very long ago when reading documents and organizing them was considered AI. But today, we call that Google.
  - tshaddox 5y ago
    Stockfish and AlphaGo are also available for humans to use. A human using Stockfish will be just as good at chess as a strong AI using Stockfish, and likewise for any future improvements to chess engines that either humans or strong AIs develop.
  - Isinlor 5y ago
    There are no superhuman craftsman AI. Something that could in one package: - hold a conversation about what it is that needs to be done - clean a room - fix a sink - make a chair or table out of wood - dig a hole We don't know how to make AI that will learn to play even simple game like montezuma's revenge in some reasonable play time like 30 min. We are currently closer to months or years of gameplay required.
    - 5y ago
upearly4 5y ago
>This is a pretty bad article. Agreed. Dunno why he thinks this: "At the core of the notion of a superhuman intelligence — particularly the view that this intelligence will keep improving itself — is the essential belief that intelligence has an infinite scale." The notion of self-improvement is both powerful and not assumed to be 'infinite'. I figure it goes like this: Q. Is AI potentially dangerous? A. Yes. Q. Will it be human-like? A. Probably not. People tend to couch these discussions as a series of anthropomorphisms.
- goatlover 5y ago
  The article is a response to claims about the singularity and subsequent worries about concerning super AI.

enord 5y ago

You're making an argument that is already in TFA. It's in addition -and only partially related to- the main point which is: The arguments that are commonly used to demonstrate the possibility or threat of "ASI" are made from claims presented _without evidence_. This will invalidate fantastical accelerationist woo until such evidence is presented, and -in the current state of the art research within relevant fields- it's not even certain we can know what would constitute such evidence _in principle_.

mindcrime 5y ago
I assume you're referring to this part: If the expectation of a superhuman AI takeover is built on five key assumptions that have no basis in evidence, then this idea is more akin to a religious belief — a myth My problem is, even this is wrong. I do not find the "five key assumptions" he cites to be the foundation of beliefs that a dangerous "Superhuman AI" could emerge. So to me, this article simply fails right from the jump. The thing he is arguing for and/or against just don't, collectively, constitute a well-developed argument for the overall thesis. Not in my opinion anyway. To go through them one by one: Artificial intelligence is already getting smarter than us, at an exponential rate. I don't know anybody making this claim, nor is it apparent that belief in this is necessary to think that dangerous "Superhuman AI" could be a problem in the future. I guess it comes down to what you think Kelly meant by "soon" in the preceding sentence. We’ll make AIs into a general purpose intelligence, like our own. It's the "like our own" part of this that I have a problem with. Yes, I think many people believe that we will create some manner of "general intelligence", but I also believe that a lot of researchers acknowledge that it may not be 100% "human like". But this may be my own bias showing, as I personally harp a lot on the importance of the distinction between "human level" intelligence and "human like" intelligence. We can make human intelligence in silicon. This is probably the most sound one of all, but here's the thing... saying "there is no evidence for this" is simply wrong. To hold to this is to completely ignore all progress made building AI in silicon to date. And while we don't yet have "general intelligence", I don't see how one can just ignore incremental progress completely and say "there's nothing there". I would argue that the null hypothesis should probably be that general intelligence on silicon is possible that that the burden of proof is on the people arguing the opposite. Also, not everybody thinking about / working on AI is focused on the "in silicon" approach. The majority? Yes. But I'm sure there are a handful of people working on ideas related to DNA computing, chemical computing, growing synthetic brain matter, etc. Intelligence can be expanded without limit. To me, this may or not not be true, but it's completely irrelevant to the thesis at hand. The question isn't "can machine intelligence be unbounded?", but rather "can machine intelligence exceed human intelligence?" Once we have exploding superintelligence it can solve most of our problems. Again, this strikes me as mostly irrelevant to the question at hand. So yes, this article raises some interesting points, but I don't see it accomplishing what it seems to purport to do.
- roywiggins 5y ago
  Yeah. All you need is an intelligence that's human level but faster, and you have something that is, to some extent, superhuman. If you can get to human intelligence levels you can just crank up the clock speed a bit and give it a better working memory, and now you have something like a super-Einstein in a box. It could be that there's no way to do that in silica without outstripping plausible bounds on hardware size or power use, I don't know...
  - goatlover 5y ago
    But Einstein was no ordinary human, he was a genius. So you would need the AGI to be more than just your average human. Would the average Joe sped up be some form of superintelligence? They would be able to perform tasks faster, but that doesn't make them smarter.
- enord 5y ago
  "Som fanden leser bibelen" - Norwegian proverb, lit.: "Like the devil reads the bible" I sense that your arguments are in concordance with TFA, broadly speaking. Also, I suspect you read it rather uncharitably for reasons having to do with tone or stated thesis. Either that, or my reading comprehension is failing me. Maybe I want it to be more convincing that it really is, as i already laid bare my sentiments towards accelerationist woo.
  - mindcrime 5y ago
    Norwegian proverb, lit.: "Like the devil reads the bible" I'll take that as a compliment! All joking aside, I'm not saying every word of the article is wrong or anything. Maybe I'm just being too literal or pedantic or something. It just seems to me that the article sets out to make one very specific claim, and I just don't think it succeeds in making a strong case for that specific claim. Is there some interesting discussion there? Yes. Are there things that Kelly says that I agree with? Yes, absolutely. So maybe we're not too far apart on this after all.

idatum 5y ago

Maybe the article was AI generated? /s

eutectic 5y ago

Humanity is an existence proof for human level AI, and it seems arrogant to assume that our brains are at the limit of what is possible. Reproductive fitness requires many attributes which obviously compete with intelligence, and evolution is a blind hill-climbing search which cannot easily escape deep local minima. To me it's obvious that the development of superhuman AI is inevitable assuming we don't wipe ourselves out before we have time to develop it.

dragontamer 5y ago
But Google can read, injest, and search for information far faster than I can. Stockfish players chess far better than I can. AlphaGo plays Go far better than I can. Random Sudoku bot built on top of minizinc / z3 / whatever SAT solver can solve NP-complete problems better than any human. -------- Here's the thing: what is "strong AI" ?? As far as I can tell, we're already at superhuman AI in most intelligence tasks (collecting data, collating it together, analyzing it, etc. etc.). Its called a computer. What if "superhuman AI" is just the stuff we have today at our fingertips? It wasnt very long ago when reading documents and organizing them was considered AI. But today, we call that Google.
- tshaddox 5y ago
  Stockfish and AlphaGo are also available for humans to use. A human using Stockfish will be just as good at chess as a strong AI using Stockfish, and likewise for any future improvements to chess engines that either humans or strong AIs develop.
- Isinlor 5y ago
  There are no superhuman craftsman AI. Something that could in one package: - hold a conversation about what it is that needs to be done - clean a room - fix a sink - make a chair or table out of wood - dig a hole We don't know how to make AI that will learn to play even simple game like montezuma's revenge in some reasonable play time like 30 min. We are currently closer to months or years of gameplay required.
  - 5y ago

upearly4 5y ago

>This is a pretty bad article. Agreed. Dunno why he thinks this: "At the core of the notion of a superhuman intelligence — particularly the view that this intelligence will keep improving itself — is the essential belief that intelligence has an infinite scale." The notion of self-improvement is both powerful and not assumed to be 'infinite'. I figure it goes like this: Q. Is AI potentially dangerous? A. Yes. Q. Will it be human-like? A. Probably not. People tend to couch these discussions as a series of anthropomorphisms.

goatlover 5y ago
The article is a response to claims about the singularity and subsequent worries about concerning super AI.