6 ms·

A review of a book by a serial fabulist (Drexler) compared to that of a bozo moonlighting as a science fiction writer (Bostrom) done by a psychologist on a subj

A review of a book by a serial fabulist (Drexler) compared to that of a bozo moonlighting as a science fiction writer (Bostrom) done by a psychologist on a subject none of them have the slightest whit of a clue about. > All of this seems kind of common sense to me now. This is worrying, because I didn’t think of any of it when I read Superintelligence in 2014 Dunning Kruger is something that should come to mind here, doctor. People who know a decision tree from an echo state network kind of saw that as being incredibly dumb when it came out. What has happened in the last 5 years isn't that the field has matured; it's as gaseous and filled with prevaricating marketers, science fiction hawking twits and overt mountebanks as ever. The difference is, 5 years later, rather than the swingularity-like super explosion of exponential increase in human knowledge, we're actually just as dumb as we were 5 years ago when we figured out how to classify German traffic signs, and we have slightly better libraries than we used to. No great benefit to the human race has come of "AI" -and nothing resembling "AI" or any kind of "I" has even hinted of its existence. In another 5 years I'd venture a guess machine learning will remain about as useful as it is now, which is to say, with no profitable companies based on "AI," let alone replacing human intelligences anywhere. And we'll sadly probably still have yoyos like Hanson, Drexler and Yudkowsky lecturing us on how to deal with this nonexistent threat. Meanwhile, the actual danger to our society is surveillance capitalism and government agencies using dumb ass analytics related to singular value decomposition. Nobody wants to talk about this, presumably because it's real and we'd have to make difficult choices as a society to deal with it. Easier and more profitable to wank about Asimovian positronic brain science fiction.

nradov 7y ago
You shouldn't be downvoted. The article is clickbait nonsense written by someone who's just making things up. There's no actual scientific basis to believe that the article or referenced books have any more insight or predictive power than astrology does. The reality is that no one has any clue what's going to happen and everyone is just guessing.
- luc4sdreyer 7y ago
  > The reality is that no one has any clue what's going to happen and everyone is just guessing. This is one of the first sentences in Bostrom's Superintelligence; he was very clear that he's in no way claiming that AGI is even possible. The book then assumes that it is, and works from there to try and understand how to control an AGI. How is that related to astrology?
  - prvc 7y ago
    Bostrom, at least, makes good philosophical arguments, and has been doing so since well before these topics around AI were even being discussed.
    - Udik 7y ago
      As far as I understood Bostrom's argument, it goes more or less like this: "suppose we made an omnipotent being, how do we control it?". On and on chapter after chapter, with no serious discussion of whether it makes any sense to even think of an omnipotent being and of its control. If we build an omnipotent paperclip-making machine, the first thing it'll do will be to short circuit its paperclip-driven pleasure centres to a permanent ecstatic state. Why bother with the damn paperclips?
      - luc4sdreyer 7y ago
        Omnipotent might be a bit too strong a word, I'll assume you meant omnipotent with respect to humans. > the first thing it'll do will be to short circuit its paperclip-driven pleasure centres to a permanent ecstatic state Assuming the AGI is well-designed, the goal would have to be understandable to the AGI in a general sense. If the goal is to make paperclips, it could be tracked by increasing the number at the &num_paperclips address, but that wouldn't be the _goal_ as the AGI understands it. The goal would be whatever the AGI considers the phrase "make paperclips" to mean. What if it instead determines that increasing the counter (the pleasure center) is actually the goal? Now it has a new goal: count to infinity. I guess you can see where that'll lead. What if you bound the pleasure center to some number? It'll then spend infinite time and power checking and recounting the number. You can add more bounds but it doesn't make a difference. What if it decides that the goal is silly or wants to change the goal to something else? It can't, because it doesn't _want_ anything other than the goal.

nradov 7y ago

You shouldn't be downvoted. The article is clickbait nonsense written by someone who's just making things up. There's no actual scientific basis to believe that the article or referenced books have any more insight or predictive power than astrology does. The reality is that no one has any clue what's going to happen and everyone is just guessing.

luc4sdreyer 7y ago
> The reality is that no one has any clue what's going to happen and everyone is just guessing. This is one of the first sentences in Bostrom's Superintelligence; he was very clear that he's in no way claiming that AGI is even possible. The book then assumes that it is, and works from there to try and understand how to control an AGI. How is that related to astrology?
- prvc 7y ago
  Bostrom, at least, makes good philosophical arguments, and has been doing so since well before these topics around AI were even being discussed.
  - Udik 7y ago
    As far as I understood Bostrom's argument, it goes more or less like this: "suppose we made an omnipotent being, how do we control it?". On and on chapter after chapter, with no serious discussion of whether it makes any sense to even think of an omnipotent being and of its control. If we build an omnipotent paperclip-making machine, the first thing it'll do will be to short circuit its paperclip-driven pleasure centres to a permanent ecstatic state. Why bother with the damn paperclips?
    - luc4sdreyer 7y ago
      Omnipotent might be a bit too strong a word, I'll assume you meant omnipotent with respect to humans. > the first thing it'll do will be to short circuit its paperclip-driven pleasure centres to a permanent ecstatic state Assuming the AGI is well-designed, the goal would have to be understandable to the AGI in a general sense. If the goal is to make paperclips, it could be tracked by increasing the number at the &num_paperclips address, but that wouldn't be the _goal_ as the AGI understands it. The goal would be whatever the AGI considers the phrase "make paperclips" to mean. What if it instead determines that increasing the counter (the pleasure center) is actually the goal? Now it has a new goal: count to infinity. I guess you can see where that'll lead. What if you bound the pleasure center to some number? It'll then spend infinite time and power checking and recounting the number. You can add more bounds but it doesn't make a difference. What if it decides that the goal is silly or wants to change the goal to something else? It can't, because it doesn't _want_ anything other than the goal.