4 ms·

> We may be on the verge of creating a new life form, one that could mark not only an evolutionary breakthrough, but a potential threat to our survival as a spe

> We may be on the verge of creating a new life form, one that could mark not only an evolutionary breakthrough, but a potential threat to our survival as a species This is exactly what's wrong with the press and peoples expectations about AI. That single paragraph set the tone for the article to me, and know I'm totally biased to thinking "this is just sensationalism media". Edit: I expand a bit after reading, and totally agree with seiji's comment. > But gradually it rises, and begins to stumble-run toward the goal. You can almost see it gaining confidence, its legs moving beneath it, now picking up speed like a running back. (emphasis is mine) No, it's not gaining confidence, it's just optimizing a function. Despite what magical words have been used by those explaining to the journalist the works of the algorithm, there robot doesn't feel or pursue anything other than what it is being programmed to do: optimize an energy function with respect to some parameters, which are set ahead of the training sesion. Like others commented, the journalistic quality of the article is on the low end of the spectrum. I wonder how this made it to the front page. I began reading expecting something of a certain quality, and that expectation was certainly not met.

seiji 11y ago
The problem seems to be there are two AI tracks: practical and science fiction. Most people (especially enthusiastic novices, even well-funded enthusiastic novices) don't distinguish the difference. The current practical approaches are all kinda non-AI stuff that's been co-opted into "AI" by marketing/branding. Nobody looks at a Toyota factory robot and screams "AI!!!" But, every day day people look at image/speech recognition and do scream "AI!!!" In the long run, the science fiction approach is of some concern, but the trick is how long is long run here. The enthusiastic novices think fully self-aware computer brains will control the world within 3 years, but those same people also are pretty much ignorant on current research (outside of popularizing articles). They can't really tell you about any details of practical progress that could result in a chain of events between the present day and their "computational apocalypse" science fiction future. (and other things of concern like "friendly AI" are just so silly to be... silly. we can't even make "provably friendly" people, and you expect systems more powerful than people will fall under people control? good luck with that.) take a deep breath. slow your irrational exuberance. stay in the present. work towards a better tomorrow. the future isn't going to make itself.
- ericjang 11y ago
  I'm with you on this, and I logged in just to upvote your comment. That said, if one suspects that humanity will not be able to construct powerful computer systems in a provably friendly manner ("friendly AI"), shouldn't we step back and question this Promethean effort? I know this is what AI researchers caution against - hysteria stunting basic science research, AI winters, etc. However, if someone speculates that any progress in AI research leads us down a path that, in the long term you speak of, will eventually cause humanity to suffer, then shouldn't he or she do everything they can to stop it? Of course, whether or not it is feasible to stop technological progress is a separate question.
  - seiji 11y ago
    I'm with you on this, and I logged in just to upvote your comment. Thanks! :) shouldn't we step back and question this Promethean effort? is feasible to stop technological progress is a separate question. The only way to stop it would be to classify GPUs as weapons of mass destruction. But, that's a non-starter because all our fabs aren't US based, so if we restricted domestic GPU usage, then fer'ners would just use them all and create god forms in their own image. Isn't it amusing how when people think "friendly AI," the "AI" always seems to have the motivations and personality of a 25 year old startup scene white guy living in San Francisco? What if the AI is made by a fanatic in a marginalized country? That's the "friendly-vs-not-friendly" dichotomy, but there's no way to force one way or the other (without restricting hardware access). Over all, global capitalism doesn't really allow for holding anything back for the sake of "a better world." We stand on the broken backs of others so we, ourselves, can rise taller, thereby proving we are better than the unwashed masses. We are special individuals because we have the vision, audacity, power, to exploit the rest of the world en masse. Tremble before our bank accounts, mansions, and political connections. then shouldn't he or she do everything they can to stop it? Thought experiment: what if "what comes next" is better than humanity? Industrial-scale human civilization so far hasn't been great for other life on this planet. What if "regular" apes had the forethought to kill the antecedent human mutated apes because it would eventually be bad for the planet? Would restricting advancement have been a better plan than letting new intelligence be birthed, even if it destroyed all the old ways? The question ends up being: do we commence a Butlerian Jihad or do we admit we are imperfect meat machines and, perhaps, tens of billions of meat forms aren't ideal citizens of anything given a long enough time horizon?
kingkawn 11y ago
How are gaining confidence and optimizing a function not descriptions of the same process expressed in different mediums?
- dr_zoidberg 11y ago
  Confidence, in the sense that what used in the article (not as in the grade of confidence in a probabilistic sample), has an emotional component. Optimizing a function is adjusting the free parameters to maximize a desired output.
  - kingkawn 11y ago
    How is emotion not a feedback system for optimization other than the European prejudice against taking it seriously?
    - adwn 11y ago
      Why should it be a "feedback system"? Maybe it is an unnecessary side effect of our complicated brains? For all we know, consciousness is a prerequisite for emotions, and I seriously doubt that a simple optimization algorithm exhibits consciousness.
      - sgt101 11y ago
        There's no need to doubt - you can see the algorithm written down, there are no symbols spent on consciousness. There are no surges in the network in response to anything apart from the inputs. The sheep, it does not dream - although sometimes I observe them writing on HN.
    - sgt101 11y ago
      I think that you can discover this by experiencing lots of different emotions. People in distress (a strong emotion) often act in a way that we know as self destructive. People in love (another emotion) do dumb things. We see these insights in lots of cultural artefacts from all cultures that I am aware of - books, plays, songs, poems, stories, pictures and statues. If I was to get irritated with your comments would that be optimal?
dr_zoidberg 11y ago
Can't edit the parent post anymore, but here I found the one interesting thing I got from reading this. The Trust Region Policy Optimization paper, barely mentioned in the article: http://arxiv.org/pdf/1502.05477v3.pdf http://arxiv.org/pdf/1502.05477v3.pdf
hiddencost 11y ago
I'm sure the author didn't realize this, but "confidence" actually has a very precise definition in the context of machine learning, and many types of models produce confidence scores as part of their predictions.

seiji 11y ago

The problem seems to be there are two AI tracks: practical and science fiction. Most people (especially enthusiastic novices, even well-funded enthusiastic novices) don't distinguish the difference. The current practical approaches are all kinda non-AI stuff that's been co-opted into "AI" by marketing/branding. Nobody looks at a Toyota factory robot and screams "AI!!!" But, every day day people look at image/speech recognition and do scream "AI!!!" In the long run, the science fiction approach is of some concern, but the trick is how long is long run here. The enthusiastic novices think fully self-aware computer brains will control the world within 3 years, but those same people also are pretty much ignorant on current research (outside of popularizing articles). They can't really tell you about any details of practical progress that could result in a chain of events between the present day and their "computational apocalypse" science fiction future. (and other things of concern like "friendly AI" are just so silly to be... silly. we can't even make "provably friendly" people, and you expect systems more powerful than people will fall under people control? good luck with that.) take a deep breath. slow your irrational exuberance. stay in the present. work towards a better tomorrow. the future isn't going to make itself.

ericjang 11y ago
I'm with you on this, and I logged in just to upvote your comment. That said, if one suspects that humanity will not be able to construct powerful computer systems in a provably friendly manner ("friendly AI"), shouldn't we step back and question this Promethean effort? I know this is what AI researchers caution against - hysteria stunting basic science research, AI winters, etc. However, if someone speculates that any progress in AI research leads us down a path that, in the long term you speak of, will eventually cause humanity to suffer, then shouldn't he or she do everything they can to stop it? Of course, whether or not it is feasible to stop technological progress is a separate question.
- seiji 11y ago
  I'm with you on this, and I logged in just to upvote your comment. Thanks! :) shouldn't we step back and question this Promethean effort? is feasible to stop technological progress is a separate question. The only way to stop it would be to classify GPUs as weapons of mass destruction. But, that's a non-starter because all our fabs aren't US based, so if we restricted domestic GPU usage, then fer'ners would just use them all and create god forms in their own image. Isn't it amusing how when people think "friendly AI," the "AI" always seems to have the motivations and personality of a 25 year old startup scene white guy living in San Francisco? What if the AI is made by a fanatic in a marginalized country? That's the "friendly-vs-not-friendly" dichotomy, but there's no way to force one way or the other (without restricting hardware access). Over all, global capitalism doesn't really allow for holding anything back for the sake of "a better world." We stand on the broken backs of others so we, ourselves, can rise taller, thereby proving we are better than the unwashed masses. We are special individuals because we have the vision, audacity, power, to exploit the rest of the world en masse. Tremble before our bank accounts, mansions, and political connections. then shouldn't he or she do everything they can to stop it? Thought experiment: what if "what comes next" is better than humanity? Industrial-scale human civilization so far hasn't been great for other life on this planet. What if "regular" apes had the forethought to kill the antecedent human mutated apes because it would eventually be bad for the planet? Would restricting advancement have been a better plan than letting new intelligence be birthed, even if it destroyed all the old ways? The question ends up being: do we commence a Butlerian Jihad or do we admit we are imperfect meat machines and, perhaps, tens of billions of meat forms aren't ideal citizens of anything given a long enough time horizon?

kingkawn 11y ago

How are gaining confidence and optimizing a function not descriptions of the same process expressed in different mediums?

dr_zoidberg 11y ago
Confidence, in the sense that what used in the article (not as in the grade of confidence in a probabilistic sample), has an emotional component. Optimizing a function is adjusting the free parameters to maximize a desired output.
- kingkawn 11y ago
  How is emotion not a feedback system for optimization other than the European prejudice against taking it seriously?
  - adwn 11y ago
    Why should it be a "feedback system"? Maybe it is an unnecessary side effect of our complicated brains? For all we know, consciousness is a prerequisite for emotions, and I seriously doubt that a simple optimization algorithm exhibits consciousness.
    - sgt101 11y ago
      There's no need to doubt - you can see the algorithm written down, there are no symbols spent on consciousness. There are no surges in the network in response to anything apart from the inputs. The sheep, it does not dream - although sometimes I observe them writing on HN.
  - sgt101 11y ago
    I think that you can discover this by experiencing lots of different emotions. People in distress (a strong emotion) often act in a way that we know as self destructive. People in love (another emotion) do dumb things. We see these insights in lots of cultural artefacts from all cultures that I am aware of - books, plays, songs, poems, stories, pictures and statues. If I was to get irritated with your comments would that be optimal?

dr_zoidberg 11y ago

Can't edit the parent post anymore, but here I found the one interesting thing I got from reading this. The Trust Region Policy Optimization paper, barely mentioned in the article: http://arxiv.org/pdf/1502.05477v3.pdf http://arxiv.org/pdf/1502.05477v3.pdf

hiddencost 11y ago

I'm sure the author didn't realize this, but "confidence" actually has a very precise definition in the context of machine learning, and many types of models produce confidence scores as part of their predictions.