4 ms·

Eliezer Yudkowsky: That Alien Message

91 points by rms 17y ago

rfreytag 17y ago
The AI-Boxing Experiment remains one of the most interesting things I have ever read: http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox In the "Alien Message" Eliezer Yudkowsky has flipped around the AI-boxing question so that now we're the AI in the box. Perhaps the results are more palatable that way. If you really want to stretch your mind dig around the archives of: http://www.sl4.org/ http://www.sl4.org/
- kwamenum86 17y ago
  re: the AI-Boxing experiment, Arguably a human can take over a human mind: http://en.wikipedia.org/wiki/Neuro-linguistic_programming http://en.wikipedia.org/wiki/Neuro-linguistic_programming
  - Luc 17y ago
    Arguably homeopathy can cure the common cold. The balance of evidence seems to be on the side of the skeptics...
- gort 17y ago
  Just because it's interesting, Eliezer mentioned later that he'd done the AI-box a couple more times and failed; I think his final record was 3 successes in 5 attempts. http://lesswrong.com/lw/up/shut_up_and_do_the_impossible/ http://lesswrong.com/lw/up/shut_up_and_do_the_impossible/
- lisper 17y ago
  http://rondam.blogspot.com/2008/10/reflections-on-being-ai-in-box.html http://rondam.blogspot.com/2008/10/reflections-on-being-ai-i...
- giardini 17y ago
  "Alien Message" reminds me of Robert E. Vardeman's "Weapons of Chaos" trilogy: http://www.fantasticfiction.co.uk/v/robert-e-vardeman/ http://www.fantasticfiction.co.uk/v/robert-e-vardeman/ Just as Yudkowsky's short story, those Vardeman novels include advanced aliens living on a much slower time-scale (and much larger space-scale) than we. Man's accidental incursion into their domain triggers a slow-scale attack response that Mankind doesn't at first recognize as intelligent.
danbmil99 17y ago
Jeez, this guy is paranoid.
- RevRal 17y ago
  What is it that you think he's paranoid about? Have you read anything else he's written? If not, you should. It's good stuff.
  - rbanffy 17y ago
    I second RevRal. His articles are some of the most interesting ones I have read in a very long time.
DanielBMarkham 17y ago
This is good, but I'd like to point out that it's at least the third time this same article has appeared on HN. I wasn't sure if I should comment about that or not. I know lots of folks haven't seen it, so thanks for posting, but for those of us who have already clicked twice, it wasn't as enjoyable the third time around as it was the first.
- megaduck 17y ago
  Normally I'm opposed to dupes, but every once in a while there's something that everybody should see. This probably falls into that category, if you couple it with the AI-Boxing experiment (previously mentioned in this thread). The results of that experiment are so profound and disturbing that everybody should have some exposure.
  - DanielBMarkham 17y ago
    I actually don't have a problem with dupes. I just wish there was some way for me to know that I've already read it. Most times I don't remember from the title alone.
    - megaduck 17y ago
      Ah, understood. Sorry if I misread you. I totally agree it would be nice to be able to flag previously-read articles, or manage dupes in some other elegant way. It seems like a non-trivial problem, though.
    - Quarrelsome 17y ago
      I agree, definately in regards to tech. Would it hurt to re-run a bit of Hibernate, Swing or even something as basic as Ant?
- swombat 17y ago
  Third time here as well, and yet I must report that I enjoyed it just as much the third time as the first two! (that is, a lot) (Just to offer a piece of counter-evidence)
deleted 17y ago
[deleted]
Eliezer 17y ago
I'm flattered by the attention this essay still gets! I just wish I understood better why some of my writing is so much more appealing than other writing. Things I designed to be viral, like http://yudkowsky.net/rational/virtues http://yudkowsky.net/rational/virtues, don't keep getting HN'ed and reddited like this does. Not knowing what you did right is almost as frustrating as not knowing what you did wrong. (Okay, so I'm complaining to the choir here.)
- adrih 17y ago
  "That Alien Message" read like a science fiction short story at first, and then delivered a number of abnormally deep insights. Ideas so new (or ways of seeing so new) get to my brain 5-6 times a year top. "Twelve virtues" however... well, I started skimming at the fourth virtue. Most of the content was a restatement of things I already agree with. Maybe your audience (HN, Reddit) is enough into rationality to care more about very rational discussions (like the AI-Box problem) than about discussions of rationality itself (like "Twelve Virtues").
  - rfreytag 17y ago
    You may enjoy readinghttp://en.wikipedia.org/wiki/Dragons_Egg http://en.wikipedia.org/wiki/Dragons_Egg
- rfreytag 17y ago
  What interested me about this essay: 1. I had not seriously considered the question of what happens when a "transhuman" AI appears (yes casually, not not in depth). 2. The ramifications were likely within my lifetime (I think Google has developing an AI as its explicit aim http://tinyurl.com/yald3f3 http://tinyurl.com/yald3f3), personally relevant, and non-trivial. 3. The answer is not immediately clear in a world in which technology makes it increasingly likely someone will break a moratorium and build such an AI (unless we are to accept our fate and either ride the singularity somehow or be marginalized). You have something something people care a lot about. Reminds me of a "minimum viable product." If you haven't already - do a popular book, a tour, rinse and repeat. Let Rudy Rucker be your guide.
- 17y ago
lispmeister 17y ago
Way back in 2000 at the Foresight Associates conference, Eliezer stood his ground against Marvin Minsky calling his ideas "rubbish". I was impressed by how Eliezer answered each objection with deep insights into the nature of AI.
- zephjc 17y ago
  Minksy is no dummy, but his ideas about AI and cognition are dated (though I admit it's been a while since I've read anything he's had to say on the subject).

91 points by rms 17y ago

The AI-Boxing Experiment remains one of the most interesting things I have ever read: http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox In the "Alien Message" Eliezer Yudkowsky has flipped around the AI-boxing question so that now we're the AI in the box. Perhaps the results are more palatable that way. If you really want to stretch your mind dig around the archives of: http://www.sl4.org/ http://www.sl4.org/

kwamenum86 17y ago
re: the AI-Boxing experiment, Arguably a human can take over a human mind: http://en.wikipedia.org/wiki/Neuro-linguistic_programming http://en.wikipedia.org/wiki/Neuro-linguistic_programming
- Luc 17y ago
  Arguably homeopathy can cure the common cold. The balance of evidence seems to be on the side of the skeptics...
gort 17y ago
Just because it's interesting, Eliezer mentioned later that he'd done the AI-box a couple more times and failed; I think his final record was 3 successes in 5 attempts. http://lesswrong.com/lw/up/shut_up_and_do_the_impossible/ http://lesswrong.com/lw/up/shut_up_and_do_the_impossible/
lisper 17y ago
http://rondam.blogspot.com/2008/10/reflections-on-being-ai-in-box.html http://rondam.blogspot.com/2008/10/reflections-on-being-ai-i...
giardini 17y ago
"Alien Message" reminds me of Robert E. Vardeman's "Weapons of Chaos" trilogy: http://www.fantasticfiction.co.uk/v/robert-e-vardeman/ http://www.fantasticfiction.co.uk/v/robert-e-vardeman/ Just as Yudkowsky's short story, those Vardeman novels include advanced aliens living on a much slower time-scale (and much larger space-scale) than we. Man's accidental incursion into their domain triggers a slow-scale attack response that Mankind doesn't at first recognize as intelligent.

danbmil99 17y ago

Jeez, this guy is paranoid.

RevRal 17y ago
What is it that you think he's paranoid about? Have you read anything else he's written? If not, you should. It's good stuff.
- rbanffy 17y ago
  I second RevRal. His articles are some of the most interesting ones I have read in a very long time.

DanielBMarkham 17y ago

This is good, but I'd like to point out that it's at least the third time this same article has appeared on HN. I wasn't sure if I should comment about that or not. I know lots of folks haven't seen it, so thanks for posting, but for those of us who have already clicked twice, it wasn't as enjoyable the third time around as it was the first.

megaduck 17y ago
Normally I'm opposed to dupes, but every once in a while there's something that everybody should see. This probably falls into that category, if you couple it with the AI-Boxing experiment (previously mentioned in this thread). The results of that experiment are so profound and disturbing that everybody should have some exposure.
- DanielBMarkham 17y ago
  I actually don't have a problem with dupes. I just wish there was some way for me to know that I've already read it. Most times I don't remember from the title alone.
  - megaduck 17y ago
    Ah, understood. Sorry if I misread you. I totally agree it would be nice to be able to flag previously-read articles, or manage dupes in some other elegant way. It seems like a non-trivial problem, though.
  - Quarrelsome 17y ago
    I agree, definately in regards to tech. Would it hurt to re-run a bit of Hibernate, Swing or even something as basic as Ant?
swombat 17y ago
Third time here as well, and yet I must report that I enjoyed it just as much the third time as the first two! (that is, a lot) (Just to offer a piece of counter-evidence)

deleted 17y ago

[deleted]

Eliezer 17y ago

I'm flattered by the attention this essay still gets! I just wish I understood better why some of my writing is so much more appealing than other writing. Things I designed to be viral, like http://yudkowsky.net/rational/virtues http://yudkowsky.net/rational/virtues, don't keep getting HN'ed and reddited like this does. Not knowing what you did right is almost as frustrating as not knowing what you did wrong. (Okay, so I'm complaining to the choir here.)

adrih 17y ago
"That Alien Message" read like a science fiction short story at first, and then delivered a number of abnormally deep insights. Ideas so new (or ways of seeing so new) get to my brain 5-6 times a year top. "Twelve virtues" however... well, I started skimming at the fourth virtue. Most of the content was a restatement of things I already agree with. Maybe your audience (HN, Reddit) is enough into rationality to care more about very rational discussions (like the AI-Box problem) than about discussions of rationality itself (like "Twelve Virtues").
- rfreytag 17y ago
  You may enjoy readinghttp://en.wikipedia.org/wiki/Dragons_Egg http://en.wikipedia.org/wiki/Dragons_Egg
rfreytag 17y ago
What interested me about this essay: 1. I had not seriously considered the question of what happens when a "transhuman" AI appears (yes casually, not not in depth). 2. The ramifications were likely within my lifetime (I think Google has developing an AI as its explicit aim http://tinyurl.com/yald3f3 http://tinyurl.com/yald3f3), personally relevant, and non-trivial. 3. The answer is not immediately clear in a world in which technology makes it increasingly likely someone will break a moratorium and build such an AI (unless we are to accept our fate and either ride the singularity somehow or be marginalized). You have something something people care a lot about. Reminds me of a "minimum viable product." If you haven't already - do a popular book, a tour, rinse and repeat. Let Rudy Rucker be your guide.
17y ago

lispmeister 17y ago

Way back in 2000 at the Foresight Associates conference, Eliezer stood his ground against Marvin Minsky calling his ideas "rubbish". I was impressed by how Eliezer answered each objection with deep insights into the nature of AI.

zephjc 17y ago
Minksy is no dummy, but his ideas about AI and cognition are dated (though I admit it's been a while since I've read anything he's had to say on the subject).