6 ms·

I'm flattered by the attention this essay still gets! I just wish I understood better why some of my writing is so much more appealing than other writing. Thi

by Eliezer 17y ago

I'm flattered by the attention this essay still gets! I just wish I understood better why some of my writing is so much more appealing than other writing. Things I designed to be viral, like http://yudkowsky.net/rational/virtues http://yudkowsky.net/rational/virtues, don't keep getting HN'ed and reddited like this does. Not knowing what you did right is almost as frustrating as not knowing what you did wrong. (Okay, so I'm complaining to the choir here.)

adrih 17y ago
"That Alien Message" read like a science fiction short story at first, and then delivered a number of abnormally deep insights. Ideas so new (or ways of seeing so new) get to my brain 5-6 times a year top. "Twelve virtues" however... well, I started skimming at the fourth virtue. Most of the content was a restatement of things I already agree with. Maybe your audience (HN, Reddit) is enough into rationality to care more about very rational discussions (like the AI-Box problem) than about discussions of rationality itself (like "Twelve Virtues").
- rfreytag 17y ago
  You may enjoy readinghttp://en.wikipedia.org/wiki/Dragons_Egg http://en.wikipedia.org/wiki/Dragons_Egg
rfreytag 17y ago
What interested me about this essay: 1. I had not seriously considered the question of what happens when a "transhuman" AI appears (yes casually, not not in depth). 2. The ramifications were likely within my lifetime (I think Google has developing an AI as its explicit aim http://tinyurl.com/yald3f3 http://tinyurl.com/yald3f3), personally relevant, and non-trivial. 3. The answer is not immediately clear in a world in which technology makes it increasingly likely someone will break a moratorium and build such an AI (unless we are to accept our fate and either ride the singularity somehow or be marginalized). You have something something people care a lot about. Reminds me of a "minimum viable product." If you haven't already - do a popular book, a tour, rinse and repeat. Let Rudy Rucker be your guide.
lisper 17y ago
I can tell you exactly why "That Alien Message" gets more attention: because it's a story, and a cracklin' good one. Human brains are hard-wired to be attracted to good stories. This is why mythology is so powerful. BTW, there is a pretty well worked out theory of what makes a good story. Geeks tend to pooh-pooh this theory because it's associated with the "squishy" field of stage and screen drama. IMHO this is a serious mistake. The world today really needs a better mythology. Those in its current inventory are getting obsolete. Some of them are becoming actively harmful. Coming up with a good mythology is not so easy. The last person I know of who did a notable job was L. Ron Hubbard (at least, he's the last one who did it intentionally). Imagine how much better off the world would be if Hubbard hadn't been a psychopath. I hope some people reading this might start devoting a little time to thinking about this problem.
- davi 17y ago
  there is a pretty well worked out theory of what makes a good story links please
  - pmichaud 17y ago
    http://kotiro.petermichaud.com/writing/on-drama/ http://kotiro.petermichaud.com/writing/on-drama/ http://kotiro.petermichaud.com/writing/the-story-conversation/ http://kotiro.petermichaud.com/writing/the-story-conversatio...
  - lisper 17y ago
    http://en.wikipedia.org/wiki/Dramatic_theory http://en.wikipedia.org/wiki/Dramatic_theory http://en.wikipedia.org/wiki/Poetics_(Aristotle) http://en.wikipedia.org/wiki/Poetics_(Aristotle) http://www.amazon.com/Story-Substance-Structure-Principles-Screenwriting/dp/0060391685 http://www.amazon.com/Story-Substance-Structure-Principles-S...
    - lisper 17y ago
      BTW, the theory is pretty easy to summarize. At the risk of oversimplifying, the essential elements of a good story are: 1. A protagonist, who is a sympathetic character with a goal. In Eliezer's story the protagonist is not a single character, but all of humanity. Their goal is to understand the universe. 2. An antagonist, who places obstacles in the path of the protagonist to prevent him (or her or them) from achieving the goal. In Eliezer's story, the antagonist is whoever is causing the stars to blink. 3. A structure that includes (more or less) the following elements in order: a) An introduction that establishes the characters and the basic ground rules of the world they live in b) An inciting incident that launches the protagonist on a journey. In Eliezer's story this is the stars starting to blink. c) The journey itself, which ultimately leads to some kind of catastrophe from which no escape seems possible. d) A resolution, which results in the protagonist being forever changed in some way. It may or may not include the actual achievement of the protagonist's goal. It may, for example, be the protagonist realizing that they chose the wrong goal. Think "Wizard of Oz". Of course, there's a lot more to say than just that, but those are the fundamentals.
rms 17y ago
It's more entertaining
idlewords 17y ago
Does it worry you that you are working on a challenging 'prediction' problem (anticipating and constraining the behavior of an AI) while unable to even make accurate guesses about human behavior?
- Eliezer 17y ago
  I don't intend to design it as poorly as the human brain.
lsd5you 17y ago
Normative writing as wide ranging as 'The Virtues' is quasi-religous and an all or nothing proposition. People are not receptive to having their values changed in such a direct manner.
zephjc 17y ago
Clearly you need to present more of your work as scifi :) As scifi incorporates more and more transhuman elements, these concepts become less radical. I recently read the decently-written novel 'Darwinia' which spoiler takes a hard 90-degree turn partway into the story revealing the protagonist's world is really a simulation/recording, running around the time of universe's heat death, using "distortion lines in the Higgs field" as the substrate on which the simulation runs. end spoiler I think an important lesson is that you can't always design something to be viral (look at how fickle YouTube users are)

by Eliezer 17y ago

adrih 17y ago

"That Alien Message" read like a science fiction short story at first, and then delivered a number of abnormally deep insights. Ideas so new (or ways of seeing so new) get to my brain 5-6 times a year top. "Twelve virtues" however... well, I started skimming at the fourth virtue. Most of the content was a restatement of things I already agree with. Maybe your audience (HN, Reddit) is enough into rationality to care more about very rational discussions (like the AI-Box problem) than about discussions of rationality itself (like "Twelve Virtues").

rfreytag 17y ago
You may enjoy readinghttp://en.wikipedia.org/wiki/Dragons_Egg http://en.wikipedia.org/wiki/Dragons_Egg

rfreytag 17y ago

What interested me about this essay: 1. I had not seriously considered the question of what happens when a "transhuman" AI appears (yes casually, not not in depth). 2. The ramifications were likely within my lifetime (I think Google has developing an AI as its explicit aim http://tinyurl.com/yald3f3 http://tinyurl.com/yald3f3), personally relevant, and non-trivial. 3. The answer is not immediately clear in a world in which technology makes it increasingly likely someone will break a moratorium and build such an AI (unless we are to accept our fate and either ride the singularity somehow or be marginalized). You have something something people care a lot about. Reminds me of a "minimum viable product." If you haven't already - do a popular book, a tour, rinse and repeat. Let Rudy Rucker be your guide.

lisper 17y ago

I can tell you exactly why "That Alien Message" gets more attention: because it's a story, and a cracklin' good one. Human brains are hard-wired to be attracted to good stories. This is why mythology is so powerful. BTW, there is a pretty well worked out theory of what makes a good story. Geeks tend to pooh-pooh this theory because it's associated with the "squishy" field of stage and screen drama. IMHO this is a serious mistake. The world today really needs a better mythology. Those in its current inventory are getting obsolete. Some of them are becoming actively harmful. Coming up with a good mythology is not so easy. The last person I know of who did a notable job was L. Ron Hubbard (at least, he's the last one who did it intentionally). Imagine how much better off the world would be if Hubbard hadn't been a psychopath. I hope some people reading this might start devoting a little time to thinking about this problem.

davi 17y ago
there is a pretty well worked out theory of what makes a good story links please
- pmichaud 17y ago
  http://kotiro.petermichaud.com/writing/on-drama/ http://kotiro.petermichaud.com/writing/on-drama/ http://kotiro.petermichaud.com/writing/the-story-conversation/ http://kotiro.petermichaud.com/writing/the-story-conversatio...
- lisper 17y ago
  http://en.wikipedia.org/wiki/Dramatic_theory http://en.wikipedia.org/wiki/Dramatic_theory http://en.wikipedia.org/wiki/Poetics_(Aristotle) http://en.wikipedia.org/wiki/Poetics_(Aristotle) http://www.amazon.com/Story-Substance-Structure-Principles-Screenwriting/dp/0060391685 http://www.amazon.com/Story-Substance-Structure-Principles-S...
  - lisper 17y ago
    BTW, the theory is pretty easy to summarize. At the risk of oversimplifying, the essential elements of a good story are: 1. A protagonist, who is a sympathetic character with a goal. In Eliezer's story the protagonist is not a single character, but all of humanity. Their goal is to understand the universe. 2. An antagonist, who places obstacles in the path of the protagonist to prevent him (or her or them) from achieving the goal. In Eliezer's story, the antagonist is whoever is causing the stars to blink. 3. A structure that includes (more or less) the following elements in order: a) An introduction that establishes the characters and the basic ground rules of the world they live in b) An inciting incident that launches the protagonist on a journey. In Eliezer's story this is the stars starting to blink. c) The journey itself, which ultimately leads to some kind of catastrophe from which no escape seems possible. d) A resolution, which results in the protagonist being forever changed in some way. It may or may not include the actual achievement of the protagonist's goal. It may, for example, be the protagonist realizing that they chose the wrong goal. Think "Wizard of Oz". Of course, there's a lot more to say than just that, but those are the fundamentals.

rms 17y ago

It's more entertaining

idlewords 17y ago

Does it worry you that you are working on a challenging 'prediction' problem (anticipating and constraining the behavior of an AI) while unable to even make accurate guesses about human behavior?

Eliezer 17y ago
I don't intend to design it as poorly as the human brain.

lsd5you 17y ago

Normative writing as wide ranging as 'The Virtues' is quasi-religous and an all or nothing proposition. People are not receptive to having their values changed in such a direct manner.

zephjc 17y ago

Clearly you need to present more of your work as scifi :) As scifi incorporates more and more transhuman elements, these concepts become less radical. I recently read the decently-written novel 'Darwinia' which spoiler takes a hard 90-degree turn partway into the story revealing the protagonist's world is really a simulation/recording, running around the time of universe's heat death, using "distortion lines in the Higgs field" as the substrate on which the simulation runs. end spoiler I think an important lesson is that you can't always design something to be viral (look at how fickle YouTube users are)