13 ms·

Rationality: From AI to Zombies

113 points by rayalez 12y ago

hunnypot 12y ago
I'll just say that "intelligence.org" is as heavy-handed as Google buying the .dev TLD.
- jtheory 12y ago
  I don't think I could bring myself to be a "Bright" for that reason (see the-brights.net)... Vague counterpoint -- this is an ebook collecting essays from lesswrong.com, which is a bit more subtle as names go.
  - adamzerner 12y ago
    > I don't think I could bring myself to be a "Bright" for that reason (see the-brights.net)... A name is enough of a reason on it's own?
    - jtheory 12y ago
      Sure. That is, my understanding of reality is the same, regardless. I think the goals of "The Brights" are laudable, generally. But I didn't register on their website, and wouldn't really want my photo and identify featured as a Bright, because the name just doesn't feel right to me. How you identify yourself makes a difference to how you're judged, and hence the influence you can bring to bear on the world. Or, you know, if I someday gain a bit more public recognition and clout, it may start to matter then how I self-identified back in 2010.
- Houshalter 12y ago
  The name of the organization is the Machine Intelligence Research Institute.
- jes 12y ago
  Would you be willing to share why you think this is heavy-handed?
rkachowski 12y ago
That Eliezer Yudkowsky is a pretty smart guy. Against all prior expectations (lol), I am now reading Harry Potter fan fiction and enjoying it.[1] The LessWrong sequences are poignant and concise and pushed me down the road of rational thought. I look forward to getting some of these books on my shelf. [1] http://hpmor.com/ http://hpmor.com/
xianshou 12y ago
This is a collection of Yudkowsky's blog posts on LessWrong, and it's making the rounds now because it forms the basis for most of Harry's rationalist insights in the extremely entertaining and recently finished Harry Potter and the Methods of Rationality. (The last chapter of HPMoR went up on Ultimate Pi Day - 3/14/15 - last week.) I picked it up on the advice of some friends, and while the editing is less than perfect, it is overall an impressively well-organized and cross-linked treatise on how to reason. At the heart of the book is the assertion that rationality is not about justifying why we are right, but thinking how we might be wrong, and paying careful attention to any evidence that puzzles us or could convince us to change our minds. The resulting philosophy is a combination of the falsification principle and the scientific method, applied to your own character. Highly recommended.
- Strilanc 12y ago
  Given that this compilation was published on March 11th, and hpmor finished on March 14th, I don't think it's making the rounds now because of the timing of hpmor. Their finish/release dates are simply close together.
  - TeMPOraL 12y ago
    I think it still may be - people are posting this around as an additional reading for those who liked HPMOR; number of people who read that fanfic is most likely much greater than the number of people that read the Sequences.
  - maaku 12y ago
    The release dates were purposefully close together.
    - jtheory 12y ago
      I don't have inside knowledge, but yeah, it seems extremely likely -- the sequences have been available for years, after all.
      - 12y ago
emiliobumachar 12y ago
I am currently reading, and loving, Yudkowsky's introduction to Quantum Mechanics from the blog: http://lesswrong.com/lw/pc/quantum_explanations/ http://lesswrong.com/lw/pc/quantum_explanations/ Does anyone know whether that sequence has made it to one of these books?
- pyrois 12y ago
  I quite liked his introduction! Be aware, however, that he dramatically overstates some things. For example, he claims that the many-worlds interpretation is obviously correct. This is basically wrong: it may be correct, but it certainly isn't obviously correct, and many scientists whose entire research is dedicated to quantum mechanics would disagree that many-worlds is correct at all. There is a long list of alternative explanations[0], with very little evidence to lend support to or contradict any of them (at least, any of the ones that are still around. Some have fallen by the wayside). [0]: https://en.wikipedia.org/wiki/Interpretations_of_quantum_mechanics https://en.wikipedia.org/wiki/Interpretations_of_quantum_mec...
  - hawkice 12y ago
    I think it's fair to say that it's "obvious" when optimizing for the same things Yudkowsky does. For instance, any issue having something to say about the Born rule seems to be his dump stat, if you will. This makes sense, because in the paper where Max Born introduced the rule, he suggests that it indicates only one possible correct interpretation (hint: it was not Many Worlds). That all being said, no one really disagrees about the predictions of the math, and at least many-worlds has less completely ridiculous misinterpretations by lay-people, so both ends of the spectrum are tolerable to me.
  - GFK_of_xmaspast 12y ago
    If you read his apologetics, it's literally faith-based science.
- adamzerner 12y ago
  Yes, it has. Chapter(?) S is titled 'Quantum Physics and Many Worlds'.
Houshalter 12y ago
There is an ebook here (http://lesswrong.com/lw/72m/an_epub_of_eliezers_blog_posts/ http://lesswrong.com/lw/72m/an_epub_of_eliezers_blog_posts/) where you can read it unedited as it was originally written. I don't know if anyone would want to do this, but it's how I read them. I imagine a lot of the content would be edited down or out, especially the standalone stuff. That's not necessarily a bad thing though.
- ciphergoth 12y ago
  I have edited that post to recommend the book over my ePub. You can always read my compilation after finishing the book, if you want to read the articles the book leaves out.
danbmil99 12y ago
Careful. Yudkowski and LW have a dark side. (google Roko's Basilisk) Rationality applied to ethics is a sticky wicket. You have to subscribe to some metric of "good" in order to weigh and compare outcomes. EY's definition of good is ruthlessly utilitarian, and that has some extreme, nay dire consequences for certain thought experiments. Consume this philosophy at your own risk.
- Moshe_Silnorin 12y ago
  Are you attempting reverse psychology?
  - biesnecker 12y ago
    I'm pretty sure it's a typo, but it's fun thinking about what reverse physiology would even be.
    - zyxley 12y ago
      Well, if you reverse everything, you've got predominantly left-handed nega-humans made out of antimatter.
- Houshalter 12y ago
  The Roko's Basilisk thing is way overblown. The author addressed the issue here: https://www.reddit.com/r/xkcd/comments/2myg86/xkcd_1450_aibox_experiment/cm8vn6e https://www.reddit.com/r/xkcd/comments/2myg86/xkcd_1450_aibo... And of course some of the topics he writes about are always going to be controversial. But utilitarianism is hardly radical.
  - zyxley 12y ago
    > Which - keeping in mind that at the time I had absolutely no idea this would all blow up the way it did - caused me to yell quite loudly at Roko for violating ethics given his own premises, I mean really, WTF? You're going to get everyone who reads your article tortured so that you can argue against an AI proposal? In the twisted alternate reality of RationalWiki, this became proof that I believed in Roko's Basilisk, since I yelled at the person who invented it without including twenty lines of disclaimers about what I didn't necessarily believe. I don't understand what Yudkwosky's trying to argue there. "Even though I don't actually believe in it, I yelled at him for invoking the wrath of the nonexistent basilisk that I don't believe in"?
Kutta 12y ago
I'm happy to see this in the preface (where EY looks back and details what he thinks he did wrong): "Today I would write more courteously, I think. The discourtesy did serve a function, and I think there were people who were helped by reading it; but I now take more seriously the risk of building communities where the normal and expected reaction to low-status outsider views is open mockery and contempt. Despite my mistake, I am happy to say that my readership has so far been amazingly good about not using my rhetoric as an excuse to bully or belittle others. (I want to single out Scott Alexander in particular here, who is a nicer person than I am and an increasingly amazing writer on these topics, and may deserve a part of the credit for making the culture of Less Wrong a healthy one.)"
- gwern 12y ago
  For those interested in Scott: his LW posts can be found at http://lesswrong.com/user/Yvain/submitted/ http://lesswrong.com/user/Yvain/submitted/ , his old blog at http://squid314.livejournal.com/ http://squid314.livejournal.com/ , his new blog at http://slatestarcodex.com/ http://slatestarcodex.com/ , and a tumblr for shorter things at http://slatestarscratchpad.tumblr.com/ http://slatestarscratchpad.tumblr.com/ If you're not sure where to start, a number of his writings have been received well on HN: https://hn.algolia.com/?query=slatestarcodex&sort=byPopularity&prefix&page=0&dateRange=all&type=story https://hn.algolia.com/?query=slatestarcodex&sort=byPopulari...
  - pja 12y ago
    gwern, Scott makes a specific point of not linking to his blog from his LJ in his penultimate LJ post - he might prefer that you didn’t make that link here?
    - gwern 12y ago
      He went back and hid the more sensitive LJ entries, IIRC, so I don't think it should be a problem.
maaku 12y ago
For those who are interested, I will be moderating a online reading group covering the book here: http://lesswrong.com/r/discussion/lw/lx0/rationality_from_ai_to_zombies_online_reading/ http://lesswrong.com/r/discussion/lw/lx0/rationality_from_ai...

113 points by rayalez 12y ago

hunnypot 12y ago

I'll just say that "intelligence.org" is as heavy-handed as Google buying the .dev TLD.

jtheory 12y ago
I don't think I could bring myself to be a "Bright" for that reason (see the-brights.net)... Vague counterpoint -- this is an ebook collecting essays from lesswrong.com, which is a bit more subtle as names go.
- adamzerner 12y ago
  > I don't think I could bring myself to be a "Bright" for that reason (see the-brights.net)... A name is enough of a reason on it's own?
  - jtheory 12y ago
    Sure. That is, my understanding of reality is the same, regardless. I think the goals of "The Brights" are laudable, generally. But I didn't register on their website, and wouldn't really want my photo and identify featured as a Bright, because the name just doesn't feel right to me. How you identify yourself makes a difference to how you're judged, and hence the influence you can bring to bear on the world. Or, you know, if I someday gain a bit more public recognition and clout, it may start to matter then how I self-identified back in 2010.
Houshalter 12y ago
The name of the organization is the Machine Intelligence Research Institute.
jes 12y ago
Would you be willing to share why you think this is heavy-handed?

rkachowski 12y ago

That Eliezer Yudkowsky is a pretty smart guy. Against all prior expectations (lol), I am now reading Harry Potter fan fiction and enjoying it.[1] The LessWrong sequences are poignant and concise and pushed me down the road of rational thought. I look forward to getting some of these books on my shelf. [1] http://hpmor.com/ http://hpmor.com/

xianshou 12y ago

This is a collection of Yudkowsky's blog posts on LessWrong, and it's making the rounds now because it forms the basis for most of Harry's rationalist insights in the extremely entertaining and recently finished Harry Potter and the Methods of Rationality. (The last chapter of HPMoR went up on Ultimate Pi Day - 3/14/15 - last week.) I picked it up on the advice of some friends, and while the editing is less than perfect, it is overall an impressively well-organized and cross-linked treatise on how to reason. At the heart of the book is the assertion that rationality is not about justifying why we are right, but thinking how we might be wrong, and paying careful attention to any evidence that puzzles us or could convince us to change our minds. The resulting philosophy is a combination of the falsification principle and the scientific method, applied to your own character. Highly recommended.

Strilanc 12y ago
Given that this compilation was published on March 11th, and hpmor finished on March 14th, I don't think it's making the rounds now because of the timing of hpmor. Their finish/release dates are simply close together.
- TeMPOraL 12y ago
  I think it still may be - people are posting this around as an additional reading for those who liked HPMOR; number of people who read that fanfic is most likely much greater than the number of people that read the Sequences.
- maaku 12y ago
  The release dates were purposefully close together.
  - jtheory 12y ago
    I don't have inside knowledge, but yeah, it seems extremely likely -- the sequences have been available for years, after all.
    - 12y ago

emiliobumachar 12y ago

I am currently reading, and loving, Yudkowsky's introduction to Quantum Mechanics from the blog: http://lesswrong.com/lw/pc/quantum_explanations/ http://lesswrong.com/lw/pc/quantum_explanations/ Does anyone know whether that sequence has made it to one of these books?

pyrois 12y ago
I quite liked his introduction! Be aware, however, that he dramatically overstates some things. For example, he claims that the many-worlds interpretation is obviously correct. This is basically wrong: it may be correct, but it certainly isn't obviously correct, and many scientists whose entire research is dedicated to quantum mechanics would disagree that many-worlds is correct at all. There is a long list of alternative explanations[0], with very little evidence to lend support to or contradict any of them (at least, any of the ones that are still around. Some have fallen by the wayside). [0]: https://en.wikipedia.org/wiki/Interpretations_of_quantum_mechanics https://en.wikipedia.org/wiki/Interpretations_of_quantum_mec...
- hawkice 12y ago
  I think it's fair to say that it's "obvious" when optimizing for the same things Yudkowsky does. For instance, any issue having something to say about the Born rule seems to be his dump stat, if you will. This makes sense, because in the paper where Max Born introduced the rule, he suggests that it indicates only one possible correct interpretation (hint: it was not Many Worlds). That all being said, no one really disagrees about the predictions of the math, and at least many-worlds has less completely ridiculous misinterpretations by lay-people, so both ends of the spectrum are tolerable to me.
- GFK_of_xmaspast 12y ago
  If you read his apologetics, it's literally faith-based science.
adamzerner 12y ago
Yes, it has. Chapter(?) S is titled 'Quantum Physics and Many Worlds'.

Houshalter 12y ago

There is an ebook here (http://lesswrong.com/lw/72m/an_epub_of_eliezers_blog_posts/ http://lesswrong.com/lw/72m/an_epub_of_eliezers_blog_posts/) where you can read it unedited as it was originally written. I don't know if anyone would want to do this, but it's how I read them. I imagine a lot of the content would be edited down or out, especially the standalone stuff. That's not necessarily a bad thing though.

ciphergoth 12y ago
I have edited that post to recommend the book over my ePub. You can always read my compilation after finishing the book, if you want to read the articles the book leaves out.

danbmil99 12y ago

Careful. Yudkowski and LW have a dark side. (google Roko's Basilisk) Rationality applied to ethics is a sticky wicket. You have to subscribe to some metric of "good" in order to weigh and compare outcomes. EY's definition of good is ruthlessly utilitarian, and that has some extreme, nay dire consequences for certain thought experiments. Consume this philosophy at your own risk.

Moshe_Silnorin 12y ago
Are you attempting reverse psychology?
- biesnecker 12y ago
  I'm pretty sure it's a typo, but it's fun thinking about what reverse physiology would even be.
  - zyxley 12y ago
    Well, if you reverse everything, you've got predominantly left-handed nega-humans made out of antimatter.
Houshalter 12y ago
The Roko's Basilisk thing is way overblown. The author addressed the issue here: https://www.reddit.com/r/xkcd/comments/2myg86/xkcd_1450_aibox_experiment/cm8vn6e https://www.reddit.com/r/xkcd/comments/2myg86/xkcd_1450_aibo... And of course some of the topics he writes about are always going to be controversial. But utilitarianism is hardly radical.
- zyxley 12y ago
  > Which - keeping in mind that at the time I had absolutely no idea this would all blow up the way it did - caused me to yell quite loudly at Roko for violating ethics given his own premises, I mean really, WTF? You're going to get everyone who reads your article tortured so that you can argue against an AI proposal? In the twisted alternate reality of RationalWiki, this became proof that I believed in Roko's Basilisk, since I yelled at the person who invented it without including twenty lines of disclaimers about what I didn't necessarily believe. I don't understand what Yudkwosky's trying to argue there. "Even though I don't actually believe in it, I yelled at him for invoking the wrath of the nonexistent basilisk that I don't believe in"?

Kutta 12y ago

I'm happy to see this in the preface (where EY looks back and details what he thinks he did wrong): "Today I would write more courteously, I think. The discourtesy did serve a function, and I think there were people who were helped by reading it; but I now take more seriously the risk of building communities where the normal and expected reaction to low-status outsider views is open mockery and contempt. Despite my mistake, I am happy to say that my readership has so far been amazingly good about not using my rhetoric as an excuse to bully or belittle others. (I want to single out Scott Alexander in particular here, who is a nicer person than I am and an increasingly amazing writer on these topics, and may deserve a part of the credit for making the culture of Less Wrong a healthy one.)"

gwern 12y ago
For those interested in Scott: his LW posts can be found at http://lesswrong.com/user/Yvain/submitted/ http://lesswrong.com/user/Yvain/submitted/ , his old blog at http://squid314.livejournal.com/ http://squid314.livejournal.com/ , his new blog at http://slatestarcodex.com/ http://slatestarcodex.com/ , and a tumblr for shorter things at http://slatestarscratchpad.tumblr.com/ http://slatestarscratchpad.tumblr.com/ If you're not sure where to start, a number of his writings have been received well on HN: https://hn.algolia.com/?query=slatestarcodex&sort=byPopularity&prefix&page=0&dateRange=all&type=story https://hn.algolia.com/?query=slatestarcodex&sort=byPopulari...
- pja 12y ago
  gwern, Scott makes a specific point of not linking to his blog from his LJ in his penultimate LJ post - he might prefer that you didn’t make that link here?
  - gwern 12y ago
    He went back and hid the more sensitive LJ entries, IIRC, so I don't think it should be a problem.

maaku 12y ago

For those who are interested, I will be moderating a online reading group covering the book here: http://lesswrong.com/r/discussion/lw/lx0/rationality_from_ai_to_zombies_online_reading/ http://lesswrong.com/r/discussion/lw/lx0/rationality_from_ai...