4 ms·

> And once you start probing the nether regions of transhumanist thought and run into concepts like Roko's Basilisk—by the way, any of you who didn't know about

> And once you start probing the nether regions of transhumanist thought and run into concepts like Roko's Basilisk—by the way, any of you who didn't know about the Basilisk before are now doomed to an eternity in AI hell—you realize they've mangled it to match some of the nastiest ideas in Presybterian Protestantism. I wish every discussion of transhumanism didn't have to involve Roko's Basilisk. It's not something anyone takes seriously (and very few ever did), but it has enough quirky weirdness that everyone seems to want to talk about it. Here's a quote from "Lesswrongwiki": > Roko's argument was broadly rejected on Less Wrong, with commenters objecting that an agent like the one Roko was describing would have no real reason to follow through on its threat > [...] > Less Wrong's founder, Eliezer Yudkowsky, banned discussion of Roko's basilisk on the blog for several years as part of a general site policy against spreading potential information hazards. This had the opposite of its intended effect: a number of outside websites began sharing information about Roko's basilisk, as the ban attracted attention to this taboo topic. Websites like RationalWiki spread the assumption that Roko's basilisk had been banned because Less Wrong users accepted the argument; thus many criticisms of Less Wrong cite Roko's basilisk as evidence that the site's users have unconventional and wrong-headed beliefs. - https://wiki.lesswrong.com/wiki/Roko's_basilisk https://wiki.lesswrong.com/wiki/Roko's_basilisk

scotty79 6y ago
For me this illustrates nicely why most discussions about AI and singularity are like counting how many angels can dance on the head of a pin. It's all like talking about nuclear weapons in 17th century. Sure you can but even if you reach some conclusions they most likely will be invalidated by first technical detail of the implementation of thing when it actually gets created.
LiquidSky 6y ago
Your comment (and your links) miss the point of mocking references to "Roko's Basilisk". Whether it's actually a widely-held belief of the Less Wrong/greater rationalist community is irrelevant, it's emblematic of the eye-rolling nonsense that they engage in. It's the exact secular equivalent of "How many angels can dance on the head of a pin?"[1], which was also not an actual accepted topic of medieval religious scholarly debate but an illustration of the usual absurdity of such debate (e.g., there was an actual centuries-long scholarly discussion of whether angels were sexless or had sexes). [1]https://en.wikipedia.org/wiki/How_many_angels_can_dance_on_the_head_of_a_pin%3F https://en.wikipedia.org/wiki/How_many_angels_can_dance_on_t...

scotty79 6y ago

For me this illustrates nicely why most discussions about AI and singularity are like counting how many angels can dance on the head of a pin. It's all like talking about nuclear weapons in 17th century. Sure you can but even if you reach some conclusions they most likely will be invalidated by first technical detail of the implementation of thing when it actually gets created.

LiquidSky 6y ago

Your comment (and your links) miss the point of mocking references to "Roko's Basilisk". Whether it's actually a widely-held belief of the Less Wrong/greater rationalist community is irrelevant, it's emblematic of the eye-rolling nonsense that they engage in. It's the exact secular equivalent of "How many angels can dance on the head of a pin?"[1], which was also not an actual accepted topic of medieval religious scholarly debate but an illustration of the usual absurdity of such debate (e.g., there was an actual centuries-long scholarly discussion of whether angels were sexless or had sexes). [1]https://en.wikipedia.org/wiki/How_many_angels_can_dance_on_the_head_of_a_pin%3F https://en.wikipedia.org/wiki/How_many_angels_can_dance_on_t...