4 ms·

"[Whole brain emulations] are frequently regarded as being potentially highly untrustworthy and dangerous, constituting an existential risk." Really? Are they

by 3JPLW 13y ago

"[Whole brain emulations] are frequently regarded as being potentially highly untrustworthy and dangerous, constituting an existential risk." Really? Are they? An ~8000 word article based on this one sentence, with weasel words and no real rationale. Also, you seem to be confounding the computing power required to emulate a brain ("straightforward engineering") with the new imaging techniques that would need to be developed to scan and "upload."

DanBC 13y ago
http://intelligence.org/files/WBE-Superorgs.pdf http://intelligence.org/files/WBE-Superorgs.pdf > Because such emulations could be freely copied and run at increased speeds, they might quickly outnumber humans and be capable of performing almost any task more cheaply. Standard economic models suggest this could produce tremendous economic growth, perhaps doubling the size of economies every few weeks or less, but also driving wages for most jobs below human subsistence level (Hanson, forthcoming). Many have suggested that such rapidly replicating and evolving minds could cause human extinction if not carefully controlled (Bostrom 2002; Yudkowsky 2008; Posner 2004; Friedman 2008; McAuliﬀe 2001; Joy 2000; Moravec 1999).
- mmagin 13y ago
  So, we're the new Neanderthals? While I'm somewhat convinced that human-equivalent or greater machine intelligence will likely happen within the next hundred years (or less), good luck convincing a majority of humans of that.
gwern 13y ago
> An ~8000 word article based on this one sentence, with weasel words and no real rationale. A more accurate summary would read "a 7900 word article on the little-known economics and vulnerabilities of modern chip fabs, with ~100 words explaining how the author came to be interested in the topic". You could replace the WBE bits with "I am an investor in Intel and was reading Taleb's _The Black Swan_ and began to wonder what black swans might affect my stock holdings", and the essay wouldn't be that different.
- 3JPLW 13y ago
  Fair enough. I guess I was more interested in the "why" than the "how" from the title. And given how thoroughly researched and annotated the "how" section was, I was surprised so little consideration was given to why in the world you'd want to do this in the first place.
  - gwern 13y ago
    > And given how thoroughly researched and annotated the "how" section was, I was surprised so little consideration was given to why in the world you'd want to do this in the first place. Yes, well... If I may quote Nietzsche: "Lessing, the most honest of theoretical men, dared to say that he took greater delight in the quest for truth than in the truth itself." I care very little for the 'why', but once the thought has arisen - 'how vulnerable are chip fabs, anyway?' - then I am completely fascinated with the 'how'.

by 3JPLW 13y ago

DanBC 13y ago

http://intelligence.org/files/WBE-Superorgs.pdf http://intelligence.org/files/WBE-Superorgs.pdf > Because such emulations could be freely copied and run at increased speeds, they might quickly outnumber humans and be capable of performing almost any task more cheaply. Standard economic models suggest this could produce tremendous economic growth, perhaps doubling the size of economies every few weeks or less, but also driving wages for most jobs below human subsistence level (Hanson, forthcoming). Many have suggested that such rapidly replicating and evolving minds could cause human extinction if not carefully controlled (Bostrom 2002; Yudkowsky 2008; Posner 2004; Friedman 2008; McAuliﬀe 2001; Joy 2000; Moravec 1999).

mmagin 13y ago
So, we're the new Neanderthals? While I'm somewhat convinced that human-equivalent or greater machine intelligence will likely happen within the next hundred years (or less), good luck convincing a majority of humans of that.

gwern 13y ago

> An ~8000 word article based on this one sentence, with weasel words and no real rationale. A more accurate summary would read "a 7900 word article on the little-known economics and vulnerabilities of modern chip fabs, with ~100 words explaining how the author came to be interested in the topic". You could replace the WBE bits with "I am an investor in Intel and was reading Taleb's _The Black Swan_ and began to wonder what black swans might affect my stock holdings", and the essay wouldn't be that different.

3JPLW 13y ago
Fair enough. I guess I was more interested in the "why" than the "how" from the title. And given how thoroughly researched and annotated the "how" section was, I was surprised so little consideration was given to why in the world you'd want to do this in the first place.
- gwern 13y ago
  > And given how thoroughly researched and annotated the "how" section was, I was surprised so little consideration was given to why in the world you'd want to do this in the first place. Yes, well... If I may quote Nietzsche: "Lessing, the most honest of theoretical men, dared to say that he took greater delight in the quest for truth than in the truth itself." I care very little for the 'why', but once the thought has arisen - 'how vulnerable are chip fabs, anyway?' - then I am completely fascinated with the 'how'.