4 ms·

> A superintelligence will be able to convince anyone it wants to do anything for it. This "just so" claim has always struck me as quite a reach. Humans are ma

> A superintelligence will be able to convince anyone it wants to do anything for it. This "just so" claim has always struck me as quite a reach. Humans are manipulable, yes, especially by (perceived) group pressure, but there are still limits. Especially if the human can easily just mute the AI's output. It's like when people believe that a sufficiently clever hacker can get a given program to do anything. There are millions of security vulnerabilities, yes, but you're unlikely to find a RCE in the program `true`. But instead of the 80s wizard hacker trope we're going back to the older B-movie hypnotist who can take full control of any person by talking > You are literally linking to the person who is the most prominent voice of AI existential risk, who is seriously suggesting doing exactly that. There's a few links in the article - which person are you referring to? Being the most prominent voice calling something unsafe is not an inherently valuable qualification. The most vocal opponents of nuclear power tend to have a poor understanding of it

cjbprime 3y ago
> This "just so" claim has always struck me as quite a reach. Humans are manipulable, yes, especially by (perceived) group pressure, but there are still limits. You could reimagine it as "convince most people to do anything, including killing people who are unconvinced" if you prefer. > There's a few links in the article - which person are you referring to? Sorry, this link, which is attached to "bomb" in the article: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no... > Being the most prominent voice calling something unsafe is not an inherently valuable qualification. That's true. But the article does something odd instead of making that critique -- it presents bombing GPU data centers as a reductio ad absurdum, suggesting if you really believed in AI risk then you would have to be willing to consider doing that. The people who really believe in AI risk already are willing to consider it, though.

cjbprime 3y ago

> This "just so" claim has always struck me as quite a reach. Humans are manipulable, yes, especially by (perceived) group pressure, but there are still limits. You could reimagine it as "convince most people to do anything, including killing people who are unconvinced" if you prefer. > There's a few links in the article - which person are you referring to? Sorry, this link, which is attached to "bomb" in the article: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no... > Being the most prominent voice calling something unsafe is not an inherently valuable qualification. That's true. But the article does something odd instead of making that critique -- it presents bombing GPU data centers as a reductio ad absurdum, suggesting if you really believed in AI risk then you would have to be willing to consider doing that. The people who really believe in AI risk already are willing to consider it, though.