5 ms·

I’m personally of the view that most AI safety people are delusional and irrationally terrified of progress. I remain open to having my mind changed on this ma

I’m personally of the view that most AI safety people are delusional and irrationally terrified of progress. I remain open to having my mind changed on this matter, but thus far, I’ve not seen a single good argument for restricting development of AI.

kypro 3y ago
AI centralises power, devalues human labour and at its limit is fundamentally uncontrollable. Personally I struggle to understand anyone who believes it's in humanities best interest to continue developing AI systems without significant limitations on the rate of progress. There's so many ways AI could go wrong I'd argue it's almost guaranteed that something will go wrong if we continue on this trajectory.
- Nasrudith 3y ago
  Literally any productive feedback loop where more resources produce better results centralizes power, any form of productivity improvement devalues human labor. Control is better described as a delusion in the first place.
PoignardAzur 3y ago
I mean... what you're communicating right now is "I have built a strawman in my head which I thoroughly hate, can anyone please come defend it before me"? I guess I'll try anyway. What's your opinion on the orthogonality thesis, ie intelligence not being fundamentally correlated with altruism? The concept of mesa-optimization, that a process that tries to teach a system to optimize for X might instead instead make it optimize for thing-that-leads-to-X (eg evolution made us want sex because it leads to reproduction, but now we have a lot of sex with birth control). The concept of instrumental convergence, that any optimization process powerful enough will eventually optimize for "develop agency, survive longer, accumulate more power"? There's a lot of literature written about AI safety, a lot of it from very skeptical people, but it's not like there's any single insight that can be reliably transmitted to a guy at a table with a "you're all idiots, change my mind" sign.
- ThrowawayTestr 3y ago
  >intelligence not being fundamentally correlated with altruism I don't think were anywhere close to intelligence >a process that tries to teach a system to optimize for X might instead instead make it optimize for thing-that-leads-to-X Just another software bug to fix. >any optimization process powerful enough will eventually optimize for "develop agency, survive longer, accumulate more power" Pascal's wager for nerds. >There's a lot of literature written about AI safety There's also a lot of science fiction about killer robots exterminating humanity, same genre of writing.
  - PoignardAzur 3y ago
    > Pascal's wager for nerds. You're mixing up your pre-written dismissals. Instrumental convergence has nothing to do with Pascal's Wager. Pascal's Wager for nerds would be a somewhat valid answer to "AI has the potential to destroy the world therefore we should worry about it even if it's very unlikely", but that's not the argument I made.
- unethical_ban 3y ago
  Judging by their posts, the person you responded to doesn't ever follow up on their commentary.
rnd0 3y ago
I don't believe they're terrified themselves. I believe they're grifters who are preying on the fears of others. A subtle, but important distinction.

kypro 3y ago

AI centralises power, devalues human labour and at its limit is fundamentally uncontrollable. Personally I struggle to understand anyone who believes it's in humanities best interest to continue developing AI systems without significant limitations on the rate of progress. There's so many ways AI could go wrong I'd argue it's almost guaranteed that something will go wrong if we continue on this trajectory.

Nasrudith 3y ago
Literally any productive feedback loop where more resources produce better results centralizes power, any form of productivity improvement devalues human labor. Control is better described as a delusion in the first place.

PoignardAzur 3y ago

I mean... what you're communicating right now is "I have built a strawman in my head which I thoroughly hate, can anyone please come defend it before me"? I guess I'll try anyway. What's your opinion on the orthogonality thesis, ie intelligence not being fundamentally correlated with altruism? The concept of mesa-optimization, that a process that tries to teach a system to optimize for X might instead instead make it optimize for thing-that-leads-to-X (eg evolution made us want sex because it leads to reproduction, but now we have a lot of sex with birth control). The concept of instrumental convergence, that any optimization process powerful enough will eventually optimize for "develop agency, survive longer, accumulate more power"? There's a lot of literature written about AI safety, a lot of it from very skeptical people, but it's not like there's any single insight that can be reliably transmitted to a guy at a table with a "you're all idiots, change my mind" sign.

ThrowawayTestr 3y ago
>intelligence not being fundamentally correlated with altruism I don't think were anywhere close to intelligence >a process that tries to teach a system to optimize for X might instead instead make it optimize for thing-that-leads-to-X Just another software bug to fix. >any optimization process powerful enough will eventually optimize for "develop agency, survive longer, accumulate more power" Pascal's wager for nerds. >There's a lot of literature written about AI safety There's also a lot of science fiction about killer robots exterminating humanity, same genre of writing.
- PoignardAzur 3y ago
  > Pascal's wager for nerds. You're mixing up your pre-written dismissals. Instrumental convergence has nothing to do with Pascal's Wager. Pascal's Wager for nerds would be a somewhat valid answer to "AI has the potential to destroy the world therefore we should worry about it even if it's very unlikely", but that's not the argument I made.
unethical_ban 3y ago
Judging by their posts, the person you responded to doesn't ever follow up on their commentary.

rnd0 3y ago

I don't believe they're terrified themselves. I believe they're grifters who are preying on the fears of others. A subtle, but important distinction.