2 ms·
>intelligence not being fundamentally correlated with altruism I don't think were anywhere close to intelligence >a process that tries to teach a system to op
by ThrowawayTestr 3y ago
>intelligence not being fundamentally correlated with altruism
I don't think were anywhere close to intelligence
>a process that tries to teach a system to optimize for X might instead instead make it optimize for thing-that-leads-to-X
Just another software bug to fix.
>any optimization process powerful enough will eventually optimize for "develop agency, survive longer, accumulate more power"
Pascal's wager for nerds.
>There's a lot of literature written about AI safety
There's also a lot of science fiction about killer robots exterminating humanity, same genre of writing.
- PoignardAzur 3y ago> Pascal's wager for nerds. You're mixing up your pre-written dismissals. Instrumental convergence has nothing to do with Pascal's Wager. Pascal's Wager for nerds would be a somewhat valid answer to "AI has the potential to destroy the world therefore we should worry about it even if it's very unlikely", but that's not the argument I made.