5 ms·
EA is at least partly a moral and ethical pyramid scheme. You get to be noble and far-sighted, do relatively little real work, and you’ll never be judged on the
by thom 4y ago
EA is at least partly a moral and ethical pyramid scheme. You get to be noble and far-sighted, do relatively little real work, and you’ll never be judged on the outcomes because of the long timelines. And so you rope in more and more acolytes who both increase and accrue these same benefits. It seems unlikely that the fall from grace of prominent figures will affect the movement, given that it’s partly based on cognitive dissonance as it is.
All that said, the amount of energy we expend on AI safety probably shouldn’t be zero. But I choose to believe that if Roko’s basilisk is going to torture us over anything, it’ll be that we didn’t raise up millions of AI scientists from the cotton fields and sweatshops. Credit to the less prominent altruists doing that, or who at least have some form of compound interest built into their utility function which compels them to be kind, now.
- dbspin 4y agoRoko's basilisk makes no sense. Sans magic physics time travel - there's no motivation. One thing all A.I. is, no matter how nefarious, is goal directed. It might have poorly aligned goals, absurd goals, dangerous goals, goals it misunderstands, goals that contradict etc. But ultimately it proceeds toward some form of goal directed reward function. There's zero reward in punishing someone for past behaviour - except in the context of altering their future behaviour. Since the A.I. would have to already exist (and there's no reason to believe it would be incentivised to even encourage humans to create other, rival A.Is, with potentially differing goals) before it could punish anyone - Q.E.D.: Roko's basilisk has essentially zero likelihood. Worse - it's an essentially christian eschatological idea. What is this hypothetical A.I. but an avenging angel, a righteous judge at the end of time? It's pure religion. Dark, manipulative, blood sacrifice religion at that.
- PartiallyTyped 4y ago> There's zero reward in punishing someone for past behaviour A similar argument regarding Roko's Basilisk is that it is effectively a prisoner's dilemma, where the Basilisk torturing us and us not helping it is the Defect-Defect scenario. It makes no sense for either party to defect. > except in the context of altering their future behaviour. It really seems like a very contrived version of Pascal's mugging.
- AgentME 4y agoPeople in EA donate lots of money to causes that help people today. AI is just one of the more interesting subjects they often talk about. AI safety isn't about Roko's basilisk, or the idea that AI will take revenge on us for our moral choices. It's about the risk that an AI smarter than us will be made and built to pursue some goal without caring about our well-being.
- PartiallyTyped 4y ago> It's about the risk that an AI smarter than us It's also about the risk that AI is dumber than what we expect it to be and fails to properly solve the issue. A sufficiently smart AI should be able to understand our intentions and - assuming it is aligned with us - attempt to fulfill that purpose. Dumb "AI" will fail in ways that don't actually solve the task, or solve it only on the surface / do reward hacking.
- AgentME 4y ago>assuming it is aligned with us If we don't figure out alignment, we can't make this assumption. This is the part that AI safety people are concerned with making reality.
- PartiallyTyped 4y agoI think I was misunderstood. My point is that not only do we need to consider the cases where the AI is smarter than us, but also the cases where it is dumber than we expect it. An aligned agent will not fall into stupid errors, thus solving the lower bound of performance is a necessity for complete alignment.
- AgentME 4y agoOhh, right on. Definitely agree now that I've re-read it right. I was a little confused by the first sentence of your other post and thought you were putting forward a disagreement in the second sentence.
- 4y ago