8 ms·

I had previously heard of EA, and knew about paperclip maximizers, trolley problems, unintended consequences, and other risks associated with unregulated AI. Bu

I had previously heard of EA, and knew about paperclip maximizers, trolley problems, unintended consequences, and other risks associated with unregulated AI. But I wasn't aware of the connections between the two communities / families of ideas until recently. Because I don't consider myself sufficiently educated in these matters, I default to using GiveWell and their mosquito net recommendations as an effective low-effort approach to making monthly contributions to charity. Overall, is there anything about this approach which is objectionable? How do philosophers go from something like GiveWell to something like fending off the AI apocalypse? Is this ideological expansion inevitable?

TheDong 4y ago
EAs premise includes basically "all human lives are equal. Saving one stranger's life is just as valuable as saving your father's life (more valuable actually if the stranger has more years left than your dad), so you should donate your kidney to a stranger, not your father" "All human lives are equal" then expands to "and there are more unborn lives than there are currently living lives, so it's more important to save unborn kids in 200 years by diverting AI catastrophe now". I personally don't think unborn lives are worth anything, and as such I don't follow that jump. I don't feel a need to justify it either. I think there's two additional easy ways to veer off the EA course. The first is "how do we know EA is right about the charities?" The answer you'll get is basically "Sure, you can donate a kidney to your father and know it saved his life, you can verify that... but the people verifying your EA kidney donation does more good are smarter than you. They're rationalists. They do math. They know selling your kidney, which would have only saved one life, to a black market and getting enough money for 50 mosquito nets, which will save 20 lives, is the right deal. The spreadsheet says so." I do not buy that the people running and choosing EA charities are actually the smartest people, especially when it comes to having a positive impact on lives. Sure, they talk a big talk, but I know for a fact that when I give money to a local shelter, the money gets used for good. I know for a fact that when I donated to a certain EA charity I won't name, the money went into "AI research" which is to say into the pockets of a rich white dude. I know for a fact that the average EA rationalist I have met does not come off as a caring human. A second way that one can veer off the EA course is by deciding that one values some lives more than others. For example, if I live in a small city, I may decide I value the lives of orphans in this city more than the lives of the orphans in some third-world country. Sure, feeding an orphan in africa is cheaper so EA would say do that, but I may decide that the intangible value of improving my local community is worth more than "getting the most bang for my buck". I think this is also a totally reasonable argument, and in fact I would much rather live in a world where everyone tried to improve their local community than one where everyone used a spreadsheet to try and maximize their "best improvement per dollar".
- Sol- 4y ago
  How they value unborn future lives also veers into absurdity when you realize that the logical goal from a utilitarian perspective then is to populate the universe with trillions upon trillions of humans, just as long as each human at least has some infinitesimally small net happiness. You don't need them to be particularly happy, just happy enough to barely not want to kill themselves. I think I'd rather prioritize making the billions on earth we have right now very happy instead.
  - YellOh 4y ago
    Not all utilitarian roads lead to the repugnant conclusion. Ex. being "in favor of making people happy but neutral about making happy people" is another valid answer. A pure negative utilitarian is the other extreme - we shouldn't make any more lives at all, because every new person (& animal?) will inevitably suffer during their existence.
  - fasterik 4y ago
    That's a logical consequence of one possible view in population ethics, which is orthogonal to utilitarianism. There are equally counter-intuitive consequences of discounting future lives or not valuing them at all. What makes this specific point in time so much more important than every other point in time? Can we do anything we want to make our lives better regardless of future consequences?
- AgentME 4y ago
  Many EA people are into forms of longtermism but AI safety arguments don't rely on longtermism at all. Many people worried about AI safety think it will be relevant in well less than 200 years. (Though uh, would you really not care at all if the world ended in 200 years? This sounds much more alien to me than any form of longtermism, and I think most people would agree that society should make decisions to sustain itself.)
michaelt 4y ago
> How do philosophers go from something like GiveWell to something like fending off the AI apocalypse? It's simply that writers like to write, forum commenters like to comment, philosophy academics like to publish papers, and there's only so many ways you can rehash The Life You Can Save. Advocating for mosquito nets and praziquantel is like advocating that programmers should use version control - it may be true, but it's not going to drive traffic to your blog or impress people with how forward-thinking you are. The people who haven't expanded their ideology in debatable directions aren't getting to the front page of HN.

TheDong 4y ago

EAs premise includes basically "all human lives are equal. Saving one stranger's life is just as valuable as saving your father's life (more valuable actually if the stranger has more years left than your dad), so you should donate your kidney to a stranger, not your father" "All human lives are equal" then expands to "and there are more unborn lives than there are currently living lives, so it's more important to save unborn kids in 200 years by diverting AI catastrophe now". I personally don't think unborn lives are worth anything, and as such I don't follow that jump. I don't feel a need to justify it either. I think there's two additional easy ways to veer off the EA course. The first is "how do we know EA is right about the charities?" The answer you'll get is basically "Sure, you can donate a kidney to your father and know it saved his life, you can verify that... but the people verifying your EA kidney donation does more good are smarter than you. They're rationalists. They do math. They know selling your kidney, which would have only saved one life, to a black market and getting enough money for 50 mosquito nets, which will save 20 lives, is the right deal. The spreadsheet says so." I do not buy that the people running and choosing EA charities are actually the smartest people, especially when it comes to having a positive impact on lives. Sure, they talk a big talk, but I know for a fact that when I give money to a local shelter, the money gets used for good. I know for a fact that when I donated to a certain EA charity I won't name, the money went into "AI research" which is to say into the pockets of a rich white dude. I know for a fact that the average EA rationalist I have met does not come off as a caring human. A second way that one can veer off the EA course is by deciding that one values some lives more than others. For example, if I live in a small city, I may decide I value the lives of orphans in this city more than the lives of the orphans in some third-world country. Sure, feeding an orphan in africa is cheaper so EA would say do that, but I may decide that the intangible value of improving my local community is worth more than "getting the most bang for my buck". I think this is also a totally reasonable argument, and in fact I would much rather live in a world where everyone tried to improve their local community than one where everyone used a spreadsheet to try and maximize their "best improvement per dollar".

Sol- 4y ago
How they value unborn future lives also veers into absurdity when you realize that the logical goal from a utilitarian perspective then is to populate the universe with trillions upon trillions of humans, just as long as each human at least has some infinitesimally small net happiness. You don't need them to be particularly happy, just happy enough to barely not want to kill themselves. I think I'd rather prioritize making the billions on earth we have right now very happy instead.
- YellOh 4y ago
  Not all utilitarian roads lead to the repugnant conclusion. Ex. being "in favor of making people happy but neutral about making happy people" is another valid answer. A pure negative utilitarian is the other extreme - we shouldn't make any more lives at all, because every new person (& animal?) will inevitably suffer during their existence.
- fasterik 4y ago
  That's a logical consequence of one possible view in population ethics, which is orthogonal to utilitarianism. There are equally counter-intuitive consequences of discounting future lives or not valuing them at all. What makes this specific point in time so much more important than every other point in time? Can we do anything we want to make our lives better regardless of future consequences?
AgentME 4y ago
Many EA people are into forms of longtermism but AI safety arguments don't rely on longtermism at all. Many people worried about AI safety think it will be relevant in well less than 200 years. (Though uh, would you really not care at all if the world ended in 200 years? This sounds much more alien to me than any form of longtermism, and I think most people would agree that society should make decisions to sustain itself.)

michaelt 4y ago

> How do philosophers go from something like GiveWell to something like fending off the AI apocalypse? It's simply that writers like to write, forum commenters like to comment, philosophy academics like to publish papers, and there's only so many ways you can rehash The Life You Can Save. Advocating for mosquito nets and praziquantel is like advocating that programmers should use version control - it may be true, but it's not going to drive traffic to your blog or impress people with how forward-thinking you are. The people who haven't expanded their ideology in debatable directions aren't getting to the front page of HN.