7 ms·

When I first heard of Effective Altruism, it was all about vaccines and mosquito nets — low cost solutions with clear, positive outcomes. I found that convincin

When I first heard of Effective Altruism, it was all about vaccines and mosquito nets — low cost solutions with clear, positive outcomes. I found that convincing, and even donated some money to some of the recommendations. When I checked back a few years later, the community seemed to be all about surviving the Rapture, I mean “Singularity”, summoning God, I mean the “Friendly AI”, and maximizing the happiness of the expected 10 to the trillion lives in Heaven, I mean the “Simulation”. I found all of that a lot less convincing. There are several social reasons why this shift happened, but the root problem, to me, is utilitarianism taken to a dangerous extreme. Effective Altruism was arguably always rooted in some form of utilitarianism as a guiding principle. The problem is, the concept seems to have attracted some very logical, abstract thinkers who somehow didn’t realize when they started to lose contact with reality. Note that all of this is just my opinion, but here’s my point. I find it always problematic to assign a numerical goodness value (“utilons”) to a human life, though I concede that it can work as a heuristic in small, non-extreme cases. What definitely doesn’t work is to take that number, multiply it by population sizes and claim to have solved ethics. No matter how much you wish it was, ethics is simply not a subbranch a mathematics. To put it very bluntly: That road has been traveled before, and it leads to genocide. To be clear, this isn’t hyperbole. I won’t link any sources to limit exposure, but you can find articles by self-proclaimed altruists who argue that it’s better to lose a billion lives to climate change than to delay AI research by even one year. The suggested reason being that delaying the Ascent into Heaven — I mean the “Upload into the Simulation” — costs humanity a collective 10 to the gazillion utilons, compared to the mere billions of utilons potentially lost to the climate catastrophe. Just think about the implications.

beebmam 4y ago
Utilitarianism is the worst ethical framework, except for all the others
hedgew 4y ago
One reason why EA takes AI risks seriously is exactly because of how hard it is to define what is "good" or what "utility" makes humans happy.
avgcorrection 4y ago
EA seems very fragile. The opposite of robust. First of all you have the potential problem of it being reductive (if one only looks at statistics) by discarding things like lived experience. You might just end up sending a million malaria nets to Africa which might contaminate the wildlife. Then you have the problem that you talk about that high income, smart nerds can have absolutely unhinged theories about how the world is going to develop. Ethical behavior doesn’t take much philosophy. Be good to your family. Be good to your friends. Be good to your community. Be good to people who you for some ideological reason think that you should be good to or serve (here philosophy reintroduces itself). Be decent to people who are reasonable. It’s much simpler, intellectually speaking, to figure out if you are on the right track by using interactions as cues. Following through might be more difficult.
- TedDoesntTalk 4y ago
  > Be good to your family. Be good to your friends. Be good to your community. Be good to people who you for some ideological reason think that you should be good to… Be decent to people who are reasonable. That’s easy. More challenging is being good to everyone, always. People you don’t like. People who push your buttons or aggravate you. Your estranged family member. The foolish man. The obnoxious bore. The guy who has polar opposite political beliefs. Try it for a day.
- pjscott 4y ago
  > You might just end up sending a million malaria nets to Africa which might contaminate the wildlife. EAs have long since internalized this idea, and do actually look at the reality on the ground. For example, here's a citation-laden summary of the empirical evidence that those malaria bed nets are helping, both on net and on the margin: https://www.givewell.org/international/technical/programs/insecticide-treated-nets https://www.givewell.org/international/technical/programs/in...
- nmca 4y ago
  But this doesn't get you to "be vegetarian" or "stop torturing the kids in that ritual with the bullet ants", both of which are pretty obviously right in any framework that doesn't overweight the action and values of your peers and family.
Smaug123 4y ago
I think there's a strong selection bias here on the arguments you're reading. It's pretty uncontroversial that vaccines and mosquito nets are generally good; although one could argue about exactly how good they are, there's not much room left for fruitful discussion about them. (Such discussion was done to death years ago.) What's left as the arena for discussion, where there may be new things to say, are the points which are hard to resolve. The fact that most talk is about controversial things does not mean most EA-affiliated people care about controversial things; it could simply mean that the uncontroversial things were fully decided years ago.
- notahacker 4y ago
  > It's pretty uncontroversial that vaccines and mosquito nets are generally good; although one could argue about exactly how good they are, there's not much room left for fruitful discussion about them. But one of the main focuses of EA is arguing about "exactly how good they are", and in particular the assumption that a cause whose general good was "fully decided years ago" shouldn't be seen as a good destination for philanthropic dollars without the organization focusing on providing metrics and RCTs to back up the idea that it's more beneficial than other definitely-has-some-positive-impact causes Applying more scrutiny to aid organizations is fine and some of their scepticism of (e.g) microfinance is completely justified, but saying "but you can't prove that this money is spent as efficiently as they could have been" to sending stuff to poor people in South Asia or "you can't prove this will ever work" to funding African entrepreneurs isn't particularly compatible with recommending money is sent to more opaque organisations paying expensive developers to pursue passion projects on because it's theoretically possible the organization will have a non-zero and non-negative impact on a possible future harm which might theoretically be big. Obviously not all EAs do this and many have very strong and consistent donation guidelines or are more interested in the maximise my personal potential to contribute side of EA than the demand more from aid agencies side, but it was the founder of GiveWell that recommended $30m of philanthropic cash was spent on buying a board seat at the non-profit subsidiary of extremly opaque and extremely well funded corporation OpenAI, which clashes quite hard with his earlier body of work. EAs who are interested in AI safety obviously aren't at all unusual in seeing moonshot research as a valid use of philanthropic dollars, but most people funding universities aren't simultaneously questioning the value of organizations for feeding people for not providing enough metrics to prove they're feeding the right people. (It cuts both ways too: the arguments for investing in "AI safety" - diversification in the face of uncertainty - are arguments against the EA principle of maximising the efficiency of philanthropic dollar based on rigorous analysis of different organizations' return)
wrnr 4y ago
Helping people can be an incredibly unthankful job, take for example the reddit user who's mom is a crack whore and to help her he drives around one day in the week to look for her and buys her something to eat all the while not being sure if she still knows who he is. EA just pisses me off, where on your list is this poor woman or her son and forgive me for asking but where on your list am I? Do good things and try to get away with them.
MrMan 4y ago
there is certainly a genocidal arrogance about EA which truly frightens me. The whole idea of Altruism itself is colonial and relies on a state where one group grovels and another either saves them or doesn't according to whim. It also reifies in-group out-group thinking. oh well.
Recursing 4y ago
That's the most controversial part of the movement, and the one that's most talked about. But most of EA is still focused on Global Health and Development, Scientific Research, Farmed Animal Welfare, and so on. There was a recent post exploring this, with some interesting comments https://forum.effectivealtruism.org/posts/LRmEezoeeqGhkWm2p/ea-is-more-than-longtermism https://forum.effectivealtruism.org/posts/LRmEezoeeqGhkWm2p/...
- sanderjd 4y ago
  Yeah I hope this is right, that the other stuff (the AI / singularity stuff) is just noisier right now.
michaelt 4y ago
> When I first heard of Effective Altruism, it was all about vaccines and mosquito nets [...] When I checked back a few years later, the community seemed to be all about surviving the Rapture, I mean “Singularity” I feel exactly the same way - but having been present through a lot of it, perhaps I can add some context. Back in ~2009, Toby Ord was at 'The Oxford Uehiro Centre for Practical Ethics' and started 'Giving What We Can' which is exactly what you remember: Demonstrably cost-effective interventions like mosquito nets and deworming tablets. Measuring charity impact by quality-adjusted life years. Membership by pledging to donate 10% of your income to such causes. Approving of earning to give. The thing is, GWWC was basically complete when it launched. Their recommended most efficient charities today are the same ones they launched with. Oh, there were still some things to discuss - is it even more efficient to spend money on lobbying government to spend the foreign aid budget on those highly efficient causes? - but not enough to create a thriving discussion community or keep philosophy professors employed. But GWWC wasn't the only thing happening at Oxford's philosophy department at the time. GWWC, The Oxford Uehiro Centre, the Centre for Effective Altruism and the Future of Humanity Institute were all in the same place at the same time - and in some senses EA is a broader version of GWWC, sharing its spreadsheet-and-QALYs-based approach. Toby Ord's focus moved from global poverty to existential risks, and GWWC and EA just sort of got rolled up together. Thus creating this strange organisation - where one branch of the organisation would say the benefits of donating to medical research aren't quantifiable enough; while another branch works on things that are a hundred times more speculative than that. And as the 'existential risk' branch has a lot more to argue about - they're a much more visible part of the community.
- sanderjd 4y ago
  Thank you for this little history. I think this would be a very interesting book to read. The history of the movement(s), summarizing the different lines of thought and debate at different times. Has that book been written?
sanderjd 4y ago
You put my own discomfort really well. I immediately responded to efforts like GiveWell when I came across them, but all the abstract long-termism and especially the focus on AI just strikes me as smarty pants faffery and I really can't connect with it.
bradlys 4y ago
> When I checked back a few years later, the community seemed to be all about surviving the Rapture, I mean “Singularity”, summoning God, I mean the “Friendly AI”, and maximizing the happiness of the expected 10 to the trillion lives in Heaven, I mean the “Simulation”. I found all of that a lot less convincing. AFAIK - one reason for this shift is because A) people who (want to) work in ML/AI want jobs B) billionaires like to fund shit that strokes their ego (AI > mosquito nets) and C) people who work in the EA field need money too and turns out mosquitos nets aren't super sexy to fund. So, inherently, people in EA will stroke the ego of billionaires with rapture-like projects because they get funding from those projects and they'll probably say stuff that is a bit extreme (along with some psychos just being thrown into any crowd). I know people who work in EA. It's complicated and difficult to balance. You have to realize - people (billionaires) contribute often just to stroke their own ego... not because they only want to help the world. At best - you can get them to do a split investment on their pet AI project and then a bit on real EA stuff. Otherwise, they'll just not contribute anything - and then you have the same capitalistic issue that you'll always have... too much wealth being concentrated into the hands of a few. The people I know in EA are often like, "Well if I take any money out of these guys - it's better than them having it. They have so much and no one person should have that much. It's inefficiently used by their hoarding."
imtringued 4y ago
That subset is known as longtermism and it is just as horrible as it sounds. I just watched a bunch of dystopian video clips on YouTube and honestly I prefer extinction over that "end game". I find longtermism to be dishonest because it has nothing to do with long term thinking. The long term nonsense is just there crank up the utility score.

beebmam 4y ago

Utilitarianism is the worst ethical framework, except for all the others

hedgew 4y ago

One reason why EA takes AI risks seriously is exactly because of how hard it is to define what is "good" or what "utility" makes humans happy.

avgcorrection 4y ago

EA seems very fragile. The opposite of robust. First of all you have the potential problem of it being reductive (if one only looks at statistics) by discarding things like lived experience. You might just end up sending a million malaria nets to Africa which might contaminate the wildlife. Then you have the problem that you talk about that high income, smart nerds can have absolutely unhinged theories about how the world is going to develop. Ethical behavior doesn’t take much philosophy. Be good to your family. Be good to your friends. Be good to your community. Be good to people who you for some ideological reason think that you should be good to or serve (here philosophy reintroduces itself). Be decent to people who are reasonable. It’s much simpler, intellectually speaking, to figure out if you are on the right track by using interactions as cues. Following through might be more difficult.

TedDoesntTalk 4y ago
> Be good to your family. Be good to your friends. Be good to your community. Be good to people who you for some ideological reason think that you should be good to… Be decent to people who are reasonable. That’s easy. More challenging is being good to everyone, always. People you don’t like. People who push your buttons or aggravate you. Your estranged family member. The foolish man. The obnoxious bore. The guy who has polar opposite political beliefs. Try it for a day.
pjscott 4y ago
> You might just end up sending a million malaria nets to Africa which might contaminate the wildlife. EAs have long since internalized this idea, and do actually look at the reality on the ground. For example, here's a citation-laden summary of the empirical evidence that those malaria bed nets are helping, both on net and on the margin: https://www.givewell.org/international/technical/programs/insecticide-treated-nets https://www.givewell.org/international/technical/programs/in...
nmca 4y ago
But this doesn't get you to "be vegetarian" or "stop torturing the kids in that ritual with the bullet ants", both of which are pretty obviously right in any framework that doesn't overweight the action and values of your peers and family.

Smaug123 4y ago

I think there's a strong selection bias here on the arguments you're reading. It's pretty uncontroversial that vaccines and mosquito nets are generally good; although one could argue about exactly how good they are, there's not much room left for fruitful discussion about them. (Such discussion was done to death years ago.) What's left as the arena for discussion, where there may be new things to say, are the points which are hard to resolve. The fact that most talk is about controversial things does not mean most EA-affiliated people care about controversial things; it could simply mean that the uncontroversial things were fully decided years ago.

notahacker 4y ago
> It's pretty uncontroversial that vaccines and mosquito nets are generally good; although one could argue about exactly how good they are, there's not much room left for fruitful discussion about them. But one of the main focuses of EA is arguing about "exactly how good they are", and in particular the assumption that a cause whose general good was "fully decided years ago" shouldn't be seen as a good destination for philanthropic dollars without the organization focusing on providing metrics and RCTs to back up the idea that it's more beneficial than other definitely-has-some-positive-impact causes Applying more scrutiny to aid organizations is fine and some of their scepticism of (e.g) microfinance is completely justified, but saying "but you can't prove that this money is spent as efficiently as they could have been" to sending stuff to poor people in South Asia or "you can't prove this will ever work" to funding African entrepreneurs isn't particularly compatible with recommending money is sent to more opaque organisations paying expensive developers to pursue passion projects on because it's theoretically possible the organization will have a non-zero and non-negative impact on a possible future harm which might theoretically be big. Obviously not all EAs do this and many have very strong and consistent donation guidelines or are more interested in the maximise my personal potential to contribute side of EA than the demand more from aid agencies side, but it was the founder of GiveWell that recommended $30m of philanthropic cash was spent on buying a board seat at the non-profit subsidiary of extremly opaque and extremely well funded corporation OpenAI, which clashes quite hard with his earlier body of work. EAs who are interested in AI safety obviously aren't at all unusual in seeing moonshot research as a valid use of philanthropic dollars, but most people funding universities aren't simultaneously questioning the value of organizations for feeding people for not providing enough metrics to prove they're feeding the right people. (It cuts both ways too: the arguments for investing in "AI safety" - diversification in the face of uncertainty - are arguments against the EA principle of maximising the efficiency of philanthropic dollar based on rigorous analysis of different organizations' return)

wrnr 4y ago

Helping people can be an incredibly unthankful job, take for example the reddit user who's mom is a crack whore and to help her he drives around one day in the week to look for her and buys her something to eat all the while not being sure if she still knows who he is. EA just pisses me off, where on your list is this poor woman or her son and forgive me for asking but where on your list am I? Do good things and try to get away with them.

MrMan 4y ago

there is certainly a genocidal arrogance about EA which truly frightens me. The whole idea of Altruism itself is colonial and relies on a state where one group grovels and another either saves them or doesn't according to whim. It also reifies in-group out-group thinking. oh well.

Recursing 4y ago

That's the most controversial part of the movement, and the one that's most talked about. But most of EA is still focused on Global Health and Development, Scientific Research, Farmed Animal Welfare, and so on. There was a recent post exploring this, with some interesting comments https://forum.effectivealtruism.org/posts/LRmEezoeeqGhkWm2p/ea-is-more-than-longtermism https://forum.effectivealtruism.org/posts/LRmEezoeeqGhkWm2p/...

sanderjd 4y ago
Yeah I hope this is right, that the other stuff (the AI / singularity stuff) is just noisier right now.

michaelt 4y ago

> When I first heard of Effective Altruism, it was all about vaccines and mosquito nets [...] When I checked back a few years later, the community seemed to be all about surviving the Rapture, I mean “Singularity” I feel exactly the same way - but having been present through a lot of it, perhaps I can add some context. Back in ~2009, Toby Ord was at 'The Oxford Uehiro Centre for Practical Ethics' and started 'Giving What We Can' which is exactly what you remember: Demonstrably cost-effective interventions like mosquito nets and deworming tablets. Measuring charity impact by quality-adjusted life years. Membership by pledging to donate 10% of your income to such causes. Approving of earning to give. The thing is, GWWC was basically complete when it launched. Their recommended most efficient charities today are the same ones they launched with. Oh, there were still some things to discuss - is it even more efficient to spend money on lobbying government to spend the foreign aid budget on those highly efficient causes? - but not enough to create a thriving discussion community or keep philosophy professors employed. But GWWC wasn't the only thing happening at Oxford's philosophy department at the time. GWWC, The Oxford Uehiro Centre, the Centre for Effective Altruism and the Future of Humanity Institute were all in the same place at the same time - and in some senses EA is a broader version of GWWC, sharing its spreadsheet-and-QALYs-based approach. Toby Ord's focus moved from global poverty to existential risks, and GWWC and EA just sort of got rolled up together. Thus creating this strange organisation - where one branch of the organisation would say the benefits of donating to medical research aren't quantifiable enough; while another branch works on things that are a hundred times more speculative than that. And as the 'existential risk' branch has a lot more to argue about - they're a much more visible part of the community.

sanderjd 4y ago
Thank you for this little history. I think this would be a very interesting book to read. The history of the movement(s), summarizing the different lines of thought and debate at different times. Has that book been written?

sanderjd 4y ago

You put my own discomfort really well. I immediately responded to efforts like GiveWell when I came across them, but all the abstract long-termism and especially the focus on AI just strikes me as smarty pants faffery and I really can't connect with it.

bradlys 4y ago

> When I checked back a few years later, the community seemed to be all about surviving the Rapture, I mean “Singularity”, summoning God, I mean the “Friendly AI”, and maximizing the happiness of the expected 10 to the trillion lives in Heaven, I mean the “Simulation”. I found all of that a lot less convincing. AFAIK - one reason for this shift is because A) people who (want to) work in ML/AI want jobs B) billionaires like to fund shit that strokes their ego (AI > mosquito nets) and C) people who work in the EA field need money too and turns out mosquitos nets aren't super sexy to fund. So, inherently, people in EA will stroke the ego of billionaires with rapture-like projects because they get funding from those projects and they'll probably say stuff that is a bit extreme (along with some psychos just being thrown into any crowd). I know people who work in EA. It's complicated and difficult to balance. You have to realize - people (billionaires) contribute often just to stroke their own ego... not because they only want to help the world. At best - you can get them to do a split investment on their pet AI project and then a bit on real EA stuff. Otherwise, they'll just not contribute anything - and then you have the same capitalistic issue that you'll always have... too much wealth being concentrated into the hands of a few. The people I know in EA are often like, "Well if I take any money out of these guys - it's better than them having it. They have so much and no one person should have that much. It's inefficiently used by their hoarding."

imtringued 4y ago

That subset is known as longtermism and it is just as horrible as it sounds. I just watched a bunch of dystopian video clips on YouTube and honestly I prefer extinction over that "end game". I find longtermism to be dishonest because it has nothing to do with long term thinking. The long term nonsense is just there crank up the utility score.