14 ms·

I think the Effective Altruism movement really belies its own values and cause with the fact that one of its own funds is for supporting: > organizations that

I think the Effective Altruism movement really belies its own values and cause with the fact that one of its own funds is for supporting: > organizations that work on improving long-term outcomes for humanity. Grants will likely go to organizations that seek to reduce global catastrophic risks, especially those relating to advanced artificial intelligence. [1] Yes, an argument can be made that it's important to fund prevention of global catastrophes, as while they're unlikely compared to the immediate threat of malaria, they'll cause much greater damage, thus increasing risk. However, to consider artificial intelligence to be a potential global catastrophe at all, let alone the single one requiring extra funding, is mostly unfounded. We currently can barely even define what the actual risk is, let alone how to mitigate it. It's one thing to walk past homeless people in my city and not give them money, because I know that money could much more easily and effectively safe a life in malaria-ridden parts of the world. I think it's absolutely morally repugnant to walk past them and not give them money, because instead you're paying people to sit around thinking about AI. [1] https://app.effectivealtruism.org/funds/far-future https://app.effectivealtruism.org/funds/far-future

mlthoughts2018 8y ago
Isn’t this just a statement that you disagree either with the relative probability of an AI-related disaster or the relative degree of harm it could cause? It seems harsh to say it’s “morally repugnant” when in the end you’re just saying the way you assign probabilities and degrees of negative outcomes would lead you to invest in a different portfolio of charities or efforts than what this other group would do. (I’m not arguing for or against the correctness of a belief that AI poses attention- and funding-worthy threats. Only that these other people motivated to allocate resources to it have studied the problem in great detail and their sincere belief after looking into it is that it is worth being part of the overall portfolio of charity investment, and this really would (in their sincere judgment) mitigate big-scale harm, no different than fundamental research into global warming or drug-resistant bacteria, regardless of whether lay people can more easily envision the types of harm from those other threats.
- monadgonad 8y ago
  > Isn’t this just a statement that you disagree either with the relative probability of an AI-related disaster or the relative degree of harm it could cause? Yes, but the point of this movement is that they're supposed to be evidenced-based and maximising ROI. There is evidence for the existence and threat of global warming and drug-resistant bacterial. When you move away from charitable efforts whose effectiveness we can directly measure, you're not doing EA any more, just regular emotional/value-based charity. I do have different values and feelings about such non-evidence-based charity, and I think theirs are morally repugnant, especially when they claim to be doing EA.
  - mlthoughts2018 8y ago
    That's a fair point about moving into areas in which the effectiveness is hard to measure. However, even apart from AI threats, I'd argue that there are many areas like this that pose potentially great harm. For example, I think that we ought to spend more effort to prevent regulatory capture in corporate-friendly legislation. But whether we can find a metric we'd all agree on to target this from an empirical point of view is a very difficult question. It reminds me of Gilb's Law, "Anything can be measured in a way that is superior to not measuring it at all." So if your sincere beliefs were such that the threat posed is high enough, and total harm would be astronomical, then you might believe that accepting a present-day metric that has a lot of variance and which is hotly debated might still be ok. Empirical progress along that rough metric might be more valuable, still in an expected value sense, than progress in other possibly less-harmful areas even if they have more clearly defined metrics. Again, not arguing for or against, just trying to represent why someone might be sincere about investing in AI safety and why, under their particular beliefs and preferences, it could still be rational from an EA point of view, despite more ambiguity. There are always risks that a certain model's loss function does not correctly correspond to the goals they wish to optimize towards, or that there is a large amount of idiosyncratic noise in the observation of the loss criteria. But you can still account for the risks of these sources of error in an overall method that is still empirical.
  - savanaly 8y ago
    It's like calling someone morally repugnant because they agreed to give you money equal to the sum of two and two and then gave you three dollars, insisting that the math actually does come out that way. If they honestly do believe that, they are simply mistaken, not morally repugnant.
ogennadi 8y ago
I used to feel similarly until I realized that the EA movement isn't a hierarchical organization: it's just a bunch of totally separate orgs who have a common philosophy about how to do good in the world. Sure, OpenPhil funds AI research. But Givewell (who, last I checked, share the same office) has in their Top Charities list, only those that work in poor countries https://www.givewell.org/charities/top-charities https://www.givewell.org/charities/top-charities [2] https://www.givewell.org/charities/top-charities https://www.givewell.org/charities/top-charities
- monadgonad 8y ago
  Thank you; this has opened my eyes
- jaggederest 8y ago
  Agreed. I've loosely held an EA-like philosophy for about a decade and I think that OpenPhil orientation towards AI is pretty disappointing. I account for my time in terms of things like number of people saved from blindness or death due to malaria, and I definitely do not count future simulated persons as worthy of any of the same concern as actual humans who exist today.
  - edanm 8y ago
    Out of curiosity: why?
    - jaggederest 8y ago
      Because they don't exist. It's like predicating your actions on the possible future existence of the easter bunny. Why are these people not attempting to research the ability to contact or create deities, or perpetual motion machines? Because they don't exist.
  - danielbigham 8y ago
    I watched a debate involving William Macaskill last summer and he poses the hypothetical question: "You are outside a burning building and are told that inside one room is a child and inside another is a painting by Picasso. You can save one of them. To do the most good, which do you choose?" The point he's trying to illustrate is that, if you knew for certain that you could turn around and sell the Picaso for millions and use that money to purchase malaria bed nets, the expected number of lives saved by using the Picaso could be hundreds, and so there's a moral dilemma present. Like many hypothetical questions, this one feels a bit "off" or "unrealistic", but if you don't get hung up on the oddities, I think one can sense the essence of his question, and it reminded me of the point you're making here as well as a responder's question asking you why you think the way you do. I do think these questions are hard for us to wrap our heads around -- how to value high probability immediacy against somewhat uncertain non-proximal/non-immediate things that might be "much higher value". Part of my human brain goes splat when I try to weigh these things. In terms of the moral dilemma with the painting, I do have quite a bit of sympathy for the argument that one should do what they feel will produce the most good, which might be to save the painting and purchase malaria nets. My father on the other hand seemed to believe that to be absolutely morally wrong, which seems to be siding with your sentiments. Practically speaking, I think I'd almost certainly save the child's life, because one's human impulses would be so strong that they would override any high-and-lofty-rationality, and one wouldn't have time anyway to do deep analysis. But the question in a hypothetical sense does seem quite valid and hard.
skybrian 8y ago
Note that GiveWell is sticking to its original mission of evaluating charities based on scientific studies and isn't recommending AI funding. However I'll push back on this. If it's okay for the government to fund basic scientific research where the benefits are long-term and unpredictable, why not private individuals? Or are you saying research shouldn't be funded at all?
- monadgonad 8y ago
  I think it's fine for private individuals to do it (though I disagree with their choice), but not to claim that it's evidence-based Effective Altruism while doing so - it is an emotionally-led value judgment, just like donating to my local dog shelter would be (which I understand others may disgree with). That said, I do recognise now that effectivealtruism.org does not represent the entirety of the movement; I have no fault with what GiveWell are doing.
  - tlb 8y ago
    Of course, there'll never be evidence about the effectiveness of research into existential risks until it's too late. So there's no way to objectively allocate funding between, say, AI safety and disease pandemics. While I think it's important to fund such research, it's a good point that funding orgs should either be evidence-based or not. Funding some things based on evidence and other things based on hunches might be an unstable combination. Once hunches are allowed, they could easily drift into funding things where the evidence says it's ineffective, but it gets overruled.
    - mkirklions 8y ago
      You could probably calculate this. Since AI can kill 7,000,000,000 people, You could probably toss a non-zero probability on it. Any math people know how to model this? Even with a bunch of different variations for the probability, you could understand why AI is so dangerous. potentially. There are probably lots of non zero probability events like getting hit by a rogue quasar. The difference is we cannot stop the quasar and that humans are required to discover AI. I dont give money to other charities, I spend money on mine. We dont have any employees, just volunteers. I spent a total of 50$ on the LLC paperwork and 8$/mo for a website host. Everything else has went directly to the kids/families we are teaching.
justinpombrio 8y ago
> However, to consider artificial intelligence to be a potential global catastrophe at all, let alone the single one requiring extra funding, is mostly unfounded. As another commenter points out, EA is not a single unified group, but rather a disparate set of people who are united by the belief that we should put money where it has the largest effect. Some, but not all, of them are convinced by the arguments in favor of doing AI alignment research. If you aren't, then don't donate money for that, donate it to what you think is a more effective cause! I'm curious, though, how people come to the conclusion that AI risk is nonsense? Is it a gut reaction, or does it come from thinking or reading about the problem? Every popsci article I've seen on the topic has been atrocious, so kudos if you thought "nonsense!" after reading such an article. Scott Alexander has a pretty readable introduction to AI risk: http://slatestarcodex.com/superintelligence-faq/ http://slatestarcodex.com/superintelligence-faq/
- jerf 8y ago
  AI risk has a profile that is very, very difficult to deal with rationally. By that I don't just mean "think about rationally rather than emotionally", but that it is difficult to even use rational tools to analyze it. It is what most people would consider a very, very small probability of what is arguably the worst possible outcome (i.e., considered from a strictly materialist perspectiv, there are outcomes worse that "the total extinction of life on Earth"!). Tiny magnitude probabilities of huge magnitude disasters are mathematically unstable; very small shifts in the value of the probability, and relatively small changes in the log value of the disaster's size, result in radically different scenarios and correct risk mitigations. It's even worse if you try to draw the probability distribution of something like "given strong AI will be acheived within the next 100 years, what is the distribution of the resulting likely outcomes?" In this case, the uncertainties are so large that, again, people can come to radically different conclusions even assuming both sides are being generally rational. It asks people trying to draw that distribution to consider what the most likely outcome is of at least a radically super-intelligent being is, and more likely, an arbitrarily large community of radically super-intelligent beings. Who can seriously claim to have an accurate probability distribution of that? If we understood radically super-intelligent beings, we'd already be radically super-intelligent beings, so we are not good at modeling them almost by definition. But certainly by observation; I am opaque in many ways even to my 7 and 10 year old children, and as with most humans, they are already exceptionally intelligent as "intelligent systems" go. (A 5 year old of normal intelligence is already exceptionally intelligent as intelligent systems go.) The idea that I could model something that was even my clone otherwise but operating at a hundred times the speed is already absurd; add an actual increase in intelligence to it and I stand no chance. Our only commonality will be those things imposed on us by the universe; i.e., I can be confident it must consume some negentropy to survive, etc. But any higher-level actions would be impossible for me to model. People who think intelligence is unlike to emerge quickly are unlikely to consider it a serious risk. People who think it's unlikely for an intelligent being to augment itself, then use its augmentations to augment itself, in an explosion of uncontrollable intelligence, will not consider AI risk a very likely outcome. And it's not as if it's an irrational or impossible outcome; for all we know, while there may be low-hanging fruit above human intelligence it is absolutely entirely possible that O(work to increase intelligence) > O(ability to perform intelligence-increasing work as intelligence goes up). After all, I can't help but look up in the sky and observe that there does not seem to be a near-lightspeed expanding bubble of computronium coming my way, per the most dramatic fears of the singularity. We could end up with something very intelligent, even dangerously intelligent, but not end up with us waking up one day to a digital god among us.
patientplatypus 8y ago
To provide the counter argument I imagine the Effective Altruism folks may give (not that I necessarily support this position): A number of specialist AIs could potentially eliminate 50%+ of all jobs in the next 50 years (a generation). What does society look like when we don't have the safety nets to support a massive unemployed population? What could it mean for geo-politics if the rich countries population is simply rich because they have the most advanced algorithms, but most people don't work? We're talking about massive social unrest worldwide potentially - capitalism cannot necessarily survive this paradigm shift and since the fall of the Berlin Wall the world has no other social organization in modern times, unless we'd like to revert to totalitarian rule. This all presumes of course, that the first AGI they turn on doesn't just turn the universe into a pile of paperclips (ducks)
joe_the_user 8y ago
It's one thing to walk past homeless people in my city and not give them money, because I know that money could much more easily and effectively safe a life in malaria-ridden parts of the world. The homeless in America live a life that's close to as miserable as the poor anywhere. One of the problem of most charities is bureaucracy eating up funds. Despite a lot of claims by bureaucracy, people who are homeless can often spend the money better on themselves than a bureaucracy. I don't think there's any particularly good not to give to the homeless if you feel like it and have the money.
- kbenson 8y ago
  Not to defend not giving money to the homeless, but isn't the calculation more complex than that? It's not just about how much money the bureaucracy eats, it's also how effectively the remaining portion is used, and I would think that would include actual resources delivered to people. More simply put, which helps more people and in a larger amount, $10 in the US or $5 in some extremely poor part of the world, where food and services may be much cheaper? I don't know the answer, but I think it's not as simple as "half is taken by bureaucracy so I shouldn't give to a bureaucracy".
  - joe_the_user 8y ago
    Not to defend not giving money to the homeless, but isn't the calculation more complex than that? It's not just about how much money the bureaucracy eats, it's also how effectively the remaining portion is used, and I would think that would include actual resources delivered to people. Certainly, altogether the calculations are extremely complicated. There's whether a given organization knows what resources to provide, whether the provided resources are going to be used even if they are otherwise "right", etc. With all the complexity and uncertainty, directly giving to people seems like one entirely legitimate approach since it guarantees that people get resources, not that other necessary are problematic but certainly other approaches deserve certainty. Direct giving at least has "what you see is what you get".
  - jimmaswell 8y ago
    >Not to defend not giving money to the homeless Then I'll defend it. The money would be much better utilized donated to programs that try to target homelessness as a whole. The old "they'll just spend it on drugs/alcohol/etc" cliche is actually backed by studies: https://www.theatlantic.com/business/archive/2011/03/should-you-give-money-to-homeless-people/72820/ https://www.theatlantic.com/business/archive/2011/03/should-... Some points from the article: "We choose to donate money based on the level of perceived need. Beggars known this, so there is an incentive on their part to exaggerate their need, by either lying about their circumstances or letting their appearance visibly deteriorate rather than seek help." "If you travel to a poor city, for example, you'll find swarms of beggars by touristy locations. If the tourists become more generous, the local beggars don't get richer, they only multiply." Further, a controversial issue is that of fake homeless people, which undoubtedly exist. Some say the easiest way to tell is if they'll accept food instead of money. From what From what I remember of someone's experience relayed in a reddit thread (which I'll admit has the potential to be exaggerated or unreliable) the majority of homeless people who said they needed "money for food" wouldn't accept food or would throw it away as soon as they thought the giver wasn't looking, indicating they were actually just after money. If you feel like you absolutely must give directly to homeless people, carry around wrapped protein bars or something similar. Encouraging panhandling is just exacerbating the problems. For some anecdotes, the last time I went to a convention in Baltimore, I heard stories about multiple people being assaulted or verbally harassed by panhandlers. In one case the provocation was that the person "didn't give them enough." The area is full of career-beggars who have no interest in actually improving themselves, like this one guy who pretends to be a youth baseball coach to collect fake donations every year. I try to stay indoors as much as possible but they even come inside the hotel lobbies to try to scam people sometimes. My experience and hearing those assault stories made me wonder how many of the people on this site decrying homeless-deterrent architecture have actually been in an affected area before.
DoreenMichele 8y ago
Re homelessness in the US, someone recently shared this with me and it looks promising, though it seems to only be in Seattle at the moment: https://www.samaritan.city/ https://www.samaritan.city/
- emodendroket 8y ago
  I don't know that I really need an app to give beggars money or buy the Spare Change News from someone.
  - DoreenMichele 8y ago
    You absolutely don't need an app to give homeless individuals money. But when I was homeless, I found that it was vastly more helpful for me if someone gave me a few bucks than if I had to stand in line for hours, fill out reams of paper, etc to try to get some assistance. I am happy to encourage any model that fosters a little more direct giving of this sort. I am also looking to create a pilot program to try to help homeless individuals begin to develop an online income from the street as I did. Charity helps you survive another day, but you need an income of your own to get your life back and many programs seem to actively be against the idea of homeless people trying to work for a living. People seem to see it as cruel to expect them to. But earned income is the only real path out of dire poverty.
    - emodendroket 8y ago
      Fair enough. I also find it unfortunate that so many people seem to see themselves as freelance interrogators of the homeless asking about what they will do with the money they're given.
      - DoreenMichele 8y ago
        Yes, thank you for saying that. The problem runs deeper than you may realize. Even very well meaning people would walk up to me and ask me "Are you homeless? What's your name?" They didn't volunteer their name, address or socioeconomic status when asking such questions. They were trying to ascertain if I would need help. A much better question format that isn't so problematic would be along the lines of "I have X (clothes, blankets, whatever). Would you like to have that?" If someone isn't actually homeless but would be happy to have a free blanket, what do you care? Maybe that blanket will help prevent them from becoming homeless.
notahacker 8y ago
Even as a massive sceptic of Cyberpunk AI (and for that matter, the efficacy of preventing it by setting up charities to do AI research), I think people are perfectly entitled to spend their money on it if it's what they're interested in. I also think it's plausible some such programmes might create extremely commercially valuable byproducts even though they don't lead anywhere useful directly, much like the Apollo missions. Trouble is, if you invest enormous amounts of time and energy in arguing that a general belief stuff might work isn't good enough for philanthropy and even writing "why we don't recommend" articles about specific charities dispensing aid that don't deprive enough people of it to have a control group for RCTs, you deserve every criticism you get when you then funnel foundation money to very well-funded AI startups with unfalsifiable solutions to a purely hypothetical problem, even if you didn't have close personal relationships with people working for them. Even though I consider some EA analysis to be good and well reasoned (and they're certainly not the only people making evidence based bearish arguments about, say, microfinance) it's a little difficult to see how AI research could possibly pass a decision-making heuristic so supposedly rigorous that it writes off sanitation as an area to invest in because the estimations of diarrhea reduction aren't blinded. Ultimately, EAs have their cognitive biases just like anyone else, and I'm more sympathetic than they are to aid organizations' view that conducting RCTs is difficult and expensive and they don't want people suffering in control groups to quantify how well common sense healthcare interventions work, and disagree that such organizations should be held to higher standards of analytical rigour than AI researchers hoping to be Sarah Connor.
throwawayjava 8y ago
> I think it's absolutely morally repugnant to walk past them and not give them money, because instead you're paying people to sit around thinking about AI. This is not a new argument. Sister Mary Jucunda, a Zambian nun, wrote to a NASA scientist (Ernst Stuhlinger) in the early 70's with a similar critique of funding for space research. You can read Ernst's cogent reply here: http://www.lettersofnote.com/2012/08/why-explore-space.html http://www.lettersofnote.com/2012/08/why-explore-space.html My favorite excerpt follows: > About 400 years ago, there lived a count in a small town in Germany. He was one of the benign counts, and he gave a large part of his income to the poor in his town. This was much appreciated, because poverty was abundant during medieval times, and there were epidemics of the plague which ravaged the country frequently. One day, the count met a strange man. He had a workbench and little laboratory in his house, and he labored hard during the daytime so that he could afford a few hours every evening to work in his laboratory. He ground small lenses from pieces of glass; he mounted the lenses in tubes, and he used these gadgets to look at very small objects. The count was particularly fascinated by the tiny creatures that could be observed with the strong magnification, and which he had never seen before. He invited the man to move with his laboratory to the castle, to become a member of the count's household, and to devote henceforth all his time to the development and perfection of his optical gadgets as a special employee of the count. > The townspeople, however, became angry when they realized that the count was wasting his money, as they thought, on a stunt without purpose. "We are suffering from this plague," they said, "while he is paying that man for a useless hobby!" But the count remained firm. "I give you as much as I can afford," he said, "but I will also support this man and his work, because I know that someday something will come out of it!" > Indeed, something very good came out of this work, and also out of similar work done by others at other places: the microscope. It is well known that the microscope has contributed more than any other invention to the progress of medicine, and that the elimination of the plague and many other contagious diseases from most parts of the world is largely a result of studies which the microscope made possible. I think there are many parallels between AI/ML and the microscope, and I think safety research is a very reasonable inquisitive lens for developing these new technologies.
- jopsen 8y ago
  Well, fighting poverty also grows the economy and number of brains available for research in the future. Today we are bringing people out of poverty for good, much charity isn't money in a bottomless pit. The pit has a bottom as evident by the very realistic goal to end extreme poverty by 2030. Still basic research is important, because it has such a long tail. Just saying both approaches have validity and will bring massive progress.
jlebar 8y ago
However, to consider artificial intelligence to be a potential global catastrophe at all, let alone the single one requiring extra funding, is mostly unfounded. We currently can barely even define what the actual risk is, let alone how to mitigate it. Although I happen agree with you personally, I don't think we should commit the fallacy of assuming that because their position seems absurd to us that it comes from a place of bias or ignorance. To the contrary, when I listen to Holden and other EA leaders talk, it's clear they've spent way more time thinking about this stuff than I have. He is and they are thoughtful and humble about exactly the questions you pose: How much should we weight "known good" good done today vs "potential good" done in the future, how confident should we be in our ability to predict the future, etc. As one example, Open Phil has hired historians to conduct research on how well people have (in the past) been able to predict the future, precisely to inform their thinking in this regard. Holden is also open about how his thoughts on the importance of AI alignment have changed over time. Again, we can disagree with him (as I do). But we absolutely should not claim that because we disagree, his position must be out of a place of thoughtlessness or bias.
- jlebar 8y ago
  See e.g. https://80000hours.org/podcast/episodes/holden-karnofsky-open-philanthropy/ https://80000hours.org/podcast/episodes/holden-karnofsky-ope...
- monktastic1 8y ago
  It's perfectly reasonable for them to hold that view. The unfortunate thing is to insist that it's the most rational and/or correct view. To say that they're biased isn't an insult. We all have biases, and it's useful to recognize and admit them.
  - pas 8y ago
    Could you link/cite where they insist that it's the Right Thing?
    - jonny_eh 8y ago
morgante 8y ago
Effective Altruism is not a single monolithic organization and I think you really go too far to assume all/most advocates think AI research is more important than feeding the homeless. Personally, I wholeheartedly subscribe to effective altruism and also don't think AI research is a good use of my funds. All of my donations go to malaria relief. Effective altruism is about principles, not particular causes.
tomcam 8y ago
The painful truth is that it is morally repugnant to donate. I hate saying this, by the way. If you had any experience with outreach, You would understand it is actually quite destructive to give them money, and it is much better to donate to, for example, Union Gospel Mission, a place in Seattle that does excellent homeless outreach. The city in fact contracts out to them for their services.
- Viliam1234 8y ago
  > The painful truth is that it is morally repugnant to donate. I hate saying this, by the way. I hate that you said it without explaining what you meant. :P Seriously, when you say "it is morally repugnant to donate" and "it is much better to donate to X" in the same paragraph, I am not sure what to think about it. Do you think that donating to X is still morally repugnant, although a bit less than donating to other causes? Or that donating can either be wrong or right depending on who you donate to (i.e. how the money is going to be used)? Because the latter seems like something that Effective Altruists would totally agree with.
  - tomcam 8y ago
    Sorry, my comment was quite unclear. I meant it is dangerous to street people to give the money directly. It is much better to give it to you in the Stabley Schomann organization that knows how to deal with their problems.
T2_t2 8y ago
I think your reply is interesting, as it exactly the sort of thing that Effective Altruism talks about. The example you gave is endemic of the way we think of charity precisely because it deals with both emotion and things we can individually see. Effective Altruism aims to make charity about things that don't directly affect us, that we can't see, and find areas that where the "bang per buck" is high. Seattle spent $195 million on the homelessness according to this: https://www.seattletimes.com/seattle-news/homeless/how-much-do-seattle-and-king-county-spend-on-homelessness/ https://www.seattletimes.com/seattle-news/homeless/how-much-..., to help a group that this advocacy organisation http://www.homelessinfo.org/what_we_do/one_night_count/ http://www.homelessinfo.org/what_we_do/one_night_count/ puts at 10,000 people. I wonder how a person can rationally look at that - $195 million at ~$19.5K per person and think they can make a dent. Compare that to https://www.givedirectly.org/basic-income https://www.givedirectly.org/basic-income and doubling the poorest of the poor's daily income by giving them $1 a day for 12 years. These are people I can't see and don't have to think about, yet for less than $5000 I can provide someone with security and an income for over a decade. If I give $70 a week for 12 years, I can take 10 people out of abject poverty for 12 years. That's fractionally above my coffee budget. That's the question Effective Altruism wants people to ask themselves. How can I, with the amount of money I have, do the most tangible good in the world, and what is an equation that helps me decide. Far from being "absolutely morally repugnant" to walk past a homeless person and preference things we can't see, I think it is morally honest to look at the risks humanity faces, and preference things that are under funded, where $1 can go far. That takes several forms, both immediate - like the Give Directly example - versus long term, where risks that could wipe out our species are considered. I think a strong argument can be made that, for minimal expense, it would be possible to have an affect on a potentially species destroying technology like AI, and that the money spent there is more likely to do good than adding that to an already well funded area. That's catastrophic risk, minimal expense vs human suffering and high cost. How you solve for the equilibrium there.

mlthoughts2018 8y ago

Isn’t this just a statement that you disagree either with the relative probability of an AI-related disaster or the relative degree of harm it could cause? It seems harsh to say it’s “morally repugnant” when in the end you’re just saying the way you assign probabilities and degrees of negative outcomes would lead you to invest in a different portfolio of charities or efforts than what this other group would do. (I’m not arguing for or against the correctness of a belief that AI poses attention- and funding-worthy threats. Only that these other people motivated to allocate resources to it have studied the problem in great detail and their sincere belief after looking into it is that it is worth being part of the overall portfolio of charity investment, and this really would (in their sincere judgment) mitigate big-scale harm, no different than fundamental research into global warming or drug-resistant bacteria, regardless of whether lay people can more easily envision the types of harm from those other threats.

monadgonad 8y ago
> Isn’t this just a statement that you disagree either with the relative probability of an AI-related disaster or the relative degree of harm it could cause? Yes, but the point of this movement is that they're supposed to be evidenced-based and maximising ROI. There is evidence for the existence and threat of global warming and drug-resistant bacterial. When you move away from charitable efforts whose effectiveness we can directly measure, you're not doing EA any more, just regular emotional/value-based charity. I do have different values and feelings about such non-evidence-based charity, and I think theirs are morally repugnant, especially when they claim to be doing EA.
- mlthoughts2018 8y ago
  That's a fair point about moving into areas in which the effectiveness is hard to measure. However, even apart from AI threats, I'd argue that there are many areas like this that pose potentially great harm. For example, I think that we ought to spend more effort to prevent regulatory capture in corporate-friendly legislation. But whether we can find a metric we'd all agree on to target this from an empirical point of view is a very difficult question. It reminds me of Gilb's Law, "Anything can be measured in a way that is superior to not measuring it at all." So if your sincere beliefs were such that the threat posed is high enough, and total harm would be astronomical, then you might believe that accepting a present-day metric that has a lot of variance and which is hotly debated might still be ok. Empirical progress along that rough metric might be more valuable, still in an expected value sense, than progress in other possibly less-harmful areas even if they have more clearly defined metrics. Again, not arguing for or against, just trying to represent why someone might be sincere about investing in AI safety and why, under their particular beliefs and preferences, it could still be rational from an EA point of view, despite more ambiguity. There are always risks that a certain model's loss function does not correctly correspond to the goals they wish to optimize towards, or that there is a large amount of idiosyncratic noise in the observation of the loss criteria. But you can still account for the risks of these sources of error in an overall method that is still empirical.
- savanaly 8y ago
  It's like calling someone morally repugnant because they agreed to give you money equal to the sum of two and two and then gave you three dollars, insisting that the math actually does come out that way. If they honestly do believe that, they are simply mistaken, not morally repugnant.

ogennadi 8y ago

I used to feel similarly until I realized that the EA movement isn't a hierarchical organization: it's just a bunch of totally separate orgs who have a common philosophy about how to do good in the world. Sure, OpenPhil funds AI research. But Givewell (who, last I checked, share the same office) has in their Top Charities list, only those that work in poor countries https://www.givewell.org/charities/top-charities https://www.givewell.org/charities/top-charities [2] https://www.givewell.org/charities/top-charities https://www.givewell.org/charities/top-charities

monadgonad 8y ago
Thank you; this has opened my eyes
jaggederest 8y ago
Agreed. I've loosely held an EA-like philosophy for about a decade and I think that OpenPhil orientation towards AI is pretty disappointing. I account for my time in terms of things like number of people saved from blindness or death due to malaria, and I definitely do not count future simulated persons as worthy of any of the same concern as actual humans who exist today.
- edanm 8y ago
  Out of curiosity: why?
  - jaggederest 8y ago
    Because they don't exist. It's like predicating your actions on the possible future existence of the easter bunny. Why are these people not attempting to research the ability to contact or create deities, or perpetual motion machines? Because they don't exist.
- danielbigham 8y ago
  I watched a debate involving William Macaskill last summer and he poses the hypothetical question: "You are outside a burning building and are told that inside one room is a child and inside another is a painting by Picasso. You can save one of them. To do the most good, which do you choose?" The point he's trying to illustrate is that, if you knew for certain that you could turn around and sell the Picaso for millions and use that money to purchase malaria bed nets, the expected number of lives saved by using the Picaso could be hundreds, and so there's a moral dilemma present. Like many hypothetical questions, this one feels a bit "off" or "unrealistic", but if you don't get hung up on the oddities, I think one can sense the essence of his question, and it reminded me of the point you're making here as well as a responder's question asking you why you think the way you do. I do think these questions are hard for us to wrap our heads around -- how to value high probability immediacy against somewhat uncertain non-proximal/non-immediate things that might be "much higher value". Part of my human brain goes splat when I try to weigh these things. In terms of the moral dilemma with the painting, I do have quite a bit of sympathy for the argument that one should do what they feel will produce the most good, which might be to save the painting and purchase malaria nets. My father on the other hand seemed to believe that to be absolutely morally wrong, which seems to be siding with your sentiments. Practically speaking, I think I'd almost certainly save the child's life, because one's human impulses would be so strong that they would override any high-and-lofty-rationality, and one wouldn't have time anyway to do deep analysis. But the question in a hypothetical sense does seem quite valid and hard.

skybrian 8y ago

Note that GiveWell is sticking to its original mission of evaluating charities based on scientific studies and isn't recommending AI funding. However I'll push back on this. If it's okay for the government to fund basic scientific research where the benefits are long-term and unpredictable, why not private individuals? Or are you saying research shouldn't be funded at all?

monadgonad 8y ago
I think it's fine for private individuals to do it (though I disagree with their choice), but not to claim that it's evidence-based Effective Altruism while doing so - it is an emotionally-led value judgment, just like donating to my local dog shelter would be (which I understand others may disgree with). That said, I do recognise now that effectivealtruism.org does not represent the entirety of the movement; I have no fault with what GiveWell are doing.
- tlb 8y ago
  Of course, there'll never be evidence about the effectiveness of research into existential risks until it's too late. So there's no way to objectively allocate funding between, say, AI safety and disease pandemics. While I think it's important to fund such research, it's a good point that funding orgs should either be evidence-based or not. Funding some things based on evidence and other things based on hunches might be an unstable combination. Once hunches are allowed, they could easily drift into funding things where the evidence says it's ineffective, but it gets overruled.
  - mkirklions 8y ago
    You could probably calculate this. Since AI can kill 7,000,000,000 people, You could probably toss a non-zero probability on it. Any math people know how to model this? Even with a bunch of different variations for the probability, you could understand why AI is so dangerous. potentially. There are probably lots of non zero probability events like getting hit by a rogue quasar. The difference is we cannot stop the quasar and that humans are required to discover AI. I dont give money to other charities, I spend money on mine. We dont have any employees, just volunteers. I spent a total of 50$ on the LLC paperwork and 8$/mo for a website host. Everything else has went directly to the kids/families we are teaching.

justinpombrio 8y ago

> However, to consider artificial intelligence to be a potential global catastrophe at all, let alone the single one requiring extra funding, is mostly unfounded. As another commenter points out, EA is not a single unified group, but rather a disparate set of people who are united by the belief that we should put money where it has the largest effect. Some, but not all, of them are convinced by the arguments in favor of doing AI alignment research. If you aren't, then don't donate money for that, donate it to what you think is a more effective cause! I'm curious, though, how people come to the conclusion that AI risk is nonsense? Is it a gut reaction, or does it come from thinking or reading about the problem? Every popsci article I've seen on the topic has been atrocious, so kudos if you thought "nonsense!" after reading such an article. Scott Alexander has a pretty readable introduction to AI risk: http://slatestarcodex.com/superintelligence-faq/ http://slatestarcodex.com/superintelligence-faq/

jerf 8y ago
AI risk has a profile that is very, very difficult to deal with rationally. By that I don't just mean "think about rationally rather than emotionally", but that it is difficult to even use rational tools to analyze it. It is what most people would consider a very, very small probability of what is arguably the worst possible outcome (i.e., considered from a strictly materialist perspectiv, there are outcomes worse that "the total extinction of life on Earth"!). Tiny magnitude probabilities of huge magnitude disasters are mathematically unstable; very small shifts in the value of the probability, and relatively small changes in the log value of the disaster's size, result in radically different scenarios and correct risk mitigations. It's even worse if you try to draw the probability distribution of something like "given strong AI will be acheived within the next 100 years, what is the distribution of the resulting likely outcomes?" In this case, the uncertainties are so large that, again, people can come to radically different conclusions even assuming both sides are being generally rational. It asks people trying to draw that distribution to consider what the most likely outcome is of at least a radically super-intelligent being is, and more likely, an arbitrarily large community of radically super-intelligent beings. Who can seriously claim to have an accurate probability distribution of that? If we understood radically super-intelligent beings, we'd already be radically super-intelligent beings, so we are not good at modeling them almost by definition. But certainly by observation; I am opaque in many ways even to my 7 and 10 year old children, and as with most humans, they are already exceptionally intelligent as "intelligent systems" go. (A 5 year old of normal intelligence is already exceptionally intelligent as intelligent systems go.) The idea that I could model something that was even my clone otherwise but operating at a hundred times the speed is already absurd; add an actual increase in intelligence to it and I stand no chance. Our only commonality will be those things imposed on us by the universe; i.e., I can be confident it must consume some negentropy to survive, etc. But any higher-level actions would be impossible for me to model. People who think intelligence is unlike to emerge quickly are unlikely to consider it a serious risk. People who think it's unlikely for an intelligent being to augment itself, then use its augmentations to augment itself, in an explosion of uncontrollable intelligence, will not consider AI risk a very likely outcome. And it's not as if it's an irrational or impossible outcome; for all we know, while there may be low-hanging fruit above human intelligence it is absolutely entirely possible that O(work to increase intelligence) > O(ability to perform intelligence-increasing work as intelligence goes up). After all, I can't help but look up in the sky and observe that there does not seem to be a near-lightspeed expanding bubble of computronium coming my way, per the most dramatic fears of the singularity. We could end up with something very intelligent, even dangerously intelligent, but not end up with us waking up one day to a digital god among us.

patientplatypus 8y ago

To provide the counter argument I imagine the Effective Altruism folks may give (not that I necessarily support this position): A number of specialist AIs could potentially eliminate 50%+ of all jobs in the next 50 years (a generation). What does society look like when we don't have the safety nets to support a massive unemployed population? What could it mean for geo-politics if the rich countries population is simply rich because they have the most advanced algorithms, but most people don't work? We're talking about massive social unrest worldwide potentially - capitalism cannot necessarily survive this paradigm shift and since the fall of the Berlin Wall the world has no other social organization in modern times, unless we'd like to revert to totalitarian rule. This all presumes of course, that the first AGI they turn on doesn't just turn the universe into a pile of paperclips (ducks)

joe_the_user 8y ago

It's one thing to walk past homeless people in my city and not give them money, because I know that money could much more easily and effectively safe a life in malaria-ridden parts of the world. The homeless in America live a life that's close to as miserable as the poor anywhere. One of the problem of most charities is bureaucracy eating up funds. Despite a lot of claims by bureaucracy, people who are homeless can often spend the money better on themselves than a bureaucracy. I don't think there's any particularly good not to give to the homeless if you feel like it and have the money.

kbenson 8y ago
Not to defend not giving money to the homeless, but isn't the calculation more complex than that? It's not just about how much money the bureaucracy eats, it's also how effectively the remaining portion is used, and I would think that would include actual resources delivered to people. More simply put, which helps more people and in a larger amount, $10 in the US or $5 in some extremely poor part of the world, where food and services may be much cheaper? I don't know the answer, but I think it's not as simple as "half is taken by bureaucracy so I shouldn't give to a bureaucracy".
- joe_the_user 8y ago
  Not to defend not giving money to the homeless, but isn't the calculation more complex than that? It's not just about how much money the bureaucracy eats, it's also how effectively the remaining portion is used, and I would think that would include actual resources delivered to people. Certainly, altogether the calculations are extremely complicated. There's whether a given organization knows what resources to provide, whether the provided resources are going to be used even if they are otherwise "right", etc. With all the complexity and uncertainty, directly giving to people seems like one entirely legitimate approach since it guarantees that people get resources, not that other necessary are problematic but certainly other approaches deserve certainty. Direct giving at least has "what you see is what you get".
- jimmaswell 8y ago
  >Not to defend not giving money to the homeless Then I'll defend it. The money would be much better utilized donated to programs that try to target homelessness as a whole. The old "they'll just spend it on drugs/alcohol/etc" cliche is actually backed by studies: https://www.theatlantic.com/business/archive/2011/03/should-you-give-money-to-homeless-people/72820/ https://www.theatlantic.com/business/archive/2011/03/should-... Some points from the article: "We choose to donate money based on the level of perceived need. Beggars known this, so there is an incentive on their part to exaggerate their need, by either lying about their circumstances or letting their appearance visibly deteriorate rather than seek help." "If you travel to a poor city, for example, you'll find swarms of beggars by touristy locations. If the tourists become more generous, the local beggars don't get richer, they only multiply." Further, a controversial issue is that of fake homeless people, which undoubtedly exist. Some say the easiest way to tell is if they'll accept food instead of money. From what From what I remember of someone's experience relayed in a reddit thread (which I'll admit has the potential to be exaggerated or unreliable) the majority of homeless people who said they needed "money for food" wouldn't accept food or would throw it away as soon as they thought the giver wasn't looking, indicating they were actually just after money. If you feel like you absolutely must give directly to homeless people, carry around wrapped protein bars or something similar. Encouraging panhandling is just exacerbating the problems. For some anecdotes, the last time I went to a convention in Baltimore, I heard stories about multiple people being assaulted or verbally harassed by panhandlers. In one case the provocation was that the person "didn't give them enough." The area is full of career-beggars who have no interest in actually improving themselves, like this one guy who pretends to be a youth baseball coach to collect fake donations every year. I try to stay indoors as much as possible but they even come inside the hotel lobbies to try to scam people sometimes. My experience and hearing those assault stories made me wonder how many of the people on this site decrying homeless-deterrent architecture have actually been in an affected area before.

DoreenMichele 8y ago

Re homelessness in the US, someone recently shared this with me and it looks promising, though it seems to only be in Seattle at the moment: https://www.samaritan.city/ https://www.samaritan.city/

emodendroket 8y ago
I don't know that I really need an app to give beggars money or buy the Spare Change News from someone.
- DoreenMichele 8y ago
  You absolutely don't need an app to give homeless individuals money. But when I was homeless, I found that it was vastly more helpful for me if someone gave me a few bucks than if I had to stand in line for hours, fill out reams of paper, etc to try to get some assistance. I am happy to encourage any model that fosters a little more direct giving of this sort. I am also looking to create a pilot program to try to help homeless individuals begin to develop an online income from the street as I did. Charity helps you survive another day, but you need an income of your own to get your life back and many programs seem to actively be against the idea of homeless people trying to work for a living. People seem to see it as cruel to expect them to. But earned income is the only real path out of dire poverty.
  - emodendroket 8y ago
    Fair enough. I also find it unfortunate that so many people seem to see themselves as freelance interrogators of the homeless asking about what they will do with the money they're given.
    - DoreenMichele 8y ago
      Yes, thank you for saying that. The problem runs deeper than you may realize. Even very well meaning people would walk up to me and ask me "Are you homeless? What's your name?" They didn't volunteer their name, address or socioeconomic status when asking such questions. They were trying to ascertain if I would need help. A much better question format that isn't so problematic would be along the lines of "I have X (clothes, blankets, whatever). Would you like to have that?" If someone isn't actually homeless but would be happy to have a free blanket, what do you care? Maybe that blanket will help prevent them from becoming homeless.

notahacker 8y ago

Even as a massive sceptic of Cyberpunk AI (and for that matter, the efficacy of preventing it by setting up charities to do AI research), I think people are perfectly entitled to spend their money on it if it's what they're interested in. I also think it's plausible some such programmes might create extremely commercially valuable byproducts even though they don't lead anywhere useful directly, much like the Apollo missions. Trouble is, if you invest enormous amounts of time and energy in arguing that a general belief stuff might work isn't good enough for philanthropy and even writing "why we don't recommend" articles about specific charities dispensing aid that don't deprive enough people of it to have a control group for RCTs, you deserve every criticism you get when you then funnel foundation money to very well-funded AI startups with unfalsifiable solutions to a purely hypothetical problem, even if you didn't have close personal relationships with people working for them. Even though I consider some EA analysis to be good and well reasoned (and they're certainly not the only people making evidence based bearish arguments about, say, microfinance) it's a little difficult to see how AI research could possibly pass a decision-making heuristic so supposedly rigorous that it writes off sanitation as an area to invest in because the estimations of diarrhea reduction aren't blinded. Ultimately, EAs have their cognitive biases just like anyone else, and I'm more sympathetic than they are to aid organizations' view that conducting RCTs is difficult and expensive and they don't want people suffering in control groups to quantify how well common sense healthcare interventions work, and disagree that such organizations should be held to higher standards of analytical rigour than AI researchers hoping to be Sarah Connor.

throwawayjava 8y ago

> I think it's absolutely morally repugnant to walk past them and not give them money, because instead you're paying people to sit around thinking about AI. This is not a new argument. Sister Mary Jucunda, a Zambian nun, wrote to a NASA scientist (Ernst Stuhlinger) in the early 70's with a similar critique of funding for space research. You can read Ernst's cogent reply here: http://www.lettersofnote.com/2012/08/why-explore-space.html http://www.lettersofnote.com/2012/08/why-explore-space.html My favorite excerpt follows: > About 400 years ago, there lived a count in a small town in Germany. He was one of the benign counts, and he gave a large part of his income to the poor in his town. This was much appreciated, because poverty was abundant during medieval times, and there were epidemics of the plague which ravaged the country frequently. One day, the count met a strange man. He had a workbench and little laboratory in his house, and he labored hard during the daytime so that he could afford a few hours every evening to work in his laboratory. He ground small lenses from pieces of glass; he mounted the lenses in tubes, and he used these gadgets to look at very small objects. The count was particularly fascinated by the tiny creatures that could be observed with the strong magnification, and which he had never seen before. He invited the man to move with his laboratory to the castle, to become a member of the count's household, and to devote henceforth all his time to the development and perfection of his optical gadgets as a special employee of the count. > The townspeople, however, became angry when they realized that the count was wasting his money, as they thought, on a stunt without purpose. "We are suffering from this plague," they said, "while he is paying that man for a useless hobby!" But the count remained firm. "I give you as much as I can afford," he said, "but I will also support this man and his work, because I know that someday something will come out of it!" > Indeed, something very good came out of this work, and also out of similar work done by others at other places: the microscope. It is well known that the microscope has contributed more than any other invention to the progress of medicine, and that the elimination of the plague and many other contagious diseases from most parts of the world is largely a result of studies which the microscope made possible. I think there are many parallels between AI/ML and the microscope, and I think safety research is a very reasonable inquisitive lens for developing these new technologies.

jopsen 8y ago
Well, fighting poverty also grows the economy and number of brains available for research in the future. Today we are bringing people out of poverty for good, much charity isn't money in a bottomless pit. The pit has a bottom as evident by the very realistic goal to end extreme poverty by 2030. Still basic research is important, because it has such a long tail. Just saying both approaches have validity and will bring massive progress.

jlebar 8y ago

However, to consider artificial intelligence to be a potential global catastrophe at all, let alone the single one requiring extra funding, is mostly unfounded. We currently can barely even define what the actual risk is, let alone how to mitigate it. Although I happen agree with you personally, I don't think we should commit the fallacy of assuming that because their position seems absurd to us that it comes from a place of bias or ignorance. To the contrary, when I listen to Holden and other EA leaders talk, it's clear they've spent way more time thinking about this stuff than I have. He is and they are thoughtful and humble about exactly the questions you pose: How much should we weight "known good" good done today vs "potential good" done in the future, how confident should we be in our ability to predict the future, etc. As one example, Open Phil has hired historians to conduct research on how well people have (in the past) been able to predict the future, precisely to inform their thinking in this regard. Holden is also open about how his thoughts on the importance of AI alignment have changed over time. Again, we can disagree with him (as I do). But we absolutely should not claim that because we disagree, his position must be out of a place of thoughtlessness or bias.

jlebar 8y ago
See e.g. https://80000hours.org/podcast/episodes/holden-karnofsky-open-philanthropy/ https://80000hours.org/podcast/episodes/holden-karnofsky-ope...
monktastic1 8y ago
It's perfectly reasonable for them to hold that view. The unfortunate thing is to insist that it's the most rational and/or correct view. To say that they're biased isn't an insult. We all have biases, and it's useful to recognize and admit them.
- pas 8y ago
  Could you link/cite where they insist that it's the Right Thing?
  - jonny_eh 8y ago

morgante 8y ago

Effective Altruism is not a single monolithic organization and I think you really go too far to assume all/most advocates think AI research is more important than feeding the homeless. Personally, I wholeheartedly subscribe to effective altruism and also don't think AI research is a good use of my funds. All of my donations go to malaria relief. Effective altruism is about principles, not particular causes.

tomcam 8y ago

The painful truth is that it is morally repugnant to donate. I hate saying this, by the way. If you had any experience with outreach, You would understand it is actually quite destructive to give them money, and it is much better to donate to, for example, Union Gospel Mission, a place in Seattle that does excellent homeless outreach. The city in fact contracts out to them for their services.

Viliam1234 8y ago
> The painful truth is that it is morally repugnant to donate. I hate saying this, by the way. I hate that you said it without explaining what you meant. :P Seriously, when you say "it is morally repugnant to donate" and "it is much better to donate to X" in the same paragraph, I am not sure what to think about it. Do you think that donating to X is still morally repugnant, although a bit less than donating to other causes? Or that donating can either be wrong or right depending on who you donate to (i.e. how the money is going to be used)? Because the latter seems like something that Effective Altruists would totally agree with.
- tomcam 8y ago
  Sorry, my comment was quite unclear. I meant it is dangerous to street people to give the money directly. It is much better to give it to you in the Stabley Schomann organization that knows how to deal with their problems.

T2_t2 8y ago

I think your reply is interesting, as it exactly the sort of thing that Effective Altruism talks about. The example you gave is endemic of the way we think of charity precisely because it deals with both emotion and things we can individually see. Effective Altruism aims to make charity about things that don't directly affect us, that we can't see, and find areas that where the "bang per buck" is high. Seattle spent $195 million on the homelessness according to this: https://www.seattletimes.com/seattle-news/homeless/how-much-do-seattle-and-king-county-spend-on-homelessness/ https://www.seattletimes.com/seattle-news/homeless/how-much-..., to help a group that this advocacy organisation http://www.homelessinfo.org/what_we_do/one_night_count/ http://www.homelessinfo.org/what_we_do/one_night_count/ puts at 10,000 people. I wonder how a person can rationally look at that - $195 million at ~$19.5K per person and think they can make a dent. Compare that to https://www.givedirectly.org/basic-income https://www.givedirectly.org/basic-income and doubling the poorest of the poor's daily income by giving them $1 a day for 12 years. These are people I can't see and don't have to think about, yet for less than $5000 I can provide someone with security and an income for over a decade. If I give $70 a week for 12 years, I can take 10 people out of abject poverty for 12 years. That's fractionally above my coffee budget. That's the question Effective Altruism wants people to ask themselves. How can I, with the amount of money I have, do the most tangible good in the world, and what is an equation that helps me decide. Far from being "absolutely morally repugnant" to walk past a homeless person and preference things we can't see, I think it is morally honest to look at the risks humanity faces, and preference things that are under funded, where $1 can go far. That takes several forms, both immediate - like the Give Directly example - versus long term, where risks that could wipe out our species are considered. I think a strong argument can be made that, for minimal expense, it would be possible to have an affect on a potentially species destroying technology like AI, and that the money spent there is more likely to do good than adding that to an already well funded area. That's catastrophic risk, minimal expense vs human suffering and high cost. How you solve for the equilibrium there.