3 ms·

Isn’t this just a statement that you disagree either with the relative probability of an AI-related disaster or the relative degree of harm it could cause? It

by mlthoughts2018 8y ago

Isn’t this just a statement that you disagree either with the relative probability of an AI-related disaster or the relative degree of harm it could cause? It seems harsh to say it’s “morally repugnant” when in the end you’re just saying the way you assign probabilities and degrees of negative outcomes would lead you to invest in a different portfolio of charities or efforts than what this other group would do. (I’m not arguing for or against the correctness of a belief that AI poses attention- and funding-worthy threats. Only that these other people motivated to allocate resources to it have studied the problem in great detail and their sincere belief after looking into it is that it is worth being part of the overall portfolio of charity investment, and this really would (in their sincere judgment) mitigate big-scale harm, no different than fundamental research into global warming or drug-resistant bacteria, regardless of whether lay people can more easily envision the types of harm from those other threats.

monadgonad 8y ago
> Isn’t this just a statement that you disagree either with the relative probability of an AI-related disaster or the relative degree of harm it could cause? Yes, but the point of this movement is that they're supposed to be evidenced-based and maximising ROI. There is evidence for the existence and threat of global warming and drug-resistant bacterial. When you move away from charitable efforts whose effectiveness we can directly measure, you're not doing EA any more, just regular emotional/value-based charity. I do have different values and feelings about such non-evidence-based charity, and I think theirs are morally repugnant, especially when they claim to be doing EA.
- mlthoughts2018 8y ago
  That's a fair point about moving into areas in which the effectiveness is hard to measure. However, even apart from AI threats, I'd argue that there are many areas like this that pose potentially great harm. For example, I think that we ought to spend more effort to prevent regulatory capture in corporate-friendly legislation. But whether we can find a metric we'd all agree on to target this from an empirical point of view is a very difficult question. It reminds me of Gilb's Law, "Anything can be measured in a way that is superior to not measuring it at all." So if your sincere beliefs were such that the threat posed is high enough, and total harm would be astronomical, then you might believe that accepting a present-day metric that has a lot of variance and which is hotly debated might still be ok. Empirical progress along that rough metric might be more valuable, still in an expected value sense, than progress in other possibly less-harmful areas even if they have more clearly defined metrics. Again, not arguing for or against, just trying to represent why someone might be sincere about investing in AI safety and why, under their particular beliefs and preferences, it could still be rational from an EA point of view, despite more ambiguity. There are always risks that a certain model's loss function does not correctly correspond to the goals they wish to optimize towards, or that there is a large amount of idiosyncratic noise in the observation of the loss criteria. But you can still account for the risks of these sources of error in an overall method that is still empirical.
- savanaly 8y ago
  It's like calling someone morally repugnant because they agreed to give you money equal to the sum of two and two and then gave you three dollars, insisting that the math actually does come out that way. If they honestly do believe that, they are simply mistaken, not morally repugnant.
  - monadgonad 8y ago
    It's not being "simply mistaken", it's a calculated belief. Someone might think it's a moral act to fund, say, ecoterrorists to bomb the headquarters of a major oil company, believing that the loss of life is worth it for helping the world by destablising the company. We might disagree, and consider that highly immoral. That's an extreme example of course, and in this case the worst that can happen is some loss of money, rather than loss of lives, but I hope it illustrates my point: them choosing to pay people to think about AI, rather than giving that money to other causes, is a moral choice with which I disagree.
  - rspeer 8y ago
    The founder of Open Philanthropy claims to have sought out the most effective uses of charity money in the entire world, and it happens to include paying his roommate and brother-in-law to think about AI. The big problem with this decision isn't that it's mistaken, it's that it's corrupt. He should have excluded things that were in his self-interest from consideration, even if he honestly, mistakenly believes they are the best thing anyone can do with their money.
    - ipioxu15 8y ago
      Corruption, nepotism, and hypocrisy all in one --- beautiful, modern American values.
    - mlthoughts2018 8y ago
      This would be an argument about corruption or disingenuous claims by a specific person involved with the charity movement, and could definitely heighten skepticism about other involved parties. But it would not be a criticism of the general idea of risk-return optimization applied to charity, and would not exclude a rational participant from belief that allocating some capital towards AI research was part of an optimized approach. The original comment made it seem like the generic idea of choosing to invest in AI safety as part of the EA framework was, in spirit, intrinsically "morally repugnant." The specific repugnant actions of one party would not necessarily support that, anymore than say a pro football player's racist comments would imply that all of football is inherently morally repugnant. It would seem valuable to separate and distinguish vitriol directed at the specific suspicious actions of one person from generic and wide-sweeping criticism of an entire framework that person happens to be associated with, especially when the framework itself attempts to be value-neutral conditioned on one's beliefs after looking at some evidence.
stale2002 8y ago
IMO, the problem with AI risk funding isn't that it is 'wrong', per se. Instead it is that it is very speculative and not grounder in real world data, or measurable benefits that can be achieved TODAY. The whole point of effective altruism is to be very ground in evidence based causes. IE, if I spend 10K on this cause, then it will definitely save X lived by next year. AI risks can't be reduced to these kinds of numbers just yet. It could still be true, we just don't have any evidence yet to prove it though.
- mlthoughts2018 8y ago
  I would say they can be reduced to numbers like that today, it's just that the implied error bars around the numbers would be huge. Then in the optimization problem of determining what to invest donations into, the amount of gain from putting money into AI research would be correspondingly penalized by the amount of uncertainty (this is known as mean-variance optimization). It's very much like a choice to invest in a solid, fundamental company which you know will give you 5% return next quarter, or invest in a very uncertain start-up which might give you 1000% return next quarter or might give you -50%. The start-up investment might have a higher expected value, but also a much higher risk. At that point, the way to distinguish between whether you prefer to invest in the "sure bet" company or the "risky and ambiguous" start-up would be down to your personal tolerance for risk. It could be perfectly rational to invest in the start-up in that situation. If you personally have a high risk tolerance. If you don't, then the start-up would look like a crazy, speculative bet with huge downside. So some people might look at AI and say, "We don't have a good idea what the capabilities will be in ~50 years. So there is a huge risk that my charity donations will be wasted because in 50 years we realize we never needed to worry about this problem. Or we might use current AI research to thwart an unimaginable global crisis." That person then looks into various sources of thought on putting actual numbers and actual error bars, the best we can, onto the problem, say by reading the Global Catastrophic Risk book, or stuff by Bostrom, or consulting surveys of current practitioners' estimates. At the end of that process, it could totally end up being the rational course of action for that person to still invest in AI charities. Yes, their estimate for the expected value of the donation might have huge error bars around it, but under their particular beliefs about risk-return trade-offs, that might not cause their optimized decision to change.

monadgonad 8y ago

> Isn’t this just a statement that you disagree either with the relative probability of an AI-related disaster or the relative degree of harm it could cause? Yes, but the point of this movement is that they're supposed to be evidenced-based and maximising ROI. There is evidence for the existence and threat of global warming and drug-resistant bacterial. When you move away from charitable efforts whose effectiveness we can directly measure, you're not doing EA any more, just regular emotional/value-based charity. I do have different values and feelings about such non-evidence-based charity, and I think theirs are morally repugnant, especially when they claim to be doing EA.

mlthoughts2018 8y ago
That's a fair point about moving into areas in which the effectiveness is hard to measure. However, even apart from AI threats, I'd argue that there are many areas like this that pose potentially great harm. For example, I think that we ought to spend more effort to prevent regulatory capture in corporate-friendly legislation. But whether we can find a metric we'd all agree on to target this from an empirical point of view is a very difficult question. It reminds me of Gilb's Law, "Anything can be measured in a way that is superior to not measuring it at all." So if your sincere beliefs were such that the threat posed is high enough, and total harm would be astronomical, then you might believe that accepting a present-day metric that has a lot of variance and which is hotly debated might still be ok. Empirical progress along that rough metric might be more valuable, still in an expected value sense, than progress in other possibly less-harmful areas even if they have more clearly defined metrics. Again, not arguing for or against, just trying to represent why someone might be sincere about investing in AI safety and why, under their particular beliefs and preferences, it could still be rational from an EA point of view, despite more ambiguity. There are always risks that a certain model's loss function does not correctly correspond to the goals they wish to optimize towards, or that there is a large amount of idiosyncratic noise in the observation of the loss criteria. But you can still account for the risks of these sources of error in an overall method that is still empirical.
savanaly 8y ago
It's like calling someone morally repugnant because they agreed to give you money equal to the sum of two and two and then gave you three dollars, insisting that the math actually does come out that way. If they honestly do believe that, they are simply mistaken, not morally repugnant.
- monadgonad 8y ago
  It's not being "simply mistaken", it's a calculated belief. Someone might think it's a moral act to fund, say, ecoterrorists to bomb the headquarters of a major oil company, believing that the loss of life is worth it for helping the world by destablising the company. We might disagree, and consider that highly immoral. That's an extreme example of course, and in this case the worst that can happen is some loss of money, rather than loss of lives, but I hope it illustrates my point: them choosing to pay people to think about AI, rather than giving that money to other causes, is a moral choice with which I disagree.
- rspeer 8y ago
  The founder of Open Philanthropy claims to have sought out the most effective uses of charity money in the entire world, and it happens to include paying his roommate and brother-in-law to think about AI. The big problem with this decision isn't that it's mistaken, it's that it's corrupt. He should have excluded things that were in his self-interest from consideration, even if he honestly, mistakenly believes they are the best thing anyone can do with their money.
  - ipioxu15 8y ago
    Corruption, nepotism, and hypocrisy all in one --- beautiful, modern American values.
  - mlthoughts2018 8y ago
    This would be an argument about corruption or disingenuous claims by a specific person involved with the charity movement, and could definitely heighten skepticism about other involved parties. But it would not be a criticism of the general idea of risk-return optimization applied to charity, and would not exclude a rational participant from belief that allocating some capital towards AI research was part of an optimized approach. The original comment made it seem like the generic idea of choosing to invest in AI safety as part of the EA framework was, in spirit, intrinsically "morally repugnant." The specific repugnant actions of one party would not necessarily support that, anymore than say a pro football player's racist comments would imply that all of football is inherently morally repugnant. It would seem valuable to separate and distinguish vitriol directed at the specific suspicious actions of one person from generic and wide-sweeping criticism of an entire framework that person happens to be associated with, especially when the framework itself attempts to be value-neutral conditioned on one's beliefs after looking at some evidence.

stale2002 8y ago

IMO, the problem with AI risk funding isn't that it is 'wrong', per se. Instead it is that it is very speculative and not grounder in real world data, or measurable benefits that can be achieved TODAY. The whole point of effective altruism is to be very ground in evidence based causes. IE, if I spend 10K on this cause, then it will definitely save X lived by next year. AI risks can't be reduced to these kinds of numbers just yet. It could still be true, we just don't have any evidence yet to prove it though.

mlthoughts2018 8y ago
I would say they can be reduced to numbers like that today, it's just that the implied error bars around the numbers would be huge. Then in the optimization problem of determining what to invest donations into, the amount of gain from putting money into AI research would be correspondingly penalized by the amount of uncertainty (this is known as mean-variance optimization). It's very much like a choice to invest in a solid, fundamental company which you know will give you 5% return next quarter, or invest in a very uncertain start-up which might give you 1000% return next quarter or might give you -50%. The start-up investment might have a higher expected value, but also a much higher risk. At that point, the way to distinguish between whether you prefer to invest in the "sure bet" company or the "risky and ambiguous" start-up would be down to your personal tolerance for risk. It could be perfectly rational to invest in the start-up in that situation. If you personally have a high risk tolerance. If you don't, then the start-up would look like a crazy, speculative bet with huge downside. So some people might look at AI and say, "We don't have a good idea what the capabilities will be in ~50 years. So there is a huge risk that my charity donations will be wasted because in 50 years we realize we never needed to worry about this problem. Or we might use current AI research to thwart an unimaginable global crisis." That person then looks into various sources of thought on putting actual numbers and actual error bars, the best we can, onto the problem, say by reading the Global Catastrophic Risk book, or stuff by Bostrom, or consulting surveys of current practitioners' estimates. At the end of that process, it could totally end up being the rational course of action for that person to still invest in AI charities. Yes, their estimate for the expected value of the donation might have huge error bars around it, but under their particular beliefs about risk-return trade-offs, that might not cause their optimized decision to change.