4 ms·

> Isn’t this just a statement that you disagree either with the relative probability of an AI-related disaster or the relative degree of harm it could cause? Y

> Isn’t this just a statement that you disagree either with the relative probability of an AI-related disaster or the relative degree of harm it could cause? Yes, but the point of this movement is that they're supposed to be evidenced-based and maximising ROI. There is evidence for the existence and threat of global warming and drug-resistant bacterial. When you move away from charitable efforts whose effectiveness we can directly measure, you're not doing EA any more, just regular emotional/value-based charity. I do have different values and feelings about such non-evidence-based charity, and I think theirs are morally repugnant, especially when they claim to be doing EA.

mlthoughts2018 8y ago
That's a fair point about moving into areas in which the effectiveness is hard to measure. However, even apart from AI threats, I'd argue that there are many areas like this that pose potentially great harm. For example, I think that we ought to spend more effort to prevent regulatory capture in corporate-friendly legislation. But whether we can find a metric we'd all agree on to target this from an empirical point of view is a very difficult question. It reminds me of Gilb's Law, "Anything can be measured in a way that is superior to not measuring it at all." So if your sincere beliefs were such that the threat posed is high enough, and total harm would be astronomical, then you might believe that accepting a present-day metric that has a lot of variance and which is hotly debated might still be ok. Empirical progress along that rough metric might be more valuable, still in an expected value sense, than progress in other possibly less-harmful areas even if they have more clearly defined metrics. Again, not arguing for or against, just trying to represent why someone might be sincere about investing in AI safety and why, under their particular beliefs and preferences, it could still be rational from an EA point of view, despite more ambiguity. There are always risks that a certain model's loss function does not correctly correspond to the goals they wish to optimize towards, or that there is a large amount of idiosyncratic noise in the observation of the loss criteria. But you can still account for the risks of these sources of error in an overall method that is still empirical.
savanaly 8y ago
It's like calling someone morally repugnant because they agreed to give you money equal to the sum of two and two and then gave you three dollars, insisting that the math actually does come out that way. If they honestly do believe that, they are simply mistaken, not morally repugnant.
- monadgonad 8y ago
  It's not being "simply mistaken", it's a calculated belief. Someone might think it's a moral act to fund, say, ecoterrorists to bomb the headquarters of a major oil company, believing that the loss of life is worth it for helping the world by destablising the company. We might disagree, and consider that highly immoral. That's an extreme example of course, and in this case the worst that can happen is some loss of money, rather than loss of lives, but I hope it illustrates my point: them choosing to pay people to think about AI, rather than giving that money to other causes, is a moral choice with which I disagree.
- rspeer 8y ago
  The founder of Open Philanthropy claims to have sought out the most effective uses of charity money in the entire world, and it happens to include paying his roommate and brother-in-law to think about AI. The big problem with this decision isn't that it's mistaken, it's that it's corrupt. He should have excluded things that were in his self-interest from consideration, even if he honestly, mistakenly believes they are the best thing anyone can do with their money.
  - ipioxu15 8y ago
    Corruption, nepotism, and hypocrisy all in one --- beautiful, modern American values.
  - mlthoughts2018 8y ago
    This would be an argument about corruption or disingenuous claims by a specific person involved with the charity movement, and could definitely heighten skepticism about other involved parties. But it would not be a criticism of the general idea of risk-return optimization applied to charity, and would not exclude a rational participant from belief that allocating some capital towards AI research was part of an optimized approach. The original comment made it seem like the generic idea of choosing to invest in AI safety as part of the EA framework was, in spirit, intrinsically "morally repugnant." The specific repugnant actions of one party would not necessarily support that, anymore than say a pro football player's racist comments would imply that all of football is inherently morally repugnant. It would seem valuable to separate and distinguish vitriol directed at the specific suspicious actions of one person from generic and wide-sweeping criticism of an entire framework that person happens to be associated with, especially when the framework itself attempts to be value-neutral conditioned on one's beliefs after looking at some evidence.

mlthoughts2018 8y ago

That's a fair point about moving into areas in which the effectiveness is hard to measure. However, even apart from AI threats, I'd argue that there are many areas like this that pose potentially great harm. For example, I think that we ought to spend more effort to prevent regulatory capture in corporate-friendly legislation. But whether we can find a metric we'd all agree on to target this from an empirical point of view is a very difficult question. It reminds me of Gilb's Law, "Anything can be measured in a way that is superior to not measuring it at all." So if your sincere beliefs were such that the threat posed is high enough, and total harm would be astronomical, then you might believe that accepting a present-day metric that has a lot of variance and which is hotly debated might still be ok. Empirical progress along that rough metric might be more valuable, still in an expected value sense, than progress in other possibly less-harmful areas even if they have more clearly defined metrics. Again, not arguing for or against, just trying to represent why someone might be sincere about investing in AI safety and why, under their particular beliefs and preferences, it could still be rational from an EA point of view, despite more ambiguity. There are always risks that a certain model's loss function does not correctly correspond to the goals they wish to optimize towards, or that there is a large amount of idiosyncratic noise in the observation of the loss criteria. But you can still account for the risks of these sources of error in an overall method that is still empirical.

savanaly 8y ago

It's like calling someone morally repugnant because they agreed to give you money equal to the sum of two and two and then gave you three dollars, insisting that the math actually does come out that way. If they honestly do believe that, they are simply mistaken, not morally repugnant.

monadgonad 8y ago
It's not being "simply mistaken", it's a calculated belief. Someone might think it's a moral act to fund, say, ecoterrorists to bomb the headquarters of a major oil company, believing that the loss of life is worth it for helping the world by destablising the company. We might disagree, and consider that highly immoral. That's an extreme example of course, and in this case the worst that can happen is some loss of money, rather than loss of lives, but I hope it illustrates my point: them choosing to pay people to think about AI, rather than giving that money to other causes, is a moral choice with which I disagree.
rspeer 8y ago
The founder of Open Philanthropy claims to have sought out the most effective uses of charity money in the entire world, and it happens to include paying his roommate and brother-in-law to think about AI. The big problem with this decision isn't that it's mistaken, it's that it's corrupt. He should have excluded things that were in his self-interest from consideration, even if he honestly, mistakenly believes they are the best thing anyone can do with their money.
- ipioxu15 8y ago
  Corruption, nepotism, and hypocrisy all in one --- beautiful, modern American values.
- mlthoughts2018 8y ago
  This would be an argument about corruption or disingenuous claims by a specific person involved with the charity movement, and could definitely heighten skepticism about other involved parties. But it would not be a criticism of the general idea of risk-return optimization applied to charity, and would not exclude a rational participant from belief that allocating some capital towards AI research was part of an optimized approach. The original comment made it seem like the generic idea of choosing to invest in AI safety as part of the EA framework was, in spirit, intrinsically "morally repugnant." The specific repugnant actions of one party would not necessarily support that, anymore than say a pro football player's racist comments would imply that all of football is inherently morally repugnant. It would seem valuable to separate and distinguish vitriol directed at the specific suspicious actions of one person from generic and wide-sweeping criticism of an entire framework that person happens to be associated with, especially when the framework itself attempts to be value-neutral conditioned on one's beliefs after looking at some evidence.