4 ms·

Firstly, something can be more fun to worry about (for some people), sound like speculative sci-fi (to some people) and nevertheless still be a real problem. Co

by dotsam 4y ago

Firstly, something can be more fun to worry about (for some people), sound like speculative sci-fi (to some people) and nevertheless still be a real problem. Consider the dawn of the nuclear age: > On the morning of September 12, 1933, on a miserable, wet, quintessentially English autumn day, at the intersection where Russell Square meets Southampton Row, Leó Szilárd waited irritably at a traffic light waiting for it to change from red to green. He had just attended a lecture by the great English physicist Ernest Rutherford. Rutherford, known to many as the father of nuclear physics, was discussing the newly prophesied release of energy from atoms, most notably by science-fiction pioneer H G Wells in his book The World Set Free. In his baritone voice, Rutherford, acknowledged master of the atomic domain, dismissed this fanciful idea as nonsense. Any thought of releasing the energy locked in atoms, he said, was "moonshine". ... > What Szilard realised as he stepped off that curb was that if we found an element that when bombarded by one neutron would release two neutrons, it could lead to a chain reaction that could possibly release vast amounts of energy https://blogs.scientificamerican.com/the-curious-wavefunction/leo-szilard-a-traffic-light-and-a-slice-of-nuclear-history/ https://blogs.scientificamerican.com/the-curious-wavefunctio... Secondly, you can have a diverse portfolio of causes you care about, and Effective Altruism has recommendations for areas including global health and animal welfare. Taking AI risk seriously doesn't mean you can't give time and money to other causes with more tangible results.

notahacker 4y ago
> Taking AI risk seriously doesn't mean you can't give time and money to other causes with more tangible results. But if you've spent the past decade arguing precisely the opposite - that instead of spreading philanthropic dollars over broad portfolios because true efficacy is unmeasurable, they should be directed to causes which provide hard evidence that they're among the most efficient use of a marginal dollar, and that charities that aren't conducting RCTs to prove how much better off people are after being fed aren't really evidence-based enough to recommend - it's going to raise eyebrows when you pivot to recommending people also give their money to opaque organisations with vague goals and zero evidence they can achieve them, whose only thing in common with the highly evidence-focused aid agencies is that their pitch is nerdier than the average nonprofit. I mean, there's nothing particularly wrong with "I give to causes because their pitch speaks to me" as a heuristic for philanthropy - it's how most altruists think - but it's not how EA pitched itself as being different from that and caring about efficacy and evidence. The Wells analogy is awkward for the AI safety crowd for another reason too. Wells' central thesis that an atomic super weapon would guarantee a peaceful, utopian world turned out to be horribly wrong. We got actually used bombs, arms races, proxy wars and very nearly used bombs instead, and that's without Nazi atomic scientists getting a heads up. To the extent his writings on the subject of atomic weapon safety had any effect at all on atomic weapons, they made them more imminent and more likely to be used. If marginal dollars spent on promoting the distribution of The World Set Free would have had any effect at all on the probability of armageddon, they would probably have increased it.
- dotsam 4y ago
  I do not think that a historical EA movement would have recommended spending marginal dollars distributing copies of The World Set Free, but of course we cannot know that for sure. Apparently the book is positive on the proliferation of the powerful new speculative technology, although I haven't read it. I would imagine EA would have aligned more with an anti-nuclear stance, which is certainly the attitude currently taken towards the possible breakthroughs of AI. On your first point, I can donate towards anti-malarial interventions and towards reducing AI risk: two very diverse causes, but both under the EA umbrella. Diversifying amongst recommended EA causes seems to me a good bet, especially given I expect some of them to require pivoting away from as more research is done. Perhaps we can summarise the last decade of EA into something like: "given the available information (largely based on RCTs, research and an understanding of moral philosophy), we expect that money spent here will do more good than money spent there. More information will change this expectation." We now have information that there is a potentially very grave risk confronting us from AI. For other cause areas we can use evidence like randomised controlled trials to inform our actions. In the case of AI risk, this type of evidence is unavailable, but there are nonetheless other types of information available to us that suggest it is worth taking seriously. This information takes the form of compelling arguments from credible people in the AI industry and the industry's achievements to date. Of course we'd like to have firmer footing, but the nature of this risk doesn't permit it. We can either ignore it, and potentially walk to our doom, or we can take it seriously and try to reduce the uncertainty and the risks. I agree with you that this raises eyebrows, but I don't think it's a gross departure from the last decade. I think it is wrong to say we have "zero evidence" that organisations committed to reducing AI risk can achieve their goals. I think it is more accurate to say it is highly uncertain that they will be able to achieve their goals. My view is that we should try to address AI risk because the stakes are high. I would have sided with Albert Einstein in trying to renounce nuclear weapons, even though we have "zero evidence" for humanity ever coming together to do such a thing in the past. Of course - nuclear weapons really do exist and AI is only a hypothetical threat at the moment. My point is that there was "zero evidence" that the world could come together to voluntarily control a technology, and yet people like Einstein tried to make it happen, and with good reason. Dismissing this is like taking Rutherford's view in the 1930s and saying there's very little point worrying about nuclear weapons because they're not possible, and it's even more pointless thinking about how we might control them. Finally - why might the pitch speak to nerds in particular? I think in part because it is based on Bayesian reasoning, which is both nerdy and incredibly effective in situations of uncertainty.

by dotsam 4y ago

notahacker 4y ago

> Taking AI risk seriously doesn't mean you can't give time and money to other causes with more tangible results. But if you've spent the past decade arguing precisely the opposite - that instead of spreading philanthropic dollars over broad portfolios because true efficacy is unmeasurable, they should be directed to causes which provide hard evidence that they're among the most efficient use of a marginal dollar, and that charities that aren't conducting RCTs to prove how much better off people are after being fed aren't really evidence-based enough to recommend - it's going to raise eyebrows when you pivot to recommending people also give their money to opaque organisations with vague goals and zero evidence they can achieve them, whose only thing in common with the highly evidence-focused aid agencies is that their pitch is nerdier than the average nonprofit. I mean, there's nothing particularly wrong with "I give to causes because their pitch speaks to me" as a heuristic for philanthropy - it's how most altruists think - but it's not how EA pitched itself as being different from that and caring about efficacy and evidence. The Wells analogy is awkward for the AI safety crowd for another reason too. Wells' central thesis that an atomic super weapon would guarantee a peaceful, utopian world turned out to be horribly wrong. We got actually used bombs, arms races, proxy wars and very nearly used bombs instead, and that's without Nazi atomic scientists getting a heads up. To the extent his writings on the subject of atomic weapon safety had any effect at all on atomic weapons, they made them more imminent and more likely to be used. If marginal dollars spent on promoting the distribution of The World Set Free would have had any effect at all on the probability of armageddon, they would probably have increased it.

dotsam 4y ago
I do not think that a historical EA movement would have recommended spending marginal dollars distributing copies of The World Set Free, but of course we cannot know that for sure. Apparently the book is positive on the proliferation of the powerful new speculative technology, although I haven't read it. I would imagine EA would have aligned more with an anti-nuclear stance, which is certainly the attitude currently taken towards the possible breakthroughs of AI. On your first point, I can donate towards anti-malarial interventions and towards reducing AI risk: two very diverse causes, but both under the EA umbrella. Diversifying amongst recommended EA causes seems to me a good bet, especially given I expect some of them to require pivoting away from as more research is done. Perhaps we can summarise the last decade of EA into something like: "given the available information (largely based on RCTs, research and an understanding of moral philosophy), we expect that money spent here will do more good than money spent there. More information will change this expectation." We now have information that there is a potentially very grave risk confronting us from AI. For other cause areas we can use evidence like randomised controlled trials to inform our actions. In the case of AI risk, this type of evidence is unavailable, but there are nonetheless other types of information available to us that suggest it is worth taking seriously. This information takes the form of compelling arguments from credible people in the AI industry and the industry's achievements to date. Of course we'd like to have firmer footing, but the nature of this risk doesn't permit it. We can either ignore it, and potentially walk to our doom, or we can take it seriously and try to reduce the uncertainty and the risks. I agree with you that this raises eyebrows, but I don't think it's a gross departure from the last decade. I think it is wrong to say we have "zero evidence" that organisations committed to reducing AI risk can achieve their goals. I think it is more accurate to say it is highly uncertain that they will be able to achieve their goals. My view is that we should try to address AI risk because the stakes are high. I would have sided with Albert Einstein in trying to renounce nuclear weapons, even though we have "zero evidence" for humanity ever coming together to do such a thing in the past. Of course - nuclear weapons really do exist and AI is only a hypothetical threat at the moment. My point is that there was "zero evidence" that the world could come together to voluntarily control a technology, and yet people like Einstein tried to make it happen, and with good reason. Dismissing this is like taking Rutherford's view in the 1930s and saying there's very little point worrying about nuclear weapons because they're not possible, and it's even more pointless thinking about how we might control them. Finally - why might the pitch speak to nerds in particular? I think in part because it is based on Bayesian reasoning, which is both nerdy and incredibly effective in situations of uncertainty.