10 ms·

I think there are more important causes than "reducing potential risks from advanced AI". Honest to god, $30M will go a long way in saving lives TODAY. Flint, M

I think there are more important causes than "reducing potential risks from advanced AI". Honest to god, $30M will go a long way in saving lives TODAY. Flint, MI anyone?

acgourley 10y ago
Do you disagree with any research looking to prevent existential risk to our species, or do you believe AI cannot become an existential risk?
- mankash666 10y ago
  I think existential risks to our species of much higher magnitude exist today. Climate change and pollution for instance. I think a "possible existential threat in the future" is a weaker case for philanthropy than existing ones.
  - curiousgal 10y ago
    Both of those issue cannot be solved by throwing money at them, especially the first. It requires political forces. Then again, the money could be used to buy politicians' votes in favor of environment-friendly policies. It's so blatant in the U.S. these days.
arikr 10y ago
It's about expected value. A 0.0001% chance of reducing an extreme risk will have a very high expected value.
- rspeer 10y ago
  You can justify lots of things with Pascal's wager, but most of them are religions.
davmre 10y ago
There are always going to be more important causes, under any particular person's view of "more important", which depends very strongly on both subjective values and (in the case of AI alignment work) on precise probabilities of far-future outcomes. A dollar given to the Against Malaria Foundation will do a lot more good, in QALY terms, than the same dollar spent in Flint. And both dollars will do more (direct) good than a dollar given in funding to, say algebraic geometry research. Yet somehow we think it's important to fund all these things, and articles announcing new NSF grants for math research are not typically met with this kind of whatabout-ism.
- mankash666 10y ago
  OK - I should have not mentioned Flint, MI. My intention is not to play-down funding for fundamental research. But really, I think the NSF would agree with the non-urgent nature of funding "reducing potential risks from advanced AI". They probably get proposals for things that are much more immediate, say algebraic geometry research that produces quantum-resistant cryptography at the similar computational complexity of existing methods.
  - nojvek 10y ago
    I don't think they are just using the money for preventing Apocalypse. As Andrew Ng says worrying about this is kin to worrying about overpopulation on Mars. Openai will probably use the money to hire world class researchers and give them tools necessary to advance AI in a meaningful way that benefits humanity. The other big labs from Google, Facebook and MS seems to be driven by gathering as much data as possible from users, learning from it and presenting click bait ads to make more money.
    - nbouscal 10y ago
      I agree with what I take to be the main point of your argument, which is that OpenAI will use the money for a variety of different things, many of which aren't averting the apocalypse, and that those things are good. I think it's important to note, though, that Andrew Ng's perspective is far from a consensus one. For one thing, overpopulation on Mars is something that can easily be dealt with when it happens, whereas an intelligence explosion is much more of a flipped switch. For another, though there is a lot of uncertainty in the expert forecasts, a non-negligible amount of experts think there's a reasonable chance of human-level AI within the next 10-20 years, and that's absolutely something we should be worrying about right now. In a 2014 survey of the top 100 most cited living AI scientists[1], the median estimate of the year at which there was a 10% chance we'd have human-level AI was 2024. Multiply that by whatever chance you think human-level AI leads to an intelligence explosion and potential apocalypse, and then multiply it by the impact of that event… I think for most reasonable inputs you get a pretty huge EV as the result. [1]: http://sophia.de/pdf/2014_PT-AI_polls.pdf http://sophia.de/pdf/2014_PT-AI_polls.pdf
rl3 10y ago
>I think there are more important causes than "reducing potential risks from advanced AI". Honest to god, $30M will go a long way in saving lives TODAY. If the intelligence explosion hypothesis proves true even 100 years from now, it will have the potential to decide the fate of humanity. Given the stakes, beginning foundational research as early as possible is prudent. $30M in the grand scheme of things isn't much at all. If you were to express that amount as a fraction of the planet's total wealth, it would amount to something utterly infinitesimal. Yet, the potential payoffs are immeasurable. Philanthropy and research isn't some crude single-threaded triage process. The timescales involved, combined with the fact you can't just throw money at some problems demands a diverse approach that pursues many ends in parallel.
- rspeer 10y ago
  That's Pascal's Wager. What makes you think that OpenAI is doing anything that has any meaningful impact on AGI? We have no idea what AGI looks like.
  - acgourley 10y ago
    You've brought up Pascal's Wager twice in this thread, as if it really can end an argument by itself. Just because you don't like what high impact / low probability EV calculations do in Theology doesn't mean you can wave away high impact / low probability EV calculations in other domains.
    - Avshalom 10y ago
      >>high impact / low probability EV calculations in other domains Yeah but there isn't a calculation of the risk/probability of AGI there's just the appeal to the idea.
      - nbouscal 10y ago
        That's… not at all true? There have been numerous thorough attempts at coming up with as reasonable an estimate as possible, involving in-depth conversations with most of the experts in the field. Maybe it's not strictly a "calculation", but it's a hell of a lot more than just an appeal to the idea of AGI.
treehau5_ 10y ago
don't worry about the downvotes, those of us outside the Valley bubble agree.
- eastWestMath 10y ago
  Yeah I feel like it's turning AI into a bit of a joke amongst other theoretical computer scientists.
- eastWestMath 10y ago
  Yeah I feel like it's turning AI into a bit of a joke amongst other theoretical computer scientists.
AJ007 10y ago
Human species extinction is something almost everyone agrees shouldn't happen.
- technotony 10y ago
  I one person who isn't sure about that. I don't see why it wouldn't be good for us to evolve into something more advanced. Whether that's through melding with AI or advanced genetic engineering that outcome could be fantastic (assuming the transition is well managed, broadly distributed and happens through choice etc). It could lead to elimination of disease, aging, reduced ecological footprint, all kinds of benefits. And of course we've evolved before, it would be sad I think if this was the final state - I think we can do better!
  - nbouscal 10y ago
    Pretty sure AJ007 wasn't including transhumanism in the things they were calling "human species extinction", so I doubt you actually disagree.
  - pmichaud 10y ago
    I don't think people are primarily concerned about this outcome. They are more concerned about outcomes like "all the matter and energy in our light cone have been repurposed to maximize the total number of paperclips".
- MichaelGG 10y ago
  On the plus side it eliminates all human suffering. So from a negative utilitarian view it's a win. Unless, of course, there are more-suffering species out there that humans could eliminate suffering for.
guelo 10y ago
Following that line of reasoning to its logical conclusion there should only ever be one charity at a time which is the most important charity in the world as determined by you.
jaibot 10y ago
"We have to defuse this bomb or it will destroy the city in one hour!" "But there are people suffering right now!"
- idlewords 10y ago
  Twist: the bomb is a cartoon someone drew on a napkin
  - FeepingCreature 10y ago
    Twist: there's an actual bomb, but everybody thinks you're talking about a napkin.
idlewords 10y ago
The cult of 'AI risk' is a religious belief that brooks no dissent, and unfortunately has its claws in a lot of otherwise smart people.
- nbouscal 10y ago
  The cult of 'AI will be totally fine' is a religious belief that brooks no dissent, and unfortunately has its claws in a lot of otherwise smart people.
  - backpropaganda 10y ago
    That's a strawman. People who scoff at the "AI risk" cult don't believe that "AI will be totally fine". They just believe that it doesn't deserve the undue amount of attention it gets, which distracts from other more important and related risks, such as what Cambridge Analytica did for Trump, fake news due to internet, Facebook's bubbles, etc. The belief is to solve problems we have right now, or will have in 10-20 years, not the problems we'd have in 100 years.
    - nbouscal 10y ago
      I've been personally told by multiple different people who scoff at AI risk that it will be totally fine, so it's not a straw man. The median expert estimate for when we'll be 10% likely to have human-level AI is ~10 years. AI risk research didn't receive a penny of funding until the last few years, and is still funded at way lower levels than a lot of things that have dramatically less impact. In nearly every debate on the topic I've seen (with a few exceptions), the people concerned about AI risk have carefully considered the topic, are aware of the areas where there's still a lot of uncertainty, and make clear and well-hedged arguments that acknowledge that uncertainty; meanwhile the people who scoff at it haven't read any of the arguments (not even in popular book form in Superintelligence), haven't thought about most of the considerations, and have a general air of "assuming things will probably be fine". That's not a straw man, that's just direct observation of the state of the debate. People are doing serious academic work on the topic and have thought about it very deeply; the standard HN middlebrow dismissal is both common and inappropriate.
probably_wrong 10y ago
I have a simpler idea: how about they show how to stop a single AI from injuring/killing a single human? The army already developed the capability of bombing your house based on your metadata. Once the Pentagon decides to remove humans from the loop, how are these people going to stop them? Hell, even stopping a single DDOS would be awesome. Until they show they can do this extremely simple thing, I'll remain skeptic.

I think there are more important causes than "reducing potential risks from advanced AI". Honest to god, $30M will go a long way in saving lives TODAY. Flint, MI anyone?

acgourley 10y ago

Do you disagree with any research looking to prevent existential risk to our species, or do you believe AI cannot become an existential risk?

mankash666 10y ago
I think existential risks to our species of much higher magnitude exist today. Climate change and pollution for instance. I think a "possible existential threat in the future" is a weaker case for philanthropy than existing ones.
- curiousgal 10y ago
  Both of those issue cannot be solved by throwing money at them, especially the first. It requires political forces. Then again, the money could be used to buy politicians' votes in favor of environment-friendly policies. It's so blatant in the U.S. these days.

arikr 10y ago

It's about expected value. A 0.0001% chance of reducing an extreme risk will have a very high expected value.

rspeer 10y ago
You can justify lots of things with Pascal's wager, but most of them are religions.

davmre 10y ago

There are always going to be more important causes, under any particular person's view of "more important", which depends very strongly on both subjective values and (in the case of AI alignment work) on precise probabilities of far-future outcomes. A dollar given to the Against Malaria Foundation will do a lot more good, in QALY terms, than the same dollar spent in Flint. And both dollars will do more (direct) good than a dollar given in funding to, say algebraic geometry research. Yet somehow we think it's important to fund all these things, and articles announcing new NSF grants for math research are not typically met with this kind of whatabout-ism.

mankash666 10y ago
OK - I should have not mentioned Flint, MI. My intention is not to play-down funding for fundamental research. But really, I think the NSF would agree with the non-urgent nature of funding "reducing potential risks from advanced AI". They probably get proposals for things that are much more immediate, say algebraic geometry research that produces quantum-resistant cryptography at the similar computational complexity of existing methods.
- nojvek 10y ago
  I don't think they are just using the money for preventing Apocalypse. As Andrew Ng says worrying about this is kin to worrying about overpopulation on Mars. Openai will probably use the money to hire world class researchers and give them tools necessary to advance AI in a meaningful way that benefits humanity. The other big labs from Google, Facebook and MS seems to be driven by gathering as much data as possible from users, learning from it and presenting click bait ads to make more money.
  - nbouscal 10y ago
    I agree with what I take to be the main point of your argument, which is that OpenAI will use the money for a variety of different things, many of which aren't averting the apocalypse, and that those things are good. I think it's important to note, though, that Andrew Ng's perspective is far from a consensus one. For one thing, overpopulation on Mars is something that can easily be dealt with when it happens, whereas an intelligence explosion is much more of a flipped switch. For another, though there is a lot of uncertainty in the expert forecasts, a non-negligible amount of experts think there's a reasonable chance of human-level AI within the next 10-20 years, and that's absolutely something we should be worrying about right now. In a 2014 survey of the top 100 most cited living AI scientists[1], the median estimate of the year at which there was a 10% chance we'd have human-level AI was 2024. Multiply that by whatever chance you think human-level AI leads to an intelligence explosion and potential apocalypse, and then multiply it by the impact of that event… I think for most reasonable inputs you get a pretty huge EV as the result. [1]: http://sophia.de/pdf/2014_PT-AI_polls.pdf http://sophia.de/pdf/2014_PT-AI_polls.pdf

rl3 10y ago

>I think there are more important causes than "reducing potential risks from advanced AI". Honest to god, $30M will go a long way in saving lives TODAY. If the intelligence explosion hypothesis proves true even 100 years from now, it will have the potential to decide the fate of humanity. Given the stakes, beginning foundational research as early as possible is prudent. $30M in the grand scheme of things isn't much at all. If you were to express that amount as a fraction of the planet's total wealth, it would amount to something utterly infinitesimal. Yet, the potential payoffs are immeasurable. Philanthropy and research isn't some crude single-threaded triage process. The timescales involved, combined with the fact you can't just throw money at some problems demands a diverse approach that pursues many ends in parallel.

rspeer 10y ago
That's Pascal's Wager. What makes you think that OpenAI is doing anything that has any meaningful impact on AGI? We have no idea what AGI looks like.
- acgourley 10y ago
  You've brought up Pascal's Wager twice in this thread, as if it really can end an argument by itself. Just because you don't like what high impact / low probability EV calculations do in Theology doesn't mean you can wave away high impact / low probability EV calculations in other domains.
  - Avshalom 10y ago
    >>high impact / low probability EV calculations in other domains Yeah but there isn't a calculation of the risk/probability of AGI there's just the appeal to the idea.
    - nbouscal 10y ago
      That's… not at all true? There have been numerous thorough attempts at coming up with as reasonable an estimate as possible, involving in-depth conversations with most of the experts in the field. Maybe it's not strictly a "calculation", but it's a hell of a lot more than just an appeal to the idea of AGI.

treehau5_ 10y ago

don't worry about the downvotes, those of us outside the Valley bubble agree.

eastWestMath 10y ago
Yeah I feel like it's turning AI into a bit of a joke amongst other theoretical computer scientists.
eastWestMath 10y ago
Yeah I feel like it's turning AI into a bit of a joke amongst other theoretical computer scientists.

AJ007 10y ago

Human species extinction is something almost everyone agrees shouldn't happen.

technotony 10y ago
I one person who isn't sure about that. I don't see why it wouldn't be good for us to evolve into something more advanced. Whether that's through melding with AI or advanced genetic engineering that outcome could be fantastic (assuming the transition is well managed, broadly distributed and happens through choice etc). It could lead to elimination of disease, aging, reduced ecological footprint, all kinds of benefits. And of course we've evolved before, it would be sad I think if this was the final state - I think we can do better!
- nbouscal 10y ago
  Pretty sure AJ007 wasn't including transhumanism in the things they were calling "human species extinction", so I doubt you actually disagree.
- pmichaud 10y ago
  I don't think people are primarily concerned about this outcome. They are more concerned about outcomes like "all the matter and energy in our light cone have been repurposed to maximize the total number of paperclips".
MichaelGG 10y ago
On the plus side it eliminates all human suffering. So from a negative utilitarian view it's a win. Unless, of course, there are more-suffering species out there that humans could eliminate suffering for.

guelo 10y ago

Following that line of reasoning to its logical conclusion there should only ever be one charity at a time which is the most important charity in the world as determined by you.

jaibot 10y ago

"We have to defuse this bomb or it will destroy the city in one hour!" "But there are people suffering right now!"

idlewords 10y ago
Twist: the bomb is a cartoon someone drew on a napkin
- FeepingCreature 10y ago
  Twist: there's an actual bomb, but everybody thinks you're talking about a napkin.

idlewords 10y ago

The cult of 'AI risk' is a religious belief that brooks no dissent, and unfortunately has its claws in a lot of otherwise smart people.

nbouscal 10y ago
The cult of 'AI will be totally fine' is a religious belief that brooks no dissent, and unfortunately has its claws in a lot of otherwise smart people.
- backpropaganda 10y ago
  That's a strawman. People who scoff at the "AI risk" cult don't believe that "AI will be totally fine". They just believe that it doesn't deserve the undue amount of attention it gets, which distracts from other more important and related risks, such as what Cambridge Analytica did for Trump, fake news due to internet, Facebook's bubbles, etc. The belief is to solve problems we have right now, or will have in 10-20 years, not the problems we'd have in 100 years.
  - nbouscal 10y ago
    I've been personally told by multiple different people who scoff at AI risk that it will be totally fine, so it's not a straw man. The median expert estimate for when we'll be 10% likely to have human-level AI is ~10 years. AI risk research didn't receive a penny of funding until the last few years, and is still funded at way lower levels than a lot of things that have dramatically less impact. In nearly every debate on the topic I've seen (with a few exceptions), the people concerned about AI risk have carefully considered the topic, are aware of the areas where there's still a lot of uncertainty, and make clear and well-hedged arguments that acknowledge that uncertainty; meanwhile the people who scoff at it haven't read any of the arguments (not even in popular book form in Superintelligence), haven't thought about most of the considerations, and have a general air of "assuming things will probably be fine". That's not a straw man, that's just direct observation of the state of the debate. People are doing serious academic work on the topic and have thought about it very deeply; the standard HN middlebrow dismissal is both common and inappropriate.

probably_wrong 10y ago

I have a simpler idea: how about they show how to stop a single AI from injuring/killing a single human? The army already developed the capability of bombing your house based on your metadata. Once the Pentagon decides to remove humans from the loop, how are these people going to stop them? Hell, even stopping a single DDOS would be awesome. Until they show they can do this extremely simple thing, I'll remain skeptic.