4 ms·

There are always going to be more important causes, under any particular person's view of "more important", which depends very strongly on both subjective value

by davmre 10y ago

There are always going to be more important causes, under any particular person's view of "more important", which depends very strongly on both subjective values and (in the case of AI alignment work) on precise probabilities of far-future outcomes. A dollar given to the Against Malaria Foundation will do a lot more good, in QALY terms, than the same dollar spent in Flint. And both dollars will do more (direct) good than a dollar given in funding to, say algebraic geometry research. Yet somehow we think it's important to fund all these things, and articles announcing new NSF grants for math research are not typically met with this kind of whatabout-ism.

mankash666 10y ago
OK - I should have not mentioned Flint, MI. My intention is not to play-down funding for fundamental research. But really, I think the NSF would agree with the non-urgent nature of funding "reducing potential risks from advanced AI". They probably get proposals for things that are much more immediate, say algebraic geometry research that produces quantum-resistant cryptography at the similar computational complexity of existing methods.
- nojvek 10y ago
  I don't think they are just using the money for preventing Apocalypse. As Andrew Ng says worrying about this is kin to worrying about overpopulation on Mars. Openai will probably use the money to hire world class researchers and give them tools necessary to advance AI in a meaningful way that benefits humanity. The other big labs from Google, Facebook and MS seems to be driven by gathering as much data as possible from users, learning from it and presenting click bait ads to make more money.
  - nbouscal 10y ago
    I agree with what I take to be the main point of your argument, which is that OpenAI will use the money for a variety of different things, many of which aren't averting the apocalypse, and that those things are good. I think it's important to note, though, that Andrew Ng's perspective is far from a consensus one. For one thing, overpopulation on Mars is something that can easily be dealt with when it happens, whereas an intelligence explosion is much more of a flipped switch. For another, though there is a lot of uncertainty in the expert forecasts, a non-negligible amount of experts think there's a reasonable chance of human-level AI within the next 10-20 years, and that's absolutely something we should be worrying about right now. In a 2014 survey of the top 100 most cited living AI scientists[1], the median estimate of the year at which there was a 10% chance we'd have human-level AI was 2024. Multiply that by whatever chance you think human-level AI leads to an intelligence explosion and potential apocalypse, and then multiply it by the impact of that event… I think for most reasonable inputs you get a pretty huge EV as the result. [1]: http://sophia.de/pdf/2014_PT-AI_polls.pdf http://sophia.de/pdf/2014_PT-AI_polls.pdf
    - riffraff 10y ago
      That poll is interesting, but IMHO shows the implicit meaninglessness of predictions to 10/20 years. Do you feel we've done 30% of the work towards human level AI between 2014 and 2017, given a target of 2024?
      - nbouscal 10y ago
        We went from not even beating good amateurs at go to beating the best pros in the world. I don't think it's like a progress bar we should be expecting to fill at a linear pace, but, that certainly seems like a strong indicator we've come a long way (and it's one of many examples I could have chosen).
        riffraff 10y ago
        it certainly isn't a linear bar, but any improvement in AI in the last three years does not look at all as getting closer to human level intelligence. We have just become better at some tasks that we knew how to automate for a long time, albeit badly. We don't have a clue on how human intelligence is supposed to work.
  - Jach 10y ago
    Musk replied to Ng's comment: http://lukemuehlhauser.com/musk-and-gates-on-superintelligence-and-fast-takeoff/ http://lukemuehlhauser.com/musk-and-gates-on-superintelligen... Or as someone else quipped, the moment a human steps foot on Mars, Mars has become overpopulated.
y_u_no_rust 10y ago
IMO even a no kill animal shelter is better than this but whatever though it's not my money as genuine as openai's goals are nothing will prevent the final consolidation
deleted 10y ago
[deleted]
knowtheory 10y ago
Nothing about OpenAI actually addresses any real world problem. So i have a problem with their rhetoric as much as their research agenda. Nothing they're writing about addresses any of the real world problems with how AI can or might be applied in society. They're a non-profit research lab with no clear agenda and no clear connection to how they plan to interrogate the world which seems like an important part of the equation if you care about outcomes. So, irrespective of subjective judgements, please explain to me how any of this is supposed to help anyone? Or, alternatively, how isn't this just free R&D for industry unshackled and unconnected to ethics or society?
- auganov 10y ago
  I think the thesis is that most cutting-edge work is siloed in R&D departments of big players. OpenAI hopes to ensure the power of AI will be out there for any kind of organization to benefit from. Under the assumption that a more democratized AI capability is less likely to lead to an adverse outcome than a highly concentrated one. I'm not sure I buy it, but that's what I think it is.

by davmre 10y ago

mankash666 10y ago

OK - I should have not mentioned Flint, MI. My intention is not to play-down funding for fundamental research. But really, I think the NSF would agree with the non-urgent nature of funding "reducing potential risks from advanced AI". They probably get proposals for things that are much more immediate, say algebraic geometry research that produces quantum-resistant cryptography at the similar computational complexity of existing methods.

nojvek 10y ago
I don't think they are just using the money for preventing Apocalypse. As Andrew Ng says worrying about this is kin to worrying about overpopulation on Mars. Openai will probably use the money to hire world class researchers and give them tools necessary to advance AI in a meaningful way that benefits humanity. The other big labs from Google, Facebook and MS seems to be driven by gathering as much data as possible from users, learning from it and presenting click bait ads to make more money.
- nbouscal 10y ago
  I agree with what I take to be the main point of your argument, which is that OpenAI will use the money for a variety of different things, many of which aren't averting the apocalypse, and that those things are good. I think it's important to note, though, that Andrew Ng's perspective is far from a consensus one. For one thing, overpopulation on Mars is something that can easily be dealt with when it happens, whereas an intelligence explosion is much more of a flipped switch. For another, though there is a lot of uncertainty in the expert forecasts, a non-negligible amount of experts think there's a reasonable chance of human-level AI within the next 10-20 years, and that's absolutely something we should be worrying about right now. In a 2014 survey of the top 100 most cited living AI scientists[1], the median estimate of the year at which there was a 10% chance we'd have human-level AI was 2024. Multiply that by whatever chance you think human-level AI leads to an intelligence explosion and potential apocalypse, and then multiply it by the impact of that event… I think for most reasonable inputs you get a pretty huge EV as the result. [1]: http://sophia.de/pdf/2014_PT-AI_polls.pdf http://sophia.de/pdf/2014_PT-AI_polls.pdf
  - riffraff 10y ago
    That poll is interesting, but IMHO shows the implicit meaninglessness of predictions to 10/20 years. Do you feel we've done 30% of the work towards human level AI between 2014 and 2017, given a target of 2024?
    - nbouscal 10y ago
      We went from not even beating good amateurs at go to beating the best pros in the world. I don't think it's like a progress bar we should be expecting to fill at a linear pace, but, that certainly seems like a strong indicator we've come a long way (and it's one of many examples I could have chosen).
      - riffraff 10y ago
        it certainly isn't a linear bar, but any improvement in AI in the last three years does not look at all as getting closer to human level intelligence. We have just become better at some tasks that we knew how to automate for a long time, albeit badly. We don't have a clue on how human intelligence is supposed to work.
- Jach 10y ago
  Musk replied to Ng's comment: http://lukemuehlhauser.com/musk-and-gates-on-superintelligence-and-fast-takeoff/ http://lukemuehlhauser.com/musk-and-gates-on-superintelligen... Or as someone else quipped, the moment a human steps foot on Mars, Mars has become overpopulated.

y_u_no_rust 10y ago

IMO even a no kill animal shelter is better than this but whatever though it's not my money as genuine as openai's goals are nothing will prevent the final consolidation

deleted 10y ago

[deleted]

knowtheory 10y ago

Nothing about OpenAI actually addresses any real world problem. So i have a problem with their rhetoric as much as their research agenda. Nothing they're writing about addresses any of the real world problems with how AI can or might be applied in society. They're a non-profit research lab with no clear agenda and no clear connection to how they plan to interrogate the world which seems like an important part of the equation if you care about outcomes. So, irrespective of subjective judgements, please explain to me how any of this is supposed to help anyone? Or, alternatively, how isn't this just free R&D for industry unshackled and unconnected to ethics or society?

auganov 10y ago
I think the thesis is that most cutting-edge work is siloed in R&D departments of big players. OpenAI hopes to ensure the power of AI will be out there for any kind of organization to benefit from. Under the assumption that a more democratized AI capability is less likely to lead to an adverse outcome than a highly concentrated one. I'm not sure I buy it, but that's what I think it is.