5 ms·

Aaaand the top thing on the list is AI x-risk. Of course. If there's anyone left in EA willing to listen: I'm begging you, please stop this foolishness. To eve

Aaaand the top thing on the list is AI x-risk. Of course. If there's anyone left in EA willing to listen: I'm begging you, please stop this foolishness. To everyone outside the bubble, you look like lunatics, and it has done immense and possibly irreparable damage to the EA "brand" which could otherwise be capable of such great things.

ramchip 9y ago
EA?
- enruwhuy 9y ago
  Effective altruism, I guess?
  - ramchip 9y ago
    Thanks, looks like it indeed.
akvadrako 9y ago
Although I consider AI risks more significant than everything else combined, you do have a point. The danger isn't communicated well and on top of that, nobody has a good defensive plan (IMHO). That means attacking the problem directly isn't an effective approach. Especially if it creates AI-risk fatigue.
- chriswarbo 9y ago
  Agreed. The immediate action we should be taking to mitigate AGI risk is basically identical to the immediate action we should already be taking to research, regulate, hold-accountable, etc. existing dumb-AI systems (e.g. autonomous/remote-control killing machines, black-box oracles being used for medical diagnoses/credit/etc., economic impacts w.r.t. increasing inequality, "taking our jobs", etc.). Even Barack Obama has said as much, so I think all of the sky-is-falling social media buzz is just a hindrance. We should absolutely be funding blue-sky research into mitigating, predicting, controlling, etc. AGIs if/when they're developed. But we are doing that (e.g. intelligence.org ) so, again, the doomsday prophets aren't contributing anything.
dmichulke 9y ago
I've been working on "general" AI quite a few years, both academically and professionally and IMO we're decades away from anything general, let alone the singularity. Given this, I wonder where is the bubble. Of course, I see a lot of blog posts and hype, startups here and there, companies pouring billions into "AI". Yet, I cannot participate in that bubble because: - big companies (GOOG and the likes) are not positioned for AI only and are quite expensive already (maybe it's rather a tech or even stock market bubble there) - small companies (I assume) are mostly start-ups, so there is no way to "get in" except as a VC So yes, I see the signs of the bubble but I don't see where "mom and pop" come into play there. Any suggestions as to what makes this bubble a real investment bubble? Or does bubble in this case mean something different?
- jholman 9y ago
  By "bubble", Analemma meant a distorting metaphorical sphere of delusion. Like "filter bubble", but more general. I think your first paragraph means you agree with the analysis. There's also an investment surge in AI, which might be a bubble, but if you're asking how to participate, then I suspect you're not going to get to participate. Plus, the word "bubble", in the context of investment, means something about to pop. If it's an actual investment bubble, you don't want in.
- BenjaminTodd 9y ago
  We think even if AGI is only invented in 50 years, then it's still worth doing some work on the control problem today, so our view isn't that dependent on whether we're currently in an AI investment bubble or not. However, if you did really believe the hype, and think that ML will lead to major breakthroughs in the next decade or so, then that would make it even more urgent.
probably_wrong 9y ago
I know this is going to sound mean, but I hope it drives the point across: have the EA people actually done something so far? Besides talking, I mean. I have yet to see a single AI's malicious behavior stopped by a proponent of AI risk, and that's considering that today's AI is still pretty dumb. I want to believe that they have released code, a specific policy, or something that I can use right now to stop the dumbest of botnets, but I'm not aware of anything. Anyone better informed that me could chime in?
- rspeer 9y ago
  They're so worried about AGI, which doesn't exist, that they won't spend any time on the non-existential risks of non-general AI. Listen to Cathy O'Neil for a more reasonable and actionable perspective on how we need to fix AI.
  - probably_wrong 9y ago
    Anything specific that I should listen to? She seems to be a prolific writer, and I'm not sure where to start.
    - rspeer 9y ago
      I suggest her book "Weapons of Math Destruction" in particular.
- tgb 9y ago
  Why is that the metric that you would judge EA on? Huge portions of EA are entirely uninterested in AI risk. And AI risk isn't interested in botnets. Honestly, have you read any of their material? If not, that's fine you don't have to be interested in it (it's getting plenty of attention at the moment and really is no longer a neglected cause) but why bother criticizing something you haven't attempted to understand? Either way, MIRI has published some papers that they think are working towards their goal. https://intelligence.org/research/ https://intelligence.org/research/ I'm not well informed enough on this subject to try to judge that claim, though, and overall I'm conflicted as to how important I should rate AI risk. But I think having some people work on it is good, in much the same way that I value basic science research.
notahacker 9y ago
You're actually being too kind. The only three "recommended" causes to give money to are "research organizations like the unusual ones we're involved with". Everything else, from global health to climate change is lower priority they'd caution you to only "sometimes" give money to. This isn't research, it's a thinly disguised and exceptionally cynical fundraising pitch. Which is a shame, because I actually really like the basic idea behind 8000hours
- BenjaminTodd 9y ago
  We're going to work on the causes that we think our most pressing, so to be consistent, we're going to end up recommending the same things we work on. At 80,000 Hours, our focus is also not on giving money. We're trying to help people find jobs in these areas, often in academia or policy and many other sectors.

ramchip 9y ago

EA?

enruwhuy 9y ago
Effective altruism, I guess?
- ramchip 9y ago
  Thanks, looks like it indeed.

akvadrako 9y ago

Although I consider AI risks more significant than everything else combined, you do have a point. The danger isn't communicated well and on top of that, nobody has a good defensive plan (IMHO). That means attacking the problem directly isn't an effective approach. Especially if it creates AI-risk fatigue.

chriswarbo 9y ago
Agreed. The immediate action we should be taking to mitigate AGI risk is basically identical to the immediate action we should already be taking to research, regulate, hold-accountable, etc. existing dumb-AI systems (e.g. autonomous/remote-control killing machines, black-box oracles being used for medical diagnoses/credit/etc., economic impacts w.r.t. increasing inequality, "taking our jobs", etc.). Even Barack Obama has said as much, so I think all of the sky-is-falling social media buzz is just a hindrance. We should absolutely be funding blue-sky research into mitigating, predicting, controlling, etc. AGIs if/when they're developed. But we are doing that (e.g. intelligence.org ) so, again, the doomsday prophets aren't contributing anything.

dmichulke 9y ago

I've been working on "general" AI quite a few years, both academically and professionally and IMO we're decades away from anything general, let alone the singularity. Given this, I wonder where is the bubble. Of course, I see a lot of blog posts and hype, startups here and there, companies pouring billions into "AI". Yet, I cannot participate in that bubble because: - big companies (GOOG and the likes) are not positioned for AI only and are quite expensive already (maybe it's rather a tech or even stock market bubble there) - small companies (I assume) are mostly start-ups, so there is no way to "get in" except as a VC So yes, I see the signs of the bubble but I don't see where "mom and pop" come into play there. Any suggestions as to what makes this bubble a real investment bubble? Or does bubble in this case mean something different?

jholman 9y ago
By "bubble", Analemma meant a distorting metaphorical sphere of delusion. Like "filter bubble", but more general. I think your first paragraph means you agree with the analysis. There's also an investment surge in AI, which might be a bubble, but if you're asking how to participate, then I suspect you're not going to get to participate. Plus, the word "bubble", in the context of investment, means something about to pop. If it's an actual investment bubble, you don't want in.
BenjaminTodd 9y ago
We think even if AGI is only invented in 50 years, then it's still worth doing some work on the control problem today, so our view isn't that dependent on whether we're currently in an AI investment bubble or not. However, if you did really believe the hype, and think that ML will lead to major breakthroughs in the next decade or so, then that would make it even more urgent.

probably_wrong 9y ago

I know this is going to sound mean, but I hope it drives the point across: have the EA people actually done something so far? Besides talking, I mean. I have yet to see a single AI's malicious behavior stopped by a proponent of AI risk, and that's considering that today's AI is still pretty dumb. I want to believe that they have released code, a specific policy, or something that I can use right now to stop the dumbest of botnets, but I'm not aware of anything. Anyone better informed that me could chime in?

rspeer 9y ago
They're so worried about AGI, which doesn't exist, that they won't spend any time on the non-existential risks of non-general AI. Listen to Cathy O'Neil for a more reasonable and actionable perspective on how we need to fix AI.
- probably_wrong 9y ago
  Anything specific that I should listen to? She seems to be a prolific writer, and I'm not sure where to start.
  - rspeer 9y ago
    I suggest her book "Weapons of Math Destruction" in particular.
tgb 9y ago
Why is that the metric that you would judge EA on? Huge portions of EA are entirely uninterested in AI risk. And AI risk isn't interested in botnets. Honestly, have you read any of their material? If not, that's fine you don't have to be interested in it (it's getting plenty of attention at the moment and really is no longer a neglected cause) but why bother criticizing something you haven't attempted to understand? Either way, MIRI has published some papers that they think are working towards their goal. https://intelligence.org/research/ https://intelligence.org/research/ I'm not well informed enough on this subject to try to judge that claim, though, and overall I'm conflicted as to how important I should rate AI risk. But I think having some people work on it is good, in much the same way that I value basic science research.

notahacker 9y ago

You're actually being too kind. The only three "recommended" causes to give money to are "research organizations like the unusual ones we're involved with". Everything else, from global health to climate change is lower priority they'd caution you to only "sometimes" give money to. This isn't research, it's a thinly disguised and exceptionally cynical fundraising pitch. Which is a shame, because I actually really like the basic idea behind 8000hours

BenjaminTodd 9y ago
We're going to work on the causes that we think our most pressing, so to be consistent, we're going to end up recommending the same things we work on. At 80,000 Hours, our focus is also not on giving money. We're trying to help people find jobs in these areas, often in academia or policy and many other sectors.