4 ms·

What's your suggested alternative? It's really easy to criticize, but a lot harder to do any better. If you aren't suggesting any other methods, you're implicit

What's your suggested alternative? It's really easy to criticize, but a lot harder to do any better. If you aren't suggesting any other methods, you're implicitly saying we should just ignore the problem. I really don't think that will end well.

argonaut 10y ago
You admit that you don't know what the actual probabilities are. Or you admit you have an extremely low confidence in your estimate of the probabilities.
- nbouscal 10y ago
  Have you read any of the estimates? Basically every expert gives a huge confidence interval, as do most of the people actively working on AI risk.
  - argonaut 10y ago
    Saying you are 95% confident AGI will come between 50-100 years (a "huge" confidence interval), is still far too confident.
    - nbouscal 10y ago
      I don't recall seeing 95% confidence intervals that narrow. Even the 90% confidence intervals I see are usually like 5-150 or something. (Also it would be super weird to me if the lower bound of someone's 95%CI was 50.)
rspeer 10y ago
My suggested alternative is prioritizing problems appropriately. This "Open Philantropy Project" is a guy taking money that used to be for curing malaria (and I respect what he used to do on that front!) and giving it to his roommate. You can do better solving a terrible problem that does exist than solving a sci-fi problem that doesn't exist. I know it's usually uncouth to compare different charities like this, but that's exactly what Effective Altruism was supposed to be about, and this cause directly competes with curing malaria. If more than zero AGI technology starts existing, we can re-prioritize. It makes no sense for a field to go from not existing to superhuman performance without anyone noticing.
- nbouscal 10y ago
  Nobody thinks we won't notice when it's more imminent; the concern is that capability research has always been faster than safety research. If we wait until it's imminent, we risk having no chance of catching up in time. Why not devote a tiny fraction of our funding to starting the research now, so we can be ready on time? Worst case we're early... which in this context is a lot better than being late.
  - Avshalom 10y ago
    Worst case the safety research ends up causing the AGI to be actively antagonistic. Slightly less worst case it never leads anywhere and AGI ends up turning us into mouthless slugs anyway. Third-ish worst it makes AGI suborn-able and whoever gets to it first becomes our god-king. many many worse situations later we get to: It never leads anywhere but AGI never happens anyway and it just ends up being welfare-for-future-phobes instead of curing debilitating disease or buying every homeless person in a city a new pair of shoes or whatever. "we're early" is in fact the best possible scenario, actually it's basically the only positive scenario.

argonaut 10y ago

You admit that you don't know what the actual probabilities are. Or you admit you have an extremely low confidence in your estimate of the probabilities.

nbouscal 10y ago
Have you read any of the estimates? Basically every expert gives a huge confidence interval, as do most of the people actively working on AI risk.
- argonaut 10y ago
  Saying you are 95% confident AGI will come between 50-100 years (a "huge" confidence interval), is still far too confident.
  - nbouscal 10y ago
    I don't recall seeing 95% confidence intervals that narrow. Even the 90% confidence intervals I see are usually like 5-150 or something. (Also it would be super weird to me if the lower bound of someone's 95%CI was 50.)

rspeer 10y ago

My suggested alternative is prioritizing problems appropriately. This "Open Philantropy Project" is a guy taking money that used to be for curing malaria (and I respect what he used to do on that front!) and giving it to his roommate. You can do better solving a terrible problem that does exist than solving a sci-fi problem that doesn't exist. I know it's usually uncouth to compare different charities like this, but that's exactly what Effective Altruism was supposed to be about, and this cause directly competes with curing malaria. If more than zero AGI technology starts existing, we can re-prioritize. It makes no sense for a field to go from not existing to superhuman performance without anyone noticing.

nbouscal 10y ago
Nobody thinks we won't notice when it's more imminent; the concern is that capability research has always been faster than safety research. If we wait until it's imminent, we risk having no chance of catching up in time. Why not devote a tiny fraction of our funding to starting the research now, so we can be ready on time? Worst case we're early... which in this context is a lot better than being late.
- Avshalom 10y ago
  Worst case the safety research ends up causing the AGI to be actively antagonistic. Slightly less worst case it never leads anywhere and AGI ends up turning us into mouthless slugs anyway. Third-ish worst it makes AGI suborn-able and whoever gets to it first becomes our god-king. many many worse situations later we get to: It never leads anywhere but AGI never happens anyway and it just ends up being welfare-for-future-phobes instead of curing debilitating disease or buying every homeless person in a city a new pair of shoes or whatever. "we're early" is in fact the best possible scenario, actually it's basically the only positive scenario.