3 ms·

Yes, the simplest explanation for this document (and the substantial internal efforts that it reflects) is that it's actually just a cynical marketing ploy, rat

Yes, the simplest explanation for this document (and the substantial internal efforts that it reflects) is that it's actually just a cynical marketing ploy, rather than the organization's actual stance with respect to advancing AI capabilities. State your accusation plainly: you think that Anthropic is spending a double-digit percentage of its headcount on pretending to care about catastrophic risks, in order to better fleece investors? Do you think those dozens or hundreds of employees are all in on it too? (They aren't; I know a bunch of people at Anthropic and they take extinction risk quite seriously. I think some of them should quit their jobs, but that's a different story.)

shmatt 2y ago
Very honestly asking - how do you convince investors you’re $100B away from an independent thinking computer if you’re not hiring to show that? I’m sure these people are very serious about their work - do they actually know how far we are - technologically, spend, and time wise from real non word generating AGI with independent thought processes? It’s an amazing research subject. And even more amazing a corporation is willing to pay people to research it. But it doesn’t mean it’s close in any way, or that anthropic would reach that goal in a decade or 3 I would compare spending this money and hiring these people to what Google Moonshot tried to do long ago. Very cool, very interesting, but also there should be a caveat on how far away it is in reality
- comp_throw7 2y ago
  I think that if I tried to rank-order strategies optimizing for fundraising, "act as if I'm trying to invent technology that I think stands a decent chance of causing human extinction, in the limit" would not come close to making the cut. I don't see Anthropic making very confident claims about when they're going to achieve AGI (however you want to define that). Predicting how long it'll take to produce a specific novel scientific result is, by its very nature, pretty difficult. (You might have some guesses, if you have a comprehensive understanding of what unsolved dependencies there are, and have some reason to believe you know how long it'll take to solve _those_, and that's very much not the case here. But if you're in that kind of situation, it's much more likely you're dealing with an engineering problem, not a research problem.) Elsewhere in the comments on this link, their CISO predicts a 50% chance of hitting capabilities that'll trigger their ASL-3 standard in the next 6 months (my guess is on the strength of its ability to find vulnerabilities in open-source codebases). That's predicting the timeline for a small advancement in a relatively narrow set of capabilities where we can at least sort of measure progress.

shmatt 2y ago

Very honestly asking - how do you convince investors you’re $100B away from an independent thinking computer if you’re not hiring to show that? I’m sure these people are very serious about their work - do they actually know how far we are - technologically, spend, and time wise from real non word generating AGI with independent thought processes? It’s an amazing research subject. And even more amazing a corporation is willing to pay people to research it. But it doesn’t mean it’s close in any way, or that anthropic would reach that goal in a decade or 3 I would compare spending this money and hiring these people to what Google Moonshot tried to do long ago. Very cool, very interesting, but also there should be a caveat on how far away it is in reality

comp_throw7 2y ago
I think that if I tried to rank-order strategies optimizing for fundraising, "act as if I'm trying to invent technology that I think stands a decent chance of causing human extinction, in the limit" would not come close to making the cut. I don't see Anthropic making very confident claims about when they're going to achieve AGI (however you want to define that). Predicting how long it'll take to produce a specific novel scientific result is, by its very nature, pretty difficult. (You might have some guesses, if you have a comprehensive understanding of what unsolved dependencies there are, and have some reason to believe you know how long it'll take to solve _those_, and that's very much not the case here. But if you're in that kind of situation, it's much more likely you're dealing with an engineering problem, not a research problem.) Elsewhere in the comments on this link, their CISO predicts a 50% chance of hitting capabilities that'll trigger their ASL-3 standard in the next 6 months (my guess is on the strength of its ability to find vulnerabilities in open-source codebases). That's predicting the timeline for a small advancement in a relatively narrow set of capabilities where we can at least sort of measure progress.