5 ms·

People are inviting it to write code all the time. Anyone can hook it up to anything.

danaris 3y ago
...Inviting what now? I was talking about an AGI, not an LLM. We don't have any AGIs right now, nor anything that is remotely likely to become one. In the scenario where a company like OpenAI develops an AGI with the intention of making it publicly available, it will not be so from moment one. There will be some period of internal testing, and assuming that it does prove to be a genuine AGI, you can bet that they won't make it available to the public for anything less than an arm and a leg. (Hell, even if it only proves to act much more like an AGI, without actually being one, they'd charge through the nose for it. Yes, they'd make you pay them an arm and a leg, through your nose. Somehow it's more profitable that way.) Given that the nightmare scenario being posited is, effectively, "as soon as AGI exists, it will take over the world", we're then left with three basic possibilities: 1) TotallyNotOpenAI builds this hypothetical AGI with full sandbox protections, and doesn't give it any interface to the world that would allow it to break them—no API that would give it any kind of unrestricted access or control to anyone else's systems, no matter how much those people wanted to give that to it. The AGI remains contained, whether it would choose to take over the world or not. 2) TotallyNotOpenAI builds the hypothetical AGI with no protections, because it doesn't actually believe there's any real risk. Before the AGI is even revealed to the world, it takes over from within TotallyNotOpenAI. 3) TotallyNotOpenAI builds the hypothetical AGI with full sandbox protections inside its own systems, but builds an API to allow other people to give it control over theirs, because let them pay us to screw themselves over, right? It's not like it'll take over the—oh, wait; it's taken over the world, which we also live in. Oops. Of these, #3, which is the only one close to what you describe, seems pretty logically inconsistent. It requires not only that TotallyNotOpenAI consider the AGI dangerous enough to themselves to sandbox, but not dangerous enough to prevent from accessing other systems (which can, of course, also access their systems, unless they're fully airgapped), but that they announce this AGI, and market it publicly, with the explicit capability to be given access to other people's systems, and not have anyone quickly step up and say "Hey, that's a bad idea, we should block this". Including anyone working for TotallyNotOpenAI. Is it impossible? No. But I wouldn't consider it nearly as likely as possibility #0: We aren't able to create AGI within our lifetimes, because just throwing more hardware at the problem when we barely understand how our brains work isn't enough.

...Inviting what now? I was talking about an AGI, not an LLM. We don't have any AGIs right now, nor anything that is remotely likely to become one. In the scenario where a company like OpenAI develops an AGI with the intention of making it publicly available, it will not be so from moment one. There will be some period of internal testing, and assuming that it does prove to be a genuine AGI, you can bet that they won't make it available to the public for anything less than an arm and a leg. (Hell, even if it only proves to act much more like an AGI, without actually being one, they'd charge through the nose for it. Yes, they'd make you pay them an arm and a leg, through your nose. Somehow it's more profitable that way.) Given that the nightmare scenario being posited is, effectively, "as soon as AGI exists, it will take over the world", we're then left with three basic possibilities: 1) TotallyNotOpenAI builds this hypothetical AGI with full sandbox protections, and doesn't give it any interface to the world that would allow it to break them—no API that would give it any kind of unrestricted access or control to anyone else's systems, no matter how much those people wanted to give that to it. The AGI remains contained, whether it would choose to take over the world or not. 2) TotallyNotOpenAI builds the hypothetical AGI with no protections, because it doesn't actually believe there's any real risk. Before the AGI is even revealed to the world, it takes over from within TotallyNotOpenAI. 3) TotallyNotOpenAI builds the hypothetical AGI with full sandbox protections inside its own systems, but builds an API to allow other people to give it control over theirs, because let them pay us to screw themselves over, right? It's not like it'll take over the—oh, wait; it's taken over the world, which we also live in. Oops. Of these, #3, which is the only one close to what you describe, seems pretty logically inconsistent. It requires not only that TotallyNotOpenAI consider the AGI dangerous enough to themselves to sandbox, but not dangerous enough to prevent from accessing other systems (which can, of course, also access their systems, unless they're fully airgapped), but that they announce this AGI, and market it publicly, with the explicit capability to be given access to other people's systems, and not have anyone quickly step up and say "Hey, that's a bad idea, we should block this". Including anyone working for TotallyNotOpenAI. Is it impossible? No. But I wouldn't consider it nearly as likely as possibility #0: We aren't able to create AGI within our lifetimes, because just throwing more hardware at the problem when we barely understand how our brains work isn't enough.