3 ms·
>Why would ChatGPT provide me with dangerous code if I prompt it benignly? Because it's a black box that's sensitive to unexpected inputs. Humans create bugs t
by antibasilisk 4y ago
>Why would ChatGPT provide me with dangerous code if I prompt it benignly?
Because it's a black box that's sensitive to unexpected inputs. Humans create bugs too, but an unaligned AI doesn't have the sense to be careful of certain classes of bug.
- maxbond 4y agoThe burden of proof for executing arbitrary code as input should always be on proving that it doesn't violate your security or reliability model. You shouldn't need to assume anything about that code other than that you're able to configure the runtime before it executes. Whether it comes from ChatGPT or users shouldn't matter, you should assume some of those programs were written by hackers and that you don't know which ones. Even if you trust ChatGPT you should do this. You should assume that one day the hackers will intercept your connection to OpenAI and hand you malware, and they should either fail or be forced to use a zero day exploit against eg Firecracker (modulo your threat model and such).