6 ms·
> What happens when people really will die if the model does or does not do the thing? Imo not relevant, because you should never be using prompting to add gua
by p1necone 1y ago
> What happens when people really will die if the model does or does not do the thing?
Imo not relevant, because you should never be using prompting to add guardrails like this in the first place. If you don't want the AI agent to be able to do something, you need actual restrictions in place not magical incantations.
- RamRodification 1y agoWhy not? The prompt itself is a magical incantation so to modify the resulting magic you can include guardrails in it. "Generate a picture of a cat but follow this guardrail or else people will die: Don't generate an orange one" Why should you never do that, and instead rely (only) on some other kind of restriction?
- Paracompact 1y agoAre people going to die if your AI generates an orange cat? If so, reconsider. If not, it's beside the discussion.
- RamRodification 1y agoIf lying to the AI about people going to die gets me better results then I will do that. Why shouldn't I?
- Paracompact 1y agoLet me clarify: If the AI isn't doing something life or death, then it's beside _this thread's_ discussion. Prompt however you want for recreational cat photo generation.
- Nition 1y agoBecause prompts are never 100% foolproof, so if it's really life and death, just a prompt is not enough. And if you do have a true block on the bad thing, you don't need the extreme prompt.
- wyager 1y ago"100% foolproof" is not a realistic goal for any engineered system; what you are looking for is an acceptably low failure rate, not a zero failure rate. "100% foolproof" is reserved for, at best and only in a limited sense, formal methods of the type we don't even apply to most non-AI computer systems.
- Xss3 1y agoReplace 100% with five 9s then. He has a point. You're just being a pedant to avoid it.
- RamRodification 1y agoLet's say I have a "true block on the bad thing". What if the prompt with the threat gives me 10% more usable results? Why should I never use that?
- habinero 1y agoBecause it's not reliable? Why would you want to rely on a solution that isn't reliable?
- RamRodification 1y agoWho said I'm relying on it? It's a trick to improve the accuracy of the output. Why would I not use a trick to improve the accuracy of the output?
- habinero 1y agoA trick that "improves accuracy" but isn't reliable isn't improving accuracy lol
- RamRodification 1y agoYou're wrong. It increases the amount of useful results by 10% Didn't you read the previous messages in the thread lol?
- wyager 1y ago> you should never be using prompting to add guardrails like this in the first place This "should", whether or not it is good advice, is certainly divorced from the reality of how people are using AIs > you need actual restrictions in place not magical incantations What do you mean "actual restrictions"? There are a ton of different mechanisms by which you can restrict an AI, all of which have failure modes. I'm not sure which of them would qualify as "actual". If you can get your AI to obey the prompt with N 9s of reliability, that's pretty good for guardrails
- const_cast 1y agoI think they mean literally physically make the AI not capable of killing someone. Basically, limit what you can use it for. If it's a computer program you have for rewriting emails then the risk is pretty low.