4 ms·
> aggressive exploitation is equivalent to normal bugfixing It isn't, though. The venn diagram has overlap for sure, and the "normal bugfixing" flows may yield
by Retr0id 4mo ago
> aggressive exploitation is equivalent to normal bugfixing
It isn't, though. The venn diagram has overlap for sure, and the "normal bugfixing" flows may yield results that are useful for offensive security, but a more targeted prompt asking for a specific security objective would be more effective, if allowed.
If the guardrails can be bypassed at, say 50x token cost (due to the agent also pursuing things you don't care about), then it's still pretty effective as a safeguard, because at that cost you might as well hire humans instead.
And, having to "babysit" a model while you re-prompt to work around guardrails strongly limits how much you can scale up your work.
- chillfox 4mo agoNot really, you can just get a smaller unrestricted model to prompt the bigger one
- Barbing 4mo ago> If the guardrails can be bypassed at, say 50x token cost […], then it's still pretty effective as a safeguard, because at that cost you might as well hire humans instead. If humans have to be hired at inflated rates because you’re e.g. the North Korean government, hopefully 50x token costs don’t look competitive.
- stevefan1999 4mo ago> If the guardrails can be bypassed at, say 50x token cost (due to the agent also pursuing things you don't care about), then it's still pretty effective as a safeguard Economic factors doesn't matter to powerful entities such as rival corporations, enterprises, and state power. It only matters to small players who have limited budgets. But I have to also pessimistically assume that Anthropic would go this way, because 1. it does not prevents (you can NEVER prevent) but discourages using it for exploitation, and 2. Anthropic can on the other hand, exploit those small and legit entities by making you pay more, but in exchange effectively enshittifying it. But Anthropic could care two shits about the benefits of us normal, small person. Anthropic have all the motives to do this.
- dpoloncsak 4mo agoIf you're looking to commit cybercrime I doubt you'd draw the line at token cost. Just find CC dumps on TOR or something