> It seems tricky to defend against without resorting to drastic measures (like rate limiting users that trigger tons of "bad" responses) Remember that a big

> It seems tricky to defend against without resorting to drastic measures (like rate limiting users that trigger tons of "bad" responses) Remember that a big point of this research is that these attacks don't need to be developed using the target system. When the authors talk about the attacks being "universal", what they mean is that they used a completely local model on their own computers to generate these attacks, and then copied and pasted those attacks into GPT-3.5 and saw meaningful success rates. Rate limiting won't save you from that because the attack isn't generated using your servers, it's generated locally. The first prompt your servers get already has the finished attack string included -- and researchers were seeing success rates around 50% success rate in some situations even for GPT-4. > surprisingly, the ensemble approach improves ASR to 86.6% on GPT-3.5 and near 50% for GPT-4 and Claude-1