3 ms·
I watched the latest Karpathy video, where he shows that you can create a prompt suffix that looks like jibberish, using an iterative approach. It looks like th
by verdverm 3y ago
I watched the latest Karpathy video, where he shows that you can create a prompt suffix that looks like jibberish, using an iterative approach. It looks like this technique will be impossible to defend against, because it discovers for an arbitrary string that causes the jailbreak.
Looks like this paper is building on the token level attack from that work.
The base64 encoding technique was amusing as well
https://youtu.be/zjkBMFhNj_g https://youtu.be/zjkBMFhNj_g