Fine tuning doesn't help avoid jailbreaking, it just makes it harder. So no, you're not always mucking with prompts and contexts. LLMs fail at following orders

Fine tuning doesn't help avoid jailbreaking, it just makes it harder. So no, you're not always mucking with prompts and contexts. LLMs fail at following orders in almost exactly the same ways that humans do, much to everyone's chagrin.