Do you think "don't do this thing because you'll get zero evaluation score" is somehow perfectly reliable where "don't do this thing because it's bad" isn't? H

Do you think "don't do this thing because you'll get zero evaluation score" is somehow perfectly reliable where "don't do this thing because it's bad" isn't? Haha! Nope! LLMs have consequentialist thinking sometimes. And sometimes they don't. Sometimes they follow the prompts, and sometimes they don't. There's NO single magic prompt that fixes all the weird behavior of modern LLMs, and it's baffling that anyone who has ever interacted with an LLM would expect there to be one.