2 ms·
Wave equations don't exist in the real world, nor do they collapse. The photon doesn't decide which slit to go through when you look at it. Electrons don't sp
by ineptech 4y ago
Wave equations don't exist in the real world, nor do they collapse. The photon doesn't decide which slit to go through when you look at it. Electrons don't spin. Quarks don't have colors. Those are stories we tell ourselves to visualize and explain why certain experiments produce certain outcomes. We teach them, not because they're true, but because they're useful: they do a better job of explaining the results we see in the real world than any other set of stories.
Similarly, the question at hand is not whether OP's essay is silly (it is) or whether it's true (like all models, it is not), but whether it's useful, as measured by whether this mental model helps people do a better job of jailbreaking/hardening LLMs. And like you, I'm not convinced by the example at the end[0], but I can at least see straightforward ways to test it, and that's a lot more than you can say of most blog posts like this. For all of the people in this comment thread calling it stupid, has anyone mentioned one they think is better?
0: Please note that OP's evidence was not that they jailbroke the chatbot - it's that, after that initial prompt, they were able to elicit further banned stuff with little prodding.