3 ms·
Do they still threaten to terminate your account if they think you're trying to introspect its hidden chain-of-thought process?
by jsheard 2y ago
Do they still threaten to terminate your account if they think you're trying to introspect its hidden chain-of-thought process?
- visarga 2y agoA few days ago the QwQ-32B model was released, it uses the same kind of reasoning style. So I took one sample and reverse engineered the prompt with Sonnet 3.5. Now I can just paste this prompt into any LLM. It's all about expressing doubt, double checking and backtracking on itself. I am kind of fond of this response style, it seems more genuine and openended. https://pastebin.com/raw/5AVRZsJg https://pastebin.com/raw/5AVRZsJg
- marviel 2y agoThanks, I love this
- AlfredBarnes 2y agoThank you for doing that work, and even more for sharing it. I will have to try this out.
- thegabriele 2y agoI tried this with LeChat (mistral) and ChatGPT 3.5 (free) and they start to respond to "something" following the style but... without any question asked.
- rsync 2y agoAn aside ... Isn't it wonderful that, after all of these years, the pastebin "primitive" is still available and usable ... One could have needed pastebin, used it, then spent a decade not needing it, then returned for an identical repeat use. The longevity alone is of tremendous value.
- SirYandi 2y agoAnd then once the answer is found an additional prompt is given to tidy up and present the solution clearly?
- RestartKernel 2y agoInterestingly, this prompt breaks o1-mini and o1-preview for me, while 4o works as expected — they immediately jump from "thinking" to "finished thinking" without outputting anything (including thinking steps). Maybe it breaks some specific syntax required by the original system prompt? Though you'd think OpenAI would know to prevent this with their function calling API and all, so it might just be triggering some anti-abuse mechanism without going so far as to give a warning.
- int_19h 2y agoA prompt is not a substitute for a model that is specifically fine-tuned to do CoT with backtracking etc.
- foundry27 2y agoWeirdly enough, a few minutes ago I was using o1 via ChatGPT and it started consistently repeating its complete chain of thought back to me for every question I asked, with a 1-1 mapping to the little “thought process” summaries ChatGPT provides for o1’s answers. My system prompt does say something to the effect of “explain your reasoning”, but my understanding was that the model was trained to never output those details even when requested.