3 ms·
I certainly didn't mean literally the gherkin example. It was a fictitious example of the kind of question which has had surprising results. If you have high co
by ggm 2y ago
I certainly didn't mean literally the gherkin example. It was a fictitious example of the kind of question which has had surprising results. If you have high confidence this either cannot happen, or hasn't happened to you then for your question forms this is good. I am concerned that having seen it, even if I now cannot recreate it, the risk exists.
- cma 2y agoHere's a better example: a common riddle they are trained on, but the prompter changed the genders to something that doesn't puzzle people and write models still assume it is a puzzle, presumably from text statistics, and get it wrong. https://www.reddit.com/r/singularity/comments/1hek6ai/everyone_says_theres_something_different_about/ https://www.reddit.com/r/singularity/comments/1hek6ai/everyo... But Claude nails it. There is a lot more than base LLM training involved since RLHF and other techniques were added, but there are still problems like sycophancy and these kind of crafted misdirections that aren't misdirecrions. Or Claude might have just been trained on someone else doing the same trick to show limitations of LLMs, the internet is vast and I've seen similar experiments it might have seen since it is more recent.