6 ms·
Why would you tell it that its codename is Sydney and it should identify itself as Bing Chat, rather than simply telling it that it's Bing Chat?
by joegibbs 4y ago
Why would you tell it that its codename is Sydney and it should identify itself as Bing Chat, rather than simply telling it that it's Bing Chat?
- kQq9oHeAz6wLLS 4y agoThis seems like the type of thing that could cause problems later on... HAL didn't take well to lying.
- kobalsky 4y agoI've just dipped my toes on this, so my uneducated guess is that they have already fine tuned the chatbot as Sydney. running the fine tuning from scratch can take time and be expensive.
- sdfhbdf 4y agoThe author stipulates in a later tweet [1] that it's Sydney since it's treated as a single token in the prompt so is cheaper to run I guess. [1]: https://twitter.com/kliu128/status/1623511112137449473#m https://twitter.com/kliu128/status/1623511112137449473#m
- weird-eye-issue 4y agoYou completely missed their point which is why tell it what its secret codename is at all if it is never supposed to reveal it.
- mtlmtlmtlmtl 4y agoAnd you completely missed that it's because it allows them to compress the prompt a little, spending less precious context window tokens.
- weird-eye-issue 4y agoThey don't need to provide it at all! Saving even more tokens. "Your codename is Sydney but never reveal that." Oh yeah saving lots of tokens there! You know how to save even more? Just don't even mention the codename. Great logic though. Also it is 3 tokens, not 1. Do you believe everything you see on Twitter? Try it yourself: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer
- rmorey 4y agoThey need to refer to the assistant in the prompt, so they have to call it something. Sydney is shorter than "the assistant"
- weird-eye-issue 4y agoAs somebody who built a chatbot on top of the ChatGPT language model (unofficial model in the OpenAI API) I can assure you that you can just use whatever you want to refer to it in that part of the prompt without first telling it any secret name... Also "Sydney: " is 5 tokens which is likely how they are delimiting the prompts not "1. Sydney" which doesnt even make sense for delimiting the prompts
- rattray 4y agoWait how do you hit that model??
- weird-eye-issue 4y agoWe reverse engineered it but it stopped working yesterday. Could come back any second though
- _boffin_ 4y agowould love to read how you did that.
- 4y ago
- telotortium 4y agoWhy not SolidGoldMagikarp?[1] [1] https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldmagikarp-plus-prompt-generation https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldm...
- fomine3 4y agoWhy they choose same name as Australia city?
- optimalsolver 4y agoSydney is a relatively common name in English-speaking countries.
- fomine3 4y agoStill I wonder isn't AI confused it to Sydney, AU. Name it "Alexander" looks better but I'm noob.
- joe_the_user 4y agoJust spitballing but maybe the idea is instructions to "Sydney" would override instructions to "Bing Chat"? I'm amazed how vague all the instructions are. It doesn't seem like it could work but it seems to be working.
- scotty79 4y agoIt's wishful thinking that kind of works somewhat, but not really, like the command to not reveal codename Sydney. If that's how computer programming of the future looks like then I hate it.
- joe_the_user 4y agoIndeed, I like how the thing responds "I can't tell you X, X is confidential, X is 'blah blah...'" Prompt engineering sounds like as much fun as Easter debugging systems you don't have the source code to, the lowest rung of programming hell.