3 ms·
I think there's practical and stylistic angles here. Practically, "chat" instruction fine-tuning is really compelling. GPT-2 demonstrated in-context learning a
by tel 3y ago
I think there's practical and stylistic angles here.
Practically, "chat" instruction fine-tuning is really compelling. GPT-2 demonstrated in-context learning and emergent behaviors, but they were tricky to see and not entirely compelling. An "AI intelligence that talks to you" is immediately compelling to human beings and made ChatGPT (the first chat-tuned GPT) immensely popular.
Practically, the idea of a system prompt is nice because it ought to act with greater strength of suggestion than mere user prompting. It also exists to guide scenarios where you might want to fix a system prompt (and thus the core rules of engagement for the AI) and then allow someone else to offer {:user} prompts.
Practically, it's all just convenience and product concerns. And it's mechanized purely through fine-tuning.
Stylistically, you're dead on: we're making explicit choices to anthropomorphize the AI. Why? Presumably, because it makes for a more compelling product when offered to humans.
- jameshart 3y agoI think we’re doing more than makign a stylistic choice. I think we’re relying on - and guiding - an ability in an LLM to effectively conjure a ‘theory of mind’ for a helpful beneficent ai chatbot.
- tel 3y agoI think that anthropomorphizes the LLM quite a lot. I don't disagree with it, I truly don't know where to draw the line and maybe nobody does yet, but to myself at least I caution the idea of whether or not us using language evocative of the AI as being conscious actually imposes any level of consciousness. At some level, as people keep saying, it's just statistics. Per Chris Olah's work, it's some level of fuzzy induction/attention head repeating plausible things from the context. The "interesting" test that I keep hearing, and agreeing with, is to somehow strip all of the training data of any notion of "consciousness" anywhere in the text, train the model, and then attempt to see if it begins to discuss consciousness/self de novo. It's be hard to believe that experiment could be actualized, but if it were and the AI still could emulate self-discussion... then we'd be seeing something really interesting/concerning.