3 ms·
Couldn’t we architect or train the models to differentiate between streams of input? It’s a current design choice for all tokens to be the same. Think of huma
by dwaltrip 3y ago
Couldn’t we architect or train the models to differentiate between streams of input? It’s a current design choice for all tokens to be the same.
Think of humans. Any sensory input we receive is continuously and automatically contextualized alongside all other simultaneous sensory inputs. I don’t consider words spoken to me by person A to be the same as those of person B.
I believe there’s a little bit of this already with the system prompt in ChatGPT?
- not2b 3y agoPossibly there's a way to do that. Right now, LLMs aren't architected that way. And no, ChatGPT doesn't do that. The system prompt comes first, hidden from the user and preceding the user input but in the same stream, and there's lots of training and feedback, but all they are doing is making it more difficult for later input to override the system prompt, it's still possible, as has been shown repeatedly.
- dragonwriter 3y ago> Couldn’t we architect or train the models to differentiate between streams of input? Could you? absolutely. Would it solve this problem? Maybe. Would it make training LLMs to do useful tasks much harder and vastly increase the volume of training data necessary? For sure. > I believe there’s a little bit of this already with the system prompt in ChatGPT? Probably not. Likely, both the controllable "system prompt" you can change via the API and probably any hidden system prompt is part of the same prompt as the rest of the prompt, though its deliminited by some token sequence when fed to the model (chat-tuned public LLMs also do this, with different delimiting patterns.)