4 ms·
I don't understand why this is significant. The context is the whole conversation. If you fill that context with examples of ChatGPT functionality it seems kind
by morsecodist 3y ago
I don't understand why this is significant. The context is the whole conversation. If you fill that context with examples of ChatGPT functionality it seems kind of obvious to me that the responses to the "who are you?" question would bias towards the examples in the context.
This doesn't really seem like it is learning through the course of the iterations, just selecting different aspects of information it already has based on the way the iterations go. For example if you take one of the new elements added to the iteration 20 response: it adds "generating creative content" to it's list of things it can do to assist. If I ask a fresh model "Is part of your purpose to generate creative content?" it answers yes. Same for the other new additions. It doesn't need the examples to know what it can do.
- andywalters 3y agoHi, author here, I do think it's a fair criticism to say the mechanism could be more clearly demonstrated if it formed and evaluated hypotheses clearly outside the domain of any likely RLHF / fine tunes / system prompt text. I'll work on that.
- intended 3y agoI would highly recommend aiming to disprove this, as opposed to proving this. HN once recommended blindsight as an interesting novel, years ago and the antagonist has been a great tool to approach LLMs reasoning.