4 ms·
> that the context-awareness is just a very well done illusion Could you expand on this? The context awareness is the part that blows my mind. I would be disap
by jsenn 4y ago
> that the context-awareness is just a very well done illusion
Could you expand on this? The context awareness is the part that blows my mind. I would be disappointed/fascinated to find out that it’s “just” a simple set of tricks!
- nico 4y agoChatGPT is “only” a chat interface to GPT. GPT basically has access to two contexts: its internals, and the prompt that it gets Then when you ask something to ChatGPT, it takes your prompt and it generates a new prompt that includes the previous messages in your session, so that GPT can use them as context. But, there’s a limit to the size of the prompt. And it’s not that big. So then ChatGPT’s magic is figuring out how to crafts prompts, within the size limits, to feed GPT, so that it has enough context to give a good answer. Essentially, ChatGPT is some really amazing prompt-engineering system with a great interface.
- jsenn 4y agoThat makes sense. So ChatGPT has to use some cleverness to present the relevant previous text in the current prompt, but the language model is still responsible for pulling out the meaning of that context and responding appropriately. I remain mind-blown :)
- nico 4y agoExactly. Very eloquently put.