3 ms·
ChatGPT remembers the things it was trained on, and things that came up in the last 2048 tokens of dialog, which is about 600 words. (2048 is the GPT-3 size and
by CGamesPlay 4y ago
ChatGPT remembers the things it was trained on, and things that came up in the last 2048 tokens of dialog, which is about 600 words. (2048 is the GPT-3 size and I'm only guessing that ChatGPT uses the same size.)
- baq 4y agoI've seen reports that it's actually 8192, which is allegedly the major improvement over GPT-3. Obviously I've no way to confirm, but it's plausible (given the article...)