4 ms·
I assumed the ChatGPT3 enhancement is that they added memory of the dialogue to it? That's a huge improvement over one off questions and answers. You can tell i
by Nathanba 4y ago
I assumed the ChatGPT3 enhancement is that they added memory of the dialogue to it? That's a huge improvement over one off questions and answers. You can tell it to correct its answers and ask follow up questions.
- DjiboutiJon 4y agoI wonder myself if the "memory" is just inputting the previous dialogue as input to every subsequent query. I think this would probably get quite slow after awhile however.
- stolsvik 4y agoThis is what it does. At least if you ask it how it does it. ;)
- Vetch 4y agoProbably, they're still using relatively simple queries. Maybe the memory is larger, but with 4096 tokens, GPT-3 is no slouch either. RLHF has tuned it so the conversational output of GPT is more inline with human expectation, it's also less likely to make glaring errors. The cost of their approach is it sometimes get stuck rephrasing things and wraps everything in boilerplate that can derail subsequent generation.