4 ms·
Are we ignoring that ChatGPT is stuck in the past and that continuously retraining it on current data is currently still prohibitively expensive?
by counttheforks 4y ago
Are we ignoring that ChatGPT is stuck in the past and that continuously retraining it on current data is currently still prohibitively expensive?
- mattnewton 4y agoYes, because this is a solvable problem probably easier than figuring out how to make it a useful product (like knowing what it doesn’t know and preventing convincing hallucinations in some scenarios while maintaining the ability to hallucinate for creative queries). There is a lot of work on approaches like using search retrieval methods like embedding similarity lookups, and then pasting summaries of the retrieved documents in as context. This allows the LLM to cite new data without retraining. (That’s also something google is very good at)