4 ms·
Prediction #1: Once enough ChatGPT output gets posted online, it will inevitably find its way into the training corpus. When that happens, ChatGPT becomes state
by andyjohnson0 4y ago
Prediction #1: Once enough ChatGPT output gets posted online, it will inevitably find its way into the training corpus. When that happens, ChatGPT becomes stateful and develops episodic memory.
Prediction #2: As more people discuss ChatGPT online, by late 2023 discussion of Roko's Basilisk exceeds discussion of ChatGPT. (half /s)
- ClumsyPilot 4y agoOr. ChatGPT will overtrain on it's own data and go to shit the way google search did
- visarga 4y agoTraining on its own data is a tradition already. For example RLHF example pairs rated by humans are generated by the model. So even our best models trained on their own outputs + rating from human labellers. The internet is a huge rating machine, AI will distill this signal and improve even while ingesting its own text.
- andyjohnson0 4y agoMeta-ChatGPT's loss function optimises for ChatGPT generating training data that maximises the shittyness of Google's LLM.
- LelouBil 4y agoDid you see the new Bing chat ? #1 is already happening ! See here (other HN thread) : https://twitter.com/tobyordoxford/status/1627414519784910849 https://twitter.com/tobyordoxford/status/1627414519784910849