4 ms·
I have the same worry about LLMs in general - I know that ‘model collapse’ seems to be an unfashionable idea, but when the internet’s just full of garbage (soon
by jrmg 8mo ago
I have the same worry about LLMs in general - I know that ‘model collapse’ seems to be an unfashionable idea, but when the internet’s just full of garbage (soon?…), what are we going to train these things on?
- tehjoker 8mo agoThey moved away from raw text and are now working with verifiable synthetic data (eg math, games, code) to improve general reasoning.