2 ms·
You are right about "LLM improving themselves" is impossible. In fact, LLM make themselves worse, it is called knowledge collapse [1]. [1] https://arxiv.org/ab
by thesz 9mo ago
You are right about "LLM improving themselves" is impossible. In fact, LLM make themselves worse, it is called knowledge collapse [1].
[1] https://arxiv.org/abs/2404.03502 https://arxiv.org/abs/2404.03502
- int_19h 8mo agoThat paper again. LLMs have been trained on synthetic outputs for quite a while since then and they do get better. Turns out there's more to it than that.