- Throwing out the bad parts, or updating the good parts with better responses. (There's a few training programming data sets which are not great) - Self impro

- Throwing out the bad parts, or updating the good parts with better responses. (There's a few training programming data sets which are not great) - Self improvement without new data, like STaR https://arxiv.org/abs/2203.14465 https://arxiv.org/abs/2203.14465 - architecture changes to support better generalisation - humans learn a lot without consuming as much content.