Generally, all that non-tech content still helps the model “to learn”. Also, the software you’re working on will generally in some way have a real-world domain

Generally, all that non-tech content still helps the model “to learn”. Also, the software you’re working on will generally in some way have a real-world domain - without knowing it the AI all likely be a less effective assistant. Design conversations with it would likely be pretty non-fun, too. Finally, the “bitter lesson” article[0] from a couple years ago is I think somewhat applicable too. [0]: http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html