Especially as we're not training the LLMs on really specialized stuff Internet content does not go as deep as it would ideally be in some areas. For example,

Especially as we're not training the LLMs on really specialized stuff Internet content does not go as deep as it would ideally be in some areas. For example, do training sets contain ArXiv data? Or other stuff behind a paywall? LLMs know very little about XX century literature.