The current training data is not the full internet worth because most of the internet is garbage data. It is removed for a reason. But sure, those 200T tokens o

The current training data is not the full internet worth because most of the internet is garbage data. It is removed for a reason. But sure, those 200T tokens of Facebook user data is the answer to smarter LLMs.