4 ms·

This says nothing about data quality.

• Salgat 3y ago
  To add, if you have a model that's too large for the dataset, you end up in a situation where it just memorizes the training dataset and performs poorly on the test data. That's why GPT works so good, you only pass the training data through it once, and there's so much good training data that it works in spite of the model being absolutely massive.
  • whimsicalism 3y ago
    it only memorizes the dataset if you give it multiple passes and are bad at regularizing overparameterization is fine