5 ms·

The quantity of information is probably higher though :)

• hef19898 3y ago
  Agreed, not that anyone really seems to care so.
• Wowfunhappy 3y ago
  Of course, but for training data current LLMs seem to need quantity above all else.
  • bee_rider 3y ago
    I’m sure it depends on the type of prediction task, right? Current LLMs are trying to predict typical human prose from samples pulled from the internet. So it isn’t as if they are sacrificing quality for quantity. A bunch of text from the internet is a very good representation of typical human prose. Whether it is well written or the descriptions contained in the prose accurately represent, like, actual physical reality is another issue. Maybe they want to predict something with, like, less dimensionality but more utility than a paragraph of fiction.