4 ms·

Would you, as a human, prefer a model that gives you accurate predictions with less data, or a model (that is simpler) but needs more data to be accurate? How

• greendestiny 19y ago
  Well put it this way, to get good predictions without a lot of data takes a whole team of researchers at bell labs working on this for over a year, with data its a student project.
• timr 19y ago
  That's a red herring, and here's why: simpler models take less data to train. Every orthogonal parameter that you add to a statistical model expands the probability space of that model exponentially. If you keep the amount of training data fixed, but keep adding parameters, you'll very quickly arrive at a point where your model will fit your training data exceptionally well, but the result will be meaningless, because you've over-fit your data. Now, I suppose you could make the case that new models are occasionally discovered that allow for more efficient uses of existing data. However, this is a decidedly rare event. Most of the time, gains are made in exactly the way the author suggests -- by adding new sources of data to well-established techniques.