3 ms·

Garbage in -> Garbage out is basically a Newtonian law at this point haha

• momenti 5y ago
  That's not entirely true. Neural networks are fairly robust to noisy training data (a.k.a. garbage).[0] Well, stochastic gradient descent has the noise in its name. More training data can compensate for noisy data to some extent.[1] I'm not sure know if model size can also compensate for noisy data though, but would not be surprised if it did. [0] https://arxiv.org/abs/1705.10694 https://arxiv.org/abs/1705.10694 [1] https://arxiv.org/abs/2202.01994 https://arxiv.org/abs/2202.01994
  • midjji 5y ago
    There are very specific conditions for this to hold, mostly that the incorrect sample is surrounded by correct ones, and that the model is small enough or the error vanishingly rare. Notably the reference you gave also shows horrendous generalization performance, so its really just showing how easy it is to overparametrize. Input errors can be accounted for to some extent, but also under specific circumstances, eg. https://openaccess.thecvf.com/content_CVPR_2020/html/Eldesokey_Uncertainty-Aware_CNNs_for_Depth_Completion_Uncertainty_from_Beginning_to_End_CVPR_2020_paper.html https://openaccess.thecvf.com/content_CVPR_2020/html/Eldesok...
  • Forgeties79 5y ago
    I mean the argument there is basically if there is enough good quality data, the bad data is somewhat (or mostly) compensated for. To which I would argue that is no longer a “garbage in garbage out” situation as most people use it.