4 ms·

Just Ask for Generalization (2021)

• xg15 1y ago
  (2021), still very interesting. Especially the "post-overfitting" training strategy is unexpected.
  • luckystarr 1y ago
    I remember vaguely that this was observed when training GPT-3 (probably?) as well. Just trained on and on, and the error went up and then down again. Like a phase transition in the model.
  • dev_hugepages 1y ago
    This is talking about the double descent phenomenon (https://en.wikipedia.org/wiki/Double_descent https://en.wikipedia.org/wiki/Double_descent)
• deleted 1y ago
  [deleted]
• esafak 1y ago
  The low sample efficiency of RL is well explained.