8 ms·

This argument ignores scaling laws

• yo-cuddles 2y ago
  It really doesn't lol. Those laws are like Moore's law, an observation rather than something Fundamental like laws in physics The scaling has been plateauing, and half that equation is quality training data which is totally out at this point. Maybe reasoning models will help produce synthetic data but that's still to be seen. So far the only benefit reasoning seems to bring is fossilizing the models and improving outputs along a narrow band of verifiable answers that you can do RL on to get correct Synthetic data maybe buys you time, but it's one turn of the crank and not much more
  • gdiamos 2y ago
    They are derived from statistical laws unlike moores law https://en.m.wikipedia.org/wiki/Chernoff_bound https://en.m.wikipedia.org/wiki/Chernoff_bound I agree with you that they require data