3 ms·
Not necessarily if you are pushing against a data wall. One could ask: after adjusting for DS efficiency gains how much more compute has OpenAI spent? Is their
by fspeech 2y ago
Not necessarily if you are pushing against a data wall. One could ask: after adjusting for DS efficiency gains how much more compute has OpenAI spent? Is their model correspondingly better? Or even DS could easily afford more than $6 million in compute but why didn't they just push the scaling？
- whimsicalism 2y agoright except that r1 is demoing the path of approach for moving beyond the data wall
- breadwinner 2y agoCan you clarify? How are they able to move beyond the data wall?
- whimsicalism 2y agobecause they’re able to pass signal on tons of newly generated tokens based on whether they result in a correct answer, rather than just fitting on existing tokens. it’s on the path to self play