4 ms·
What's funny is that most of this "progress" is new datasets + post-training shaping the model's behavior (instruction + preference tuning). There is no moat be
by lossolo 8mo ago
What's funny is that most of this "progress" is new datasets + post-training shaping the model's behavior (instruction + preference tuning). There is no moat besides that.
- Davidzheng 8mo ago"post-training shaping the models behavior" it seems from your wording that you find it not that dramatic. I rather find the fact that RL on novel environments providing steady improvements after base-model an incredibly bullish signal on future AI improvements. I also believe that the capability increase are transferring to other domains (or at least covers enough domains) that it represents a real rise in intelligence in the human sense (when measured in capabilities - not necessarily innate learning ability)
- CuriouslyC 8mo agoWhat evidence do you base your opinions on capability transfer on?
- WarmWash 8mo ago>There is no moat besides that. Compute. Google didn't announce $185 billion in capex to do cataloguing and flash cards.
- causalmodels 8mo agoGoogle didn't buy 30% of Anthropic to starve them of compute
- WarmWash 8mo agoProbably why it's selling them TPUs.
- riku_iki 8mo ago> is new datasets + post-training shaping the model's behavior (instruction + preference tuning). There is no moat besides that. sure, but acquiring/generating/creating/curating so much high quality data is still significant moat.