3 ms·
Idk about end of the road, I’m sure they can squeeze out some more performance by curating even more data and doing even more RL. But I would bet that pretty m
by slopinthebag 3mo ago
Idk about end of the road, I’m sure they can squeeze out some more performance by curating even more data and doing even more RL.
But I would bet that pretty much all of the improvement we’ve seen over the last year with coding has come from RL, not from the models becoming particularly stronger. And this makes sense, if models grow sublinearly with compute. And it seems like they do.
- gr_norm 3mo agoIt seems pretty obvious from the steep 'intelligence' drop-off on out-of-distribution tasks that the performance improvement is from throwing untold tens of billions at RL. There are legions of highly skilled people employed solely to feed the RL loop. Evidently effective, but there's an unmistakable feeling this won't ultimately be the way forward.
- freeone3000 3mo agoWell, why not? Won’t it get “good enough” at every task eventually?
- slopinthebag 3mo agoWhy would that be the default assumption?
- freeone3000 3mo agoBecause they’ve gotten good enough at lots of other things, and the RL keeps improving them, so enough RL should make them good enough at the focus areas.
- slopinthebag 3mo agoI've gotten pretty strong in the gym, my bench has improved to two plates. I see no reason why it won't continue to improve until I can bench my house.
- freeone3000 3mo agoI think it’s a bit closer than that. You can bench two plates, and “good enough” is two plates on the bench and a three plate squat. LLMs haven’t started leg day.