3 ms·
We got better and better models when we threw more and more compute? I gotta work on my snarkiness. Seriously, that's pretty good empirical evidence. The smalle
by slowtrek 2y ago
We got better and better models when we threw more and more compute? I gotta work on my snarkiness. Seriously, that's pretty good empirical evidence. The smaller models we get are all some kind of distillation or student model of a larger model, so they can never claim they are not the result of large compute.
- reportgunner 2y agoBetter and better how ? Isn't this also based on an overcomplicated web of anecdotes ?
- neverokay 2y agoNo.
- t_mann 2y agoAn empirical study could start by quantifying model quality and available compute and plot them over time, see how they correlate, investigate whether there might be confounding factors,... in essence, putting numbers behind the qualitative statements. I see no reason for snarkiness here.