3 ms·
the scaling laws / bitter lesson would disagree I have to note that taking the "bitter lesson" position as a claim that more data will result in better LLMs is
by joe_the_user 1y ago
the scaling laws / bitter lesson would disagree
I have to note that taking the "bitter lesson" position as a claim that more data will result in better LLMs is a wild misinterpretation (or perhaps a "telephone version) of the original bitter lesson article, which say only that general, scalable algorithms do better than knowledge-carrying, problem-specific algorithms. And the last I heard it was the "scaling hypothesis" that hardly had consensus among those in the field.
- williamtrask 1y agoAgree with you on the nuance.