3 ms·
When The Bitter Lesson essay came out it was a bunch of important things: addressing an audience of serious practitioners, contrarian and challenging entrenched
by benreesman 1y ago
When The Bitter Lesson essay came out it was a bunch of important things: addressing an audience of serious practitioners, contrarian and challenging entrenched dogma, written without any serious reputational or (especially) financial stake in the outcome. It needed saying and it was important.
But its become a lazy crutch for a bunch of people who meet none of those criteria and perverted into a statement more along the lines of "LLMs trained on NVIDIA cards by one of a handful of US companies are guaranteed to outperform every other approach from here to the Singularity".
Nope. Not at all guaranteed, and at the moment? Not even looking likely.
It will have other stuff in it. Maybe that's prediction in representation space like JEPA, maybe its MCTS like Alpha*, maybe its some totally new thing.
And maybe it happens in Hangzhou.