3 ms·
Yes that's the intuition, but there's an idea called 'the bitter lesson' by Richard Sutton where every 'feature engineering' is swamped and overtaken by the raw
by ftxbro 3y ago
Yes that's the intuition, but there's an idea called 'the bitter lesson' by Richard Sutton where every 'feature engineering' is swamped and overtaken by the raw power of stacking more layers with increased parameters and exaflops and dataset sizes.
- mjburgess 3y agoAll that's really happening there is a brute-force search for engineered features. Such brute-forcing has a very very low ROI when the necessarily huge datasets don't already exist. Indeed, creating the experimental conditions to collect that data often means engaging in some form of imagined feature engineering.