2 ms·
One of the lessons I'm learning in my work right now is: Anyone can throw a mess of features into a huge hog of a model and get some sufficiently accurate inpu
by uoaei 4y ago
One of the lessons I'm learning in my work right now is:
Anyone can throw a mess of features into a huge hog of a model and get some sufficiently accurate input. But with feature engineering, you essentially absolve the computational model from needing the vast majority of the capacity in the naive case, instead "pre-computing" those transforms which would otherwise be learned by the model. With the reduced need for capacity, the ability to refine the model for interpretability's sake (and optimization stability) also improves.
Of course part of success is also collecting the right data, but focusing on the right features is a slightly more nuanced and focused way to look at it IMO.
- sumy23 4y agoThis doesn’t seem like good advice. As a human, the data might not be engineerable by you. The data might seem like it is, but you may end up losing information through your feature engineering. The combinatorial power of NNs is so great that it’s almost always better to give the model as much data as possible and let it figure out which parts of the data are important.
- uoaei 4y agoYou're right, my advice may be biased. I work on physical systems operating under tight control and near equilibrium most of the time.