3 ms·
This assumes that our processes and algorithms don't get more targeted or improve. The rate of new approach discovery is staggering. For every problem, some com
by gibolt 5y ago
This assumes that our processes and algorithms don't get more targeted or improve. The rate of new approach discovery is staggering. For every problem, some combination of approaches will more efficiently pre-process and understand the training data.
The article also ignores training vs running tradeoffs. Training a model once may be extremely resource intensive, but running the resulting model on millions of devices can be negligible while having huge value add.
- culi 5y agoKeep reading, the article directly addresses this point
- sayonaraman 5y ago> new approach discovery a good example is discovery of attention mechanisms/transformers replacing more cumbersome and computationally expensive RNNs and LSTMs in NLP and more recently outperforming more expensive models in computer vision.
- visarga 5y agoTransformers are pretty huge and expensive to run, LSTMs are lighthweight by comparison.
- et1337 5y agoKeep reading, the article includes a whole section about training vs running tradeoffs.