3 ms·

And yet people here have no trouble crying about electricity wastage of crypto. Also from my limited knowledge I think DNN models are not very transferable in

And yet people here have no trouble crying about electricity wastage of crypto. Also from my limited knowledge I think DNN models are not very transferable in real world setting requiring constant retraining even for a small drift in signal or change in noise modes.

nerdponx 5y ago
> And yet people here have no trouble crying about electricity wastage of crypto Which is many orders of magnitude more energy-intensive, on the scale of a small nation-state, and in most cases fundamentally wasteful by design. A very large pre-trained model can be reused very cheaply once it's finished. > Also from my limited knowledge I think DNN models are not very transferable in real world setting requiring constant retraining even for a small drift in signal or change in noise modes. This is FUD, promulgated by people who expected deep learning to solve all their problems overnight. All models will suffer from "drift" whenever the underlying data changes. Part of what made deep learning so good was that it was able to generalize exceptionally well from exceptionally complicated input data. It is unreasonable to expect that a model pre-trained on a huge generic corpus will be a perfect match for your very specific business problem. However it is _not_ unreasonable to expect that said model will be a useful baseline and starting point for your very specific business problem. We are not yet (and might never be) at the point where you can dump a pile of garbage data into an API and get great predictions out the other end on the first try. But nobody ever thought you could do that, except the people selling expensive subscriptions to those kinds of APIs. The fact that they work at all should be taken as evidence of how amazing deep learning is; the fact that they don't work perfectly should not be taken as evidence that deep learning is bad/useless/wasteful/hype/whatever. Don't let the clueless tech media set your expectations. Professional data scientists and machine learning practitioners for the most part take their work very seriously and take pride in delivering good outcomes, just like professional software engineers. If deep learning wasn't useful to that end, nobody would be using it.
- sanxiyn 5y ago
  Facebook's News Feed model was trained daily in 2018. (Source: https://research.fb.com/wp-content/uploads/2017/12/hpca-2018-facebook.pdf https://research.fb.com/wp-content/uploads/2017/12/hpca-2018.... If you know of update please let us know.) "requiring constant retraining" is pretty accurate in this case. Facebook trained image understanding CNN ("Lumos" in the paper) only once in many months, but this is not usual.

nerdponx 5y ago

> And yet people here have no trouble crying about electricity wastage of crypto Which is many orders of magnitude more energy-intensive, on the scale of a small nation-state, and in most cases fundamentally wasteful by design. A very large pre-trained model can be reused very cheaply once it's finished. > Also from my limited knowledge I think DNN models are not very transferable in real world setting requiring constant retraining even for a small drift in signal or change in noise modes. This is FUD, promulgated by people who expected deep learning to solve all their problems overnight. All models will suffer from "drift" whenever the underlying data changes. Part of what made deep learning so good was that it was able to generalize exceptionally well from exceptionally complicated input data. It is unreasonable to expect that a model pre-trained on a huge generic corpus will be a perfect match for your very specific business problem. However it is _not_ unreasonable to expect that said model will be a useful baseline and starting point for your very specific business problem. We are not yet (and might never be) at the point where you can dump a pile of garbage data into an API and get great predictions out the other end on the first try. But nobody ever thought you could do that, except the people selling expensive subscriptions to those kinds of APIs. The fact that they work at all should be taken as evidence of how amazing deep learning is; the fact that they don't work perfectly should not be taken as evidence that deep learning is bad/useless/wasteful/hype/whatever. Don't let the clueless tech media set your expectations. Professional data scientists and machine learning practitioners for the most part take their work very seriously and take pride in delivering good outcomes, just like professional software engineers. If deep learning wasn't useful to that end, nobody would be using it.

sanxiyn 5y ago
Facebook's News Feed model was trained daily in 2018. (Source: https://research.fb.com/wp-content/uploads/2017/12/hpca-2018-facebook.pdf https://research.fb.com/wp-content/uploads/2017/12/hpca-2018.... If you know of update please let us know.) "requiring constant retraining" is pretty accurate in this case. Facebook trained image understanding CNN ("Lumos" in the paper) only once in many months, but this is not usual.