50 ms·

This is also why machine learning scientists should build their own GPU workstations and servers. Someone did the calculations: https://www.reddit.com/r/Machi

This is also why machine learning scientists should build their own GPU workstations and servers. Someone did the calculations: https://www.reddit.com/r/MachineLearning/comments/9iqcr3/d_why_building_your_own_deep_learning_computer_is/ https://www.reddit.com/r/MachineLearning/comments/9iqcr3/d_w... This guy built a 4 x 2080 ti workhorse for 7,000 USD, and it would probably be even cheaper now: https://l7.curtisnorthcutt.com/the-best-4-gpu-deep-learning-rig https://l7.curtisnorthcutt.com/the-best-4-gpu-deep-learning-...

zests 6y ago
Even with that build it might still take several days to train some deep learning models. Much better would be multiple machine distributed model training. You can pay the true cost either by waiting for your one machine model to train, by setting up complicated infrastructure on your own (either in house or in the cloud) or by using a cloud/third party distributed computing option.
- delfinom 6y ago
  > by setting up complicated infrastructure on your own (either in house or in the cloud) or by using a cloud/third party distributed computing option. The argument is/would be, if your business is heavily based upon deep learning and is that compute heavy on a hourly basis, you may in fact save millions daily by having your own in-house infrastructure. It seriously isn't complicated to run as much as AWS and co want to say it is to push their own sales. But it does cost capital investment and maybe 1 or 2 IT guys. There's an inflection point where it makes sense.
  - zests 6y ago
    What do you expect the salary is for an IT guy that knows how to set up and maintain distributed machine learning infrastructure on prem?
    - bsenftner 6y ago
      Don't fool yourself, it is not hard.
    - manigandham 6y ago
      Not that much. It's really no different than setting up other servers. You can also rent services instead of maintaining a FT position. IT has been commoditized, it's not a special domain of expensive and rare experts anymore.
- wongarsu 6y ago
  For about three times the prices you can get double that performance in one machine (dual socket with 8-10 GPUs). You can get a lot done with that, and in some cases waiting days is actually acceptable. If you need more power than that it does indeed get more complicated and the trade-off might change.

zests 6y ago

Even with that build it might still take several days to train some deep learning models. Much better would be multiple machine distributed model training. You can pay the true cost either by waiting for your one machine model to train, by setting up complicated infrastructure on your own (either in house or in the cloud) or by using a cloud/third party distributed computing option.

delfinom 6y ago
> by setting up complicated infrastructure on your own (either in house or in the cloud) or by using a cloud/third party distributed computing option. The argument is/would be, if your business is heavily based upon deep learning and is that compute heavy on a hourly basis, you may in fact save millions daily by having your own in-house infrastructure. It seriously isn't complicated to run as much as AWS and co want to say it is to push their own sales. But it does cost capital investment and maybe 1 or 2 IT guys. There's an inflection point where it makes sense.
- zests 6y ago
  What do you expect the salary is for an IT guy that knows how to set up and maintain distributed machine learning infrastructure on prem?
  - bsenftner 6y ago
    Don't fool yourself, it is not hard.
  - manigandham 6y ago
    Not that much. It's really no different than setting up other servers. You can also rent services instead of maintaining a FT position. IT has been commoditized, it's not a special domain of expensive and rare experts anymore.
wongarsu 6y ago
For about three times the prices you can get double that performance in one machine (dual socket with 8-10 GPUs). You can get a lot done with that, and in some cases waiting days is actually acceptable. If you need more power than that it does indeed get more complicated and the trade-off might change.