Amazon SageMaker attempts to address this problem. I have also read some google papers about using tens or hundreds of GPUs to solve problems like with alphago.

Amazon SageMaker attempts to address this problem. I have also read some google papers about using tens or hundreds of GPUs to solve problems like with alphago. The issue with most enterprises is that they do not know how to get value out of ML. And they won’t need distributed ML until they can make money with ML. My experience is with pitching cable companies on ML projects.