2 ms·
The current machine learning paradigms are hard to distribute efficiently - they can be parallelized but require significant ongoing communication between nodes
by PeterisP 9y ago
The current machine learning paradigms are hard to distribute efficiently - they can be parallelized but require significant ongoing communication between nodes, you can't really split a problem into separate subtasks and merge them only in the end, you need to transfer all the calculated values after, say, each iteration.
I'd guess that having ten extra machines in the same rack would be more valuable than a thousand remote machines with limited network bandwidth.