This is not really true. ML applications in general do not scale linearly. There is the systems (scaling) overhead, and then there are the algorithm payoffs, wh

This is not really true. ML applications in general do not scale linearly. There is the systems (scaling) overhead, and then there are the algorithm payoffs, which start to diminish depending on the algorithm. If they went from 4 to 8 GPUs, the only thing they can guarantee to double is how fast they burn power.