4 ms·
Time to train * flops per chip * number of chips in training fleet You can estimate it quite accurately, actually
by ketzo 3y ago
Time to train * flops per chip * number of chips in training fleet
You can estimate it quite accurately, actually
- jakobson14 3y agoThen you're including all kinds of time for other delays, like copying data over PCIe into the GPU for the next batch, or waiting for the CPU to orchestrate things. You could probably get a better estimate based on the network structure, the number of training epochs, and the size of each batch.