My (limited) understanding is that 4-8 GPUs will be around 2 orders of magnitude faster than a CPU at training, and that it's not very realistic to use a CPU fo

My (limited) understanding is that 4-8 GPUs will be around 2 orders of magnitude faster than a CPU at training, and that it's not very realistic to use a CPU for training anything but the smallest models.