The speed "only" really matters for practical reasons, but they're pretty big reasons. In principle, anything you do on a GPU can be done on a CPU or even with

The speed "only" really matters for practical reasons, but they're pretty big reasons. In principle, anything you do on a GPU can be done on a CPU or even with pencil and paper--it's just arithmetic, after all. However, the speedup is so big that it's almost impossible to ignore. One way to measure compute speed is in terms of Floating Point Operations per Second, or FLOPS. A recent-ish CPU is probably ~500 gigaflops, while a single A100 GPU is ~150 teraflops[0]. OpenAI reportedly has a cluster with 10,000 V100 GPUs (the A100's predecessor, but still...). GTP-3 still supposedly took about a month to train on that cluster, so it would never—for all intents and purposes—finish on a MacBook. Few groups operate at that scale, but using even one decent GPU is still such a huge speedup that I doubt many people start with less. It's also worth noting that "training" the model from examples is often a lot more compute-intensive than using it to do "inference." For example, image recognition models are often trained on large clusters, but can be deployed to something much smaller (a phone or laptop). There's a whole subfield of "distillation", which takes large models and finds ways to simplify them for deployment. [0] There's some marketing involved in these numbers, different precisions, and the GPU does work in parallel, so you're not getting one operation every 1/150e12 second.