The resources devoted to training don't really translate to the resources needed to run the trained model. A model trained on a thousand GPUS for a week might s

The resources devoted to training don't really translate to the resources needed to run the trained model. A model trained on a thousand GPUS for a week might still run in real time on a single GPU once trained. The restricted training resources are just part of the challenge. They point out that a human child can learn the necessary steps in minutes by watching someone else do it, so they wanted to see if anyone could make a computer learn it with relatively limited resources.