It seems extremely unlikely that the total energy cost of GPT models is dominated by training. The training of GPT-3 was estimated at 1287 MW-h, which is less t

It seems extremely unlikely that the total energy cost of GPT models is dominated by training. The training of GPT-3 was estimated at 1287 MW-h, which is less than 1 day of a medium-sized cloud datacenter.