> There's lots of research on doing the same learning with much less resources (e.g. recent paper https://eng.uber.com/accelerated-neuroevolution/ https://eng.

> There's lots of research on doing the same learning with much less resources (e.g. recent paper https://eng.uber.com/accelerated-neuroevolution/ https://eng.uber.com/accelerated-neuroevolution/ , or the example visible in this very article of AlphaZero having much, much less compute than AlphaGoZero and doing better anyways), and even without that AlphaZero could not have been created without going through many many iterations of AlphaGo, each one of which cost several GPU-years, and calling AlphaZero cheap is serious moving of goalposts as it required thousands of TPUs for days, and Facebook's recent replication for chess also used thousands of GPUs for 3 weeks. (Zero is cheap only in comparison to the previous AlphaGos using weeks or months of hundreds/thousands of GPUs/TPUs.) Note that that is a log graph; flip over the linear scale and you get an idea of how extraordinarily expensive Zero is compared to everything not named 'AlphaGo'. If anything, this observation implies that AI risk is more dangerous than thought because it implies a 'hardware overhang': it will take vast computational resources to create the first slow inefficient AI but it will then rapidly be optimized (either by itself or human researchers) and able to run far faster/more copies/on more devices/for less money, experiencing a sudden burst in capabilities. Like model compression/distillation where you can take the slow big model you normally train and then turn it into something which is 10x faster or 100x smaller or just plain performs better (see 'born again networks' or ensembling). > simple hardware progress means that random gaming GPUs can handle datasets that were inconvenient a few years ago. ...which means using a lot more compute, yes.