I agree and I find some of the statements a bit odd (not thought out?). I think Moore’s Law could keep going for decades.[2] But even if it doesn’t... If

I agree and I find some of the statements a bit odd (not thought out?). I think Moore’s Law could keep going for decades.[2] But even if it doesn’t... If 1e35 FLOP is enough to train a transformative AI (henceforth, TAI) system, which seems plausible, I think we could get TAI by 2040... First, I don't see Moore's Law going sub-atomic without a complete change in methodology, which would delay the results. Wafer/die stacking are cool, but stop-gap measures. In particular he acknowledges that power consumption hasn't scaled since 2005 (in fact chiplets help by only sqrt2!). The challenge is that it doesn't help that much to increase the number of transistors, if their latency/power/cost don't fall as well. We're approaching that even ignoring any geopolitical issues. Second, power consumption is not improving except by going to 8&16 bit... there's not a lot of room there. Currently, we get <1000GFlOPs/W and even if we get 100x up to 100TFLOPs/W, you still need 10^17kWhr. OK algorithms get us another 10,000x... and it costs $1T to train 1 transformative AI. How many proof of concepts trials will be do at only $100B a pop? It all just seems a bit flip and hopeful like Feb 2000. The 10^35 number seems like its just pulled out to be a number, when it could orders of magnitude up/dn. https://www.researchgate.net/publication/354573934_Compute_and_Energy_Consumption_Trends_in_Deep_Learning_Inference/figures?lo=1 https://www.researchgate.net/publication/354573934_Compute_a...