To me, we're obviously in a bubble. Everyone assumes that it's going to continue to take economically disrupting amounts of compute to train larger and larger m

To me, we're obviously in a bubble. Everyone assumes that it's going to continue to take economically disrupting amounts of compute to train larger and larger models into the foreseeable future. They think that the need for compute will always go up. Just like they thought that the price of real estate would always go up, until it didn't in 2006 or so. We're in for a correction, a strong correction, in hardware demand. Someone will figure out a better algorithm, a better way to do transfer learning, or maybe a better architecture for AI chips that greatly increases performance. At the rate things are going, we'll be using 2 bit arithmetic (0,1,infinity,-1) for models and all of the huge multiply-accumulate arrays won't be quite so useful. I don't see how "line goes up" continues forever.