Id say we are still very much on the flat part and going to be there at sometime. ChatGPT is just a really, really, REALLY good search. Things that used to tak

Id say we are still very much on the flat part and going to be there at sometime. ChatGPT is just a really, really, REALLY good search. Things that used to take multiple google searches now take one query. This will definitely disrupt quite a few things and render some obsolete, but generally the effect will just be making things more efficient. There are three things IMO that is going to make us start shooting up the final part of S curve. First is efficient on the fly learning. If you think about how humans learn, we don't retrain ourselves every time we recieve a new piece of information that doesn't go contrary to what we know - a neural network however needs to be iteratively trained on it, with weight changes in most every single one of its parameters. We need models like Chat GPT to be presented with new text, and not only "remember" it, but also contextualize it. Second is the contextualization mechanic itself. Its well known that generative models are statistical at their core, ChatGPT is wrong in a lot of cases. There needs to be some way to encode truth into the model. Third is applying neural nets towards generating neural net architectures (or more specifically optimized compute graphs). The tricky part here is to determine what the loss function is (you wouldn't want to generate a net, train said net, evaluate its performance, then feed that back into the original generative net).