There is going to be a huge shift in scientific practice as a result of the success of GPT-3 and other similar breakthroughs in NLP. The methodology that produc

There is going to be a huge shift in scientific practice as a result of the success of GPT-3 and other similar breakthroughs in NLP. The methodology that produced GPT-3 is going to work for many other domains, not just text. The process is basically this: 1. Compile a huge dataset of observations relevant to your domain 2. Using the cross-entropy loss or some close variant, train a raw data statistical model of the observations 3. Make the model immensely complex (GPT-3 has 175 billion parameters); this can be done without overfitting because of the huge size of the dataset. 4. Once the raw data model is available, reuse it for various related tasks of practical interest. The immense sophistication of the raw data model will make the practical task much easier. GPT-3 demonstrates clearly that this methodology works, and produces far more powerful results than other more mainstream approaches in NLP that depend on laborious human annotation. In my view it is nearly certain to produce comparably dramatic results in other domains (I'd be happy to bet on this if anyone is interested). Here's a longer essay on this topic: http://danburfoot.net/research/RediscoveryInteriority.html http://danburfoot.net/research/RediscoveryInteriority.html