GPT-3 does only thing, predict the next character based on the previous n characters. The nuance of the context (previous n characters) and the quality of the

GPT-3 does only thing, predict the next character based on the previous n characters. The nuance of the context (previous n characters) and the quality of the prediction(s) (following character, a function of gpt-3 you can repeat by just concatenating its last prediction to the previous n characters) is what has significantly improved. It has now learned on a significant amount of the writing available on the internet. Which includes guides on writing code, stack overflow, and github repos. So one is able to write something that looks like a python comment // This function adds two numbers And the following n character predictions of gpt-3 will with some unreliability spit out the python code for that function as its predictions def add(x, y): return x+y This has been shown to work with simple descriptions of pages, with it returning html with css embedded that matches that description.