> the most probable continuation based on the training dataset This is not wrong, but it's easy to misread it as implying little more than a glorified Markov m

> the most probable continuation based on the training dataset This is not wrong, but it's easy to misread it as implying little more than a glorified Markov model. If it's like https://www.gwern.net/GPT-3 https://www.gwern.net/GPT-3 then it's already significantly cleverer, and so you should expect to sometimes get the kind of less-blatant derivation that companies aim to avoid using a cleanroom process or otherwise forbidding engineers from reading particular sources.