> I know that the internet is not full of training data for this API because it's a new API. 1) are you sure? That's a bold guess. It was also a really stupid

> I know that the internet is not full of training data for this API because it's a new API. 1) are you sure? That's a bold guess. It was also a really stupid assumption made by the HumanEval benchmark authors. That if you "hand write" simple leet code style questions then you can train on all of GitHub. Go ahead, go look at what kinds of questions are in that benchmark... 2) LLMs aren't discrete databases. They are curve fitting functions. Compression. They work in very very high dimensions. They can generate new data but that is limited. People mostly aren't saying that LLMs can't create novel things but that they can't reason in the way that humans can. Humans can't memorize half of what a LLM can yet are able to figure out lots of crazy shit.