With big generative models, seeing data even once is more than sufficient to memorize it. So your claim that performance relates to frequency is not exactly cor

With big generative models, seeing data even once is more than sufficient to memorize it. So your claim that performance relates to frequency is not exactly correct. The whole point of this model class is that one can learn one word from one sample, another pixel from another one and so on to master the domain. The emergent, non-trivial generalization is what makes them so fascinating. There is no simple, linear/first order relationship with data and behaviour. Case in point: GPT3 can do few-shot learning despite not having used any explicit few-shot formatted data during training. Not saying you are wrong, but the story is not as simple as simple supervised learning with small datasets