The number of tokens in the training set is not a good measure of the information content of the training set - if it was then the training would not work as th

The number of tokens in the training set is not a good measure of the information content of the training set - if it was then the training would not work as the training set would be statistically random. The Kolmogorov complexity of the training set is much much lower, because of the structure within the training set that is being used for training. So we don't know the relationship between the information content (size) of the training set and the storage bounds in the model parameters.