Here's the link about neural scaling law: https://en.wikipedia.org/wiki/Neural_scaling_law https://en.wikipedia.org/wiki/Neural_scaling_law Can you make a napk

Here's the link about neural scaling law: https://en.wikipedia.org/wiki/Neural_scaling_law https://en.wikipedia.org/wiki/Neural_scaling_law Can you make a napkin calculation of how much better training should be, how larger models and datasets should be to overcome difference in widely separated but related pixels of the image? I did that and results are not promising: 200 billions parameter model will not perform much better than 100 billions' one. The loss is already too small. Also, the phenomena exemplified in the article is a problem of relation between distant entities in generated media. The same can be seen in LLM where they have inconsistencies at the beginning and end of sentences. If generated eyes start to lie, there will be different separated but related objects to rely on.