You don't need 10,000 more data to train a model with 10,000 more parameters. You can use the same data and the model will perform better. Much better. The prob

You don't need 10,000 more data to train a model with 10,000 more parameters. You can use the same data and the model will perform better. Much better. The problem is that currently it is nearly impossible to run and train the model with more parameters. ChatGPT has nearly 200 billion parameters. GPT-4 we don't know, there are rumors that it has 100 trillion parameters, but they are probably unfounded. In any case, we've seen how much more powerful GPT-4 is. Imagine a GPT-5 with 1 quadrillion parameters. And then imagine that after you've trained it to some reasonable level, you "downsample" the parameters, using the SVD approach described in LoRA, and get a GPT-5 with the same 200 billion parameters as ChatGPT, but with many, many times more power, even than GPT-4.