4 ms·
I disagree. With more computation you can train a bigger model on the same size training data and it will be better. There is a lot if knowledge on the internet
by golol 2y ago
I disagree. With more computation you can train a bigger model on the same size training data and it will be better. There is a lot if knowledge on the internet that GPT-4 etc. have not yet learned.
- Retric 2y agoThe issue is the training data isn’t some constant. Let’s suppose OpenAI had 10x the computing power but a vastly worse dataset, do you expect a better or worse result? The question is ambiguous without defining how much worse the dataset is.
- golol 2y agoBut why would the dataset be worse, they can just use the same one as before.
- Retric 2y agoA dataset that’s 2 years old is worse than one that’s 20 years old even if it contains the same data. Even facts age. In 2004, Pluto was still classified as a planet. Not such a big deal on its own, but stale data gets a little bit worse every day.