3 ms·
If someone showed you the progression of GPT-2 -> GPT-3, and then informed you that they were going to train a new model with 100x the compute and tons more hig
by skepticATX 3y ago
If someone showed you the progression of GPT-2 -> GPT-3, and then informed you that they were going to train a new model with 100x the compute and tons more high quality data, what would you expect this model to be able to do? And how would it compare to the GPT-4 that we actually got?
I think that while GPT-4 is very impressive and a useful tool, seeing the jump, or lack thereof in the places that I expected, between GPT-3 and GPT-4 resulted in me lengthening my timeline.