3 ms·
Right, but GPT-2 had 1.5 billion neurons in 2019 so GPT-3 was a 100x increase. 1e15 may not be that far off, especially since these operations are relatively s
by aeternum 6y ago
Right, but GPT-2 had 1.5 billion neurons in 2019 so GPT-3 was a 100x increase. 1e15 may not be that far off, especially since these operations are relatively straightforward to run in parallel.