2 ms·
10^16 -> 10^24 flops isn't 100x, it's 100,000,000x This happened in 10 years (training cost of largest model according to the batshit crazy god AI article in t
by sgt101 4y ago
10^16 -> 10^24 flops isn't 100x, it's 100,000,000x
This happened in 10 years (training cost of largest model according to the batshit crazy god AI article in the FT).
On the one hand compute power increased by about 100x and then some lunatics came and spent 10000x on it...
- jimsimmons 4y agoA lot of it is also backlog: modern GPUs are optimised for these workloads in a way old models weren’t