3 ms·
> the sleeping giant, Google has woken up, and they are iterating on a pace that will smash GPT-4 total pre-training FLOPS by 5x before the end of the year. Op
by nomilk 3y ago
> the sleeping giant, Google has woken up, and they are iterating on a pace that will smash GPT-4 total pre-training FLOPS by 5x before the end of the year.
OpenAI are not standing still though. Is there an established (or estimated) Moore's Law for LLMs?
- toth 3y agoLLM growth (or frontier ML model growth in general) is driven in part by the original Moore's Law, but right now the main driving factor is the growth of investment. I.e., how much companies are prepared to spend on training run. This has been growing at more than 10x per year.