4 ms·
and anyone notice that the pace has broken xAI and they were just dropped behind? The frontier improvement release loop is now ant -> openai -> google
by nikcub 8mo ago
and anyone notice that the pace has broken xAI and they were just dropped behind? The frontier improvement release loop is now ant -> openai -> google
- dist-epoch 8mo agoMusk said Grok 5 is currently being trained, and it has 7 trillion params (Grok 4 had 3)
- svara 8mo agoMy understanding is that all recent gains are from post training and no one (publicly) knows how much scaling pretraining will still help at this point. Happy to learn more about this if anyone has more information.
- dist-epoch 8mo agoYou gain more benefit spending compute on post-training than on pre-training. But scaling pre-training is still worth it if you can afford it.
- Squarex 8mo agoI still remember gemini 1.5 ultra and gpt 4.5 as extremely strong at some areas that no benchmark capture. It was probably not economical to use them at 20 usd subscription, but they felt differently and smarter at some ways. The benchmarks seems to be missing something, because flash 3 was very close on some benchmarks to 3 pro, but much, much dumber.
- gavinray 8mo agoxAI just released Grok 4.20 beta yesterday or day before?