3 ms·

> has me wondering: are we seeing the limits of current "just scale the data" approaches to ML? This was my thought too. You'd think Gemini would set higher go

> has me wondering: are we seeing the limits of current "just scale the data" approaches to ML? This was my thought too. You'd think Gemini would set higher goals (higher than GPT-4) given that it will come out one year after GPT-4, but no, they are just achieving GPT-4-level performance in some respects. Maybe this says less about Google's lack of AI capabilities and more about OpenAI's first-mover advantage that made them pour billions into what was (like you said) the asymptote of AI intelligence using current arch.

echelon 3y ago
> pour billions into what was (like you said) the asymptote of AI intelligence using current arch. What a wild change in fortune that would be. OpenAI/Microsoft looked to be forever ahead, but maybe this is the break everyone else needs to catch up. I'll be really happy if we wind up with more companies owning different parts of the market rather than just FAANG at the top. Or more competition in general.
jiggawatts 3y ago
The bulk of the GPT-4 training was concluded back in 2021, before the billions of dollars thrown at them by Microsoft. It's notable that the scale, design, and architecture of GPT-4 is a closely guarded trade secret of "Open"AI. This strongly hints at some critical discovery, some new technique, or some aspect dialed up to 11. Whatever it is, the competition hasn't figured it out, or managed to copy it.