3 ms·

I agree, probably for a different reason. IMO the fact that Sam is saying this likely means that the LLMs are close to the upper knee of the S-Curve and after G

I agree, probably for a different reason. IMO the fact that Sam is saying this likely means that the LLMs are close to the upper knee of the S-Curve and after GPT5 they likely dont have many new fundamental ideas (additional SCurves) to throw at it. since they are ahead of the rest, it likely means we might be headed for an AI improvement pause for couple of years after GPT5. Still good enough to upset the balance in search/ad market. Interesting times.

dragonwriter 4y ago
> it likely means we might be headed for an AI improvement pause for couple of years after GPT5. I suspect that a pause in base LLM performance won’t be an AI improvement pause; there’s a whole lot of space to improve the parts of AI systems around the core “brain in a jar” model.
- DesiLurker 4y ago
  I agree, there will be other things to be improved in AI system, but IMHO (tea leaves reading really) it would only lead to incremental improvements in overall systems. Also there is a lot of 'interfacing' work that needs to happen & i suspect that would end up filling the pause, which really is LLM productization loosely speaking. far as AGI is concerned I dont believe LLMs are really the right architecture for it, AGI likely needs some symbolic logic and a notion of physicality (ie.. physical laws & energy/power).
  - dragonwriter 4y ago
    > but IMHO (tea leaves reading really) it would only lead to incremental improvements in overall systems. It will reach a point where that is the case, sure; it is not there now, and if we are within one model generation of exhausting (for now) major core model improvements, I don’t think we’ll have reached the point of gradual incremental improvement from rest-of-system improvements yet.