4 ms·
My opinion shifts every other day it seems, recently, Roon, the "famous" anon X account that works at OpenAI, and presumably has access to some unreleased info,
by lambdaba 2y ago
My opinion shifts every other day it seems, recently, Roon, the "famous" anon X account that works at OpenAI, and presumably has access to some unreleased info, said, in response to "are we plateauing soon", "not even close". I honestly don't know what to think.
- whimsicalism 2y agoI know his real identity and I doubt there is some key insider info there although he does work at OAI
- mlyle 2y agoI feel like for LLM's at least, we've not seen a lot of progress on the high end for a year or so. GPT4-Turbo is a little better than GPT4, but mainly only for things that can benefit from huge contexts.
- deleted 2y ago[deleted]
- whimsicalism 2y agoWe have seen progress, but mostly price progress.
- mlyle 2y agoYes, and also not-always-bumping-into-capacity-limits (availability) progress.
- mjhay 2y agoOf course somebody working for OpenAI would say that.
- HarHarVeryFunny 2y agoOTOH Altman has said that next release will be about same advance on GTP-4 as that was over GPT-3. Dario Amodei has said something roughly equivalent - to expect solid gains this year but no reality bender. I think the real potential gains come when they start extending the architecture, but as long as it's just scaling up and different training/inference regimes, then it seems it'll be more of the same rather than game changer. Given how connected the whole SF/AI scene appears to be (alchohol + drugs too?), it's hard to imagine a company the size of OpenAI not leaking. If there had been any amazing discoveries there, I think there'd at least be rumors (and not just "they seem to have something called Q* going on").