4 ms·
I remember some people were saying things in vaguely but not explicitly that direction, but given OpenAI were "we're not trying to make bigger models for now, w
by ben_w 3y ago
I remember some people were saying things in vaguely but not explicitly that direction, but given OpenAI were "we're not trying to make bigger models for now, we're trying to learn more about the ones we've already got and how to make sure they're safe" I dismissed them as fantasists.
What has happened is GPT-4 came out (which is certainly better in some domains but not everywhere), but mainly the models have become much cheaper and slightly easier to run, and people are pairing LLMs with other things rather than using them as a single solution for all possible tasks — which they probably could do in principle if scaled up sufficiently, but there may well not be enough training data and there certainly aren't computers with enough RAM.
And, like with the self-driving cars, we've learned a lot of surprising failure modes.
(As I'm currently job-hunting, I hope what I wrote here is true and not just… is "copium" the appropriate neologism?)