3 ms·
>Progress in LLMs has come much more rapidly than expected I hear this very often, but I'm not sure, what were the expectations 10 years ago? What are the expe
by TylerLives 3y ago
>Progress in LLMs has come much more rapidly than expected
I hear this very often, but I'm not sure, what were the expectations 10 years ago? What are the expectations for the next 10 years?
- bjourne 3y agoAlmost to the date seven years ago: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/ "We downloaded the raw Latex source file (a 16MB file) and trained a multilayer LSTM. Amazingly, the resulting sampled Latex almost compiles. We had to step in and fix a few issues manually but then you get plausible looking math, it’s quite astonishing:" My emphasis.
- june_twenty 3y ago10 years ago we knew you needed a huge amount of data. These LLM are proof of that
- digging 3y agoWell, it's what I've heard from many "doomerists" in the last year. It's not even a question of 10 years; ChatGPT at launch was pretty surprising to a lot of people. Could be many experts weren't surprised by ChatGPT, though, since we got Sydney and GPT-4 and others pretty soon afterward.