5 ms·

I'm not deeply in this space, but from what I understand, the problems are thus: 1. Progress in LLMs has come much more rapidly than expected. This means that

I'm not deeply in this space, but from what I understand, the problems are thus: 1. Progress in LLMs has come much more rapidly than expected. This means that when [arbitrary threshold] is crossed, we probably won't have much, if any, advance warning. 2. Nobody on earth knows what the path to AGI (or even narrow-but-still-superhuman intelligence with enough agency to be dangerous) looks like. So, it's not currently possible to know if LLMs are a path to existential threat. (No, it is not correct to say that LLMs are too simple to ever be a threat, as far as I can tell.) (Recall also that we don't know at all where we get our own consciousness or how it works.) So it seems like #2 is more where you're hung up, and frankly, it's just unknown. If we knew what the path to AGI looked like, we woul be in a very different world. When you combine with #1, it becomes very scary because we might be on the cusp of an irreversible change, so it's useful to assume we're there if doing so allows you to potentially alter or avert disaster.

TylerLives 3y ago
>Progress in LLMs has come much more rapidly than expected I hear this very often, but I'm not sure, what were the expectations 10 years ago? What are the expectations for the next 10 years?
- bjourne 3y ago
  Almost to the date seven years ago: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/ "We downloaded the raw Latex source file (a 16MB file) and trained a multilayer LSTM. Amazingly, the resulting sampled Latex almost compiles. We had to step in and fix a few issues manually but then you get plausible looking math, it’s quite astonishing:" My emphasis.
- june_twenty 3y ago
  10 years ago we knew you needed a huge amount of data. These LLM are proof of that
- digging 3y ago
  Well, it's what I've heard from many "doomerists" in the last year. It's not even a question of 10 years; ChatGPT at launch was pretty surprising to a lot of people. Could be many experts weren't surprised by ChatGPT, though, since we got Sydney and GPT-4 and others pretty soon afterward.

TylerLives 3y ago

>Progress in LLMs has come much more rapidly than expected I hear this very often, but I'm not sure, what were the expectations 10 years ago? What are the expectations for the next 10 years?

bjourne 3y ago
Almost to the date seven years ago: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/ "We downloaded the raw Latex source file (a 16MB file) and trained a multilayer LSTM. Amazingly, the resulting sampled Latex almost compiles. We had to step in and fix a few issues manually but then you get plausible looking math, it’s quite astonishing:" My emphasis.
june_twenty 3y ago
10 years ago we knew you needed a huge amount of data. These LLM are proof of that
digging 3y ago
Well, it's what I've heard from many "doomerists" in the last year. It's not even a question of 10 years; ChatGPT at launch was pretty surprising to a lot of people. Could be many experts weren't surprised by ChatGPT, though, since we got Sydney and GPT-4 and others pretty soon afterward.