4 ms·

I think the two modes of LLM discourse: “they’re conscious!/they’re just next token predictors with impressive datasets” comes largely from two different groups

I think the two modes of LLM discourse: “they’re conscious!/they’re just next token predictors with impressive datasets” comes largely from two different groups of people: those who learned about LLMs before learning about ML fundamentals, and those who learned ML fundamentals before encountering LLMs of today. While I fall in the second group, there is a real risk that my prior concepts about the fundamentals is limiting my view of the bigger picture, so I at least welcome the debate. Re: chain of thought, I at least know that in practice a lot of the results from the original paper has not been quite reproducible at later attempts. Whether that is a quirk of models changing everyday or something deeper, I do not know.

YeGoblynQueenne 3y ago
Instinctively I'd trust the people with the knowledge that goes farther back in time. On the other hand, I once whinged to my thesis advisor that a lot of people in machine learning don't seem to know much about older machine learning and AI work and he, with 30+ years of research on me, pointed out that people complained about that already when he was a PhD student. There is so much work on AI that goes back about 80 years (counting from Pitts and McCulloch, because why not; or you could count from Turing) and it's very hard to both keep up with what everyone else is doing, and go deep in your own subject. e.g. you pick up a Reinforcement Learning book and it's basically attacking the same problems as in planning, and with very similar assumptions (states and action spaces) but it's like planning doesn't even exist. Btw: they're next token predictors :P
- wayeq 3y ago
  > Btw: they're next token predictors :P Why not (possibly) both? After all, we can't even define consciousness, other than "what's it like to be a bat" makes more sense to us intuitively than "what's it like to be a rock". Consciousness may just come along with the ride with a certain amount of information processing, we just have no clue.
  - YeGoblynQueenne 3y ago
    If we can't define consciousness then that's a very good reason not to assume that a computer program is conscious, especially when it wasn't created with the purpose of being conscious in the first place. Consciousness is something we say animals or humans have, but not computers, so to say that a computer program is conscious it takes a lot more work and, I guess, a formal definition that most of us can agree on.
moffkalast 3y ago
At this point I think I'm leaning towards "organic brains are just next token predictors with impressive secondary heuristic systems". The fact that we can get such impressive results from transformers which are such a poor approximation and completely stateless makes me think there really isn't any special sauce to it.

YeGoblynQueenne 3y ago

Instinctively I'd trust the people with the knowledge that goes farther back in time. On the other hand, I once whinged to my thesis advisor that a lot of people in machine learning don't seem to know much about older machine learning and AI work and he, with 30+ years of research on me, pointed out that people complained about that already when he was a PhD student. There is so much work on AI that goes back about 80 years (counting from Pitts and McCulloch, because why not; or you could count from Turing) and it's very hard to both keep up with what everyone else is doing, and go deep in your own subject. e.g. you pick up a Reinforcement Learning book and it's basically attacking the same problems as in planning, and with very similar assumptions (states and action spaces) but it's like planning doesn't even exist. Btw: they're next token predictors :P

wayeq 3y ago
> Btw: they're next token predictors :P Why not (possibly) both? After all, we can't even define consciousness, other than "what's it like to be a bat" makes more sense to us intuitively than "what's it like to be a rock". Consciousness may just come along with the ride with a certain amount of information processing, we just have no clue.
- YeGoblynQueenne 3y ago
  If we can't define consciousness then that's a very good reason not to assume that a computer program is conscious, especially when it wasn't created with the purpose of being conscious in the first place. Consciousness is something we say animals or humans have, but not computers, so to say that a computer program is conscious it takes a lot more work and, I guess, a formal definition that most of us can agree on.

moffkalast 3y ago

At this point I think I'm leaning towards "organic brains are just next token predictors with impressive secondary heuristic systems". The fact that we can get such impressive results from transformers which are such a poor approximation and completely stateless makes me think there really isn't any special sauce to it.