3 ms·

They are not statistical text generators. They are black boxes wrapped in an interface part of which is a statistical text generator. You can write programs in

They are not statistical text generators. They are black boxes wrapped in an interface part of which is a statistical text generator. You can write programs into transformer models and run them deterministically; they aren't "statistical".

godelski 2y ago
Can you cite the deterministic part? And I don't think fixing your seeds count as making a model not statistical, though yes deterministic. It's also still statistical if it gives you the same answer 99999/100000 times. You can definitely tune these things to be much more certain about certain problems but it tends to decrease capabilities in others. I also don't like the term "black box." While they aren't transparent they aren't completely opaque either. A big reason I didn't like the term is that I feel it also encourages people to not research this more. While we don't completely know what's going on I see no reason we can't. Is there some proof they their calculations are an unbreakable encryption or is it just a hard problem. I'm pretty sure it's the latter, and I think it's good to differentiate "we're dumb" from "indeterminate"
- taylorius 2y ago
  The issue with comprehending LLM's internal workings is just that they're so big. The operation of the network itself is deterministic - but it's output is a set of probabilities over the value of the next token in the stream. At that point you have to choose one, based on that probability distribution, and then start the process again. That choice is where the randomness comes in.
  - godelski 2y ago
    Just to clarify, are you disagreing with me? Correcting me? Adding context for others? I guess I'm missing the intent
    - taylorius 2y ago
      Just adding context, elaborating a little on the deterministic / statistical aspects of an LLM. I don't disagree with what you said, I'm afraid - so not much prospect for a back and forth :-)
- astrange 2y ago
  > Can you cite the deterministic part? And I don't think fixing your seeds count as making a model not statistical, though yes deterministic. I was thinking temperature=0, but what more do you need than that? (nb there is also some randomness from non-associative FP operations happening in different orders) > I also don't like the term "black box." While they aren't transparent they aren't completely opaque either. A white box is a subset of a black box here. The important part is that the sampler/text generation is wrapped around the transformer model and is not actually the same thing as it.
  - godelski 2y ago
    > I was thinking temperature=0, but what more do you need than that? Temperature isn't a thing exclusive to LLMs or ML. It's a general technique from statistics where you're modifying the variance of the distribution (also see truncation). And as you're aware, temperature zero isn't actually temperature zero. So say such a thing makes something non-statistical is the same as saying prime numbers don't exist because any prime p * 0 is 0. > A white box is a subset of a black box here No. A white box and black box sit at opposite ends of a spectrum which is just saying how much we understand something. Complexity isn't what makes something a black box, it's our lack of understanding that does. Of course this tends to coinside, but there's plenty of extremely complex things we have a great understanding of, even if not to infinite precision. Please take your passion and use it to drive you to learn about these things deeper. It's actually the same advice I give to the LLM fanboys who think it's AGI. But trying to dispel that myth without correct information makes it harder. It muddies the waters. These topics are extremely complex and the issue with the ML field is itself a hypocritical claim that it is easy (scale is all you need while calling these machines black boxes). I mean we can talk about the elephant in the room but certainly Von Neumann has said enough.
    - astrange 2y ago
      > No. A white box and black box sit at opposite ends of a spectrum which is just saying how much we understand something. So it's a subset, because "understanding something" is a refinement of the state of "not understanding something" - you can just pretend you don't know what's happening in that part of the system and it is now a black box.
HarHarVeryFunny 2y ago
LLMs are statistical next-word predictors, wrapped in a stochastic next-word generator. The next-word predictions output by the model itself are based on the patterns and statistics of the training set. The next word that is actually generated is typically a random selection per a top-k or top-p sampling scheme.

godelski 2y ago

Can you cite the deterministic part? And I don't think fixing your seeds count as making a model not statistical, though yes deterministic. It's also still statistical if it gives you the same answer 99999/100000 times. You can definitely tune these things to be much more certain about certain problems but it tends to decrease capabilities in others. I also don't like the term "black box." While they aren't transparent they aren't completely opaque either. A big reason I didn't like the term is that I feel it also encourages people to not research this more. While we don't completely know what's going on I see no reason we can't. Is there some proof they their calculations are an unbreakable encryption or is it just a hard problem. I'm pretty sure it's the latter, and I think it's good to differentiate "we're dumb" from "indeterminate"

taylorius 2y ago
The issue with comprehending LLM's internal workings is just that they're so big. The operation of the network itself is deterministic - but it's output is a set of probabilities over the value of the next token in the stream. At that point you have to choose one, based on that probability distribution, and then start the process again. That choice is where the randomness comes in.
- godelski 2y ago
  Just to clarify, are you disagreing with me? Correcting me? Adding context for others? I guess I'm missing the intent
  - taylorius 2y ago
    Just adding context, elaborating a little on the deterministic / statistical aspects of an LLM. I don't disagree with what you said, I'm afraid - so not much prospect for a back and forth :-)
astrange 2y ago
> Can you cite the deterministic part? And I don't think fixing your seeds count as making a model not statistical, though yes deterministic. I was thinking temperature=0, but what more do you need than that? (nb there is also some randomness from non-associative FP operations happening in different orders) > I also don't like the term "black box." While they aren't transparent they aren't completely opaque either. A white box is a subset of a black box here. The important part is that the sampler/text generation is wrapped around the transformer model and is not actually the same thing as it.
- godelski 2y ago
  > I was thinking temperature=0, but what more do you need than that? Temperature isn't a thing exclusive to LLMs or ML. It's a general technique from statistics where you're modifying the variance of the distribution (also see truncation). And as you're aware, temperature zero isn't actually temperature zero. So say such a thing makes something non-statistical is the same as saying prime numbers don't exist because any prime p * 0 is 0. > A white box is a subset of a black box here No. A white box and black box sit at opposite ends of a spectrum which is just saying how much we understand something. Complexity isn't what makes something a black box, it's our lack of understanding that does. Of course this tends to coinside, but there's plenty of extremely complex things we have a great understanding of, even if not to infinite precision. Please take your passion and use it to drive you to learn about these things deeper. It's actually the same advice I give to the LLM fanboys who think it's AGI. But trying to dispel that myth without correct information makes it harder. It muddies the waters. These topics are extremely complex and the issue with the ML field is itself a hypocritical claim that it is easy (scale is all you need while calling these machines black boxes). I mean we can talk about the elephant in the room but certainly Von Neumann has said enough.
  - astrange 2y ago
    > No. A white box and black box sit at opposite ends of a spectrum which is just saying how much we understand something. So it's a subset, because "understanding something" is a refinement of the state of "not understanding something" - you can just pretend you don't know what's happening in that part of the system and it is now a black box.

HarHarVeryFunny 2y ago

LLMs are statistical next-word predictors, wrapped in a stochastic next-word generator. The next-word predictions output by the model itself are based on the patterns and statistics of the training set. The next word that is actually generated is typically a random selection per a top-k or top-p sampling scheme.