3 ms·

> “it only predicts the next token” doesn’t (in principle/theory) preclude it having such an understanding of concepts, unless no computational process ever can

> “it only predicts the next token” doesn’t (in principle/theory) preclude it having such an understanding of concepts, unless no computational process ever can. In my opinion, this is highly reductive and academic. Whether these models are transformers or not, lookup likelihood is not indicative of understanding of concepts in any reasonable way. If the response to a algebraic equation was based upon probability of tokens in a corpus... and not an actual deterministic application of the rules of algebra, would that response know concepts? Would it be intelligent? With math, specifically given the unbounded size of the tokens compared to language, it's clear that token prediction is not a useful methodology. Let's say we're just trying to multiply two integers. Even if a model had Rain Man powers of memorization, and it memorized phone book after phone book of multiplication tables, the probabilistic likelihood model would fail for the very obvious reason that we cannot enumerate (and train on) all the possible outcomes of math and calculate their frequencies. We can however understand and use the concepts of math, which is distinct from their symbolic representation.

drdeca 2y ago
> lookup likelihood is not indicative of understanding of concepts in any reasonable way. Where did I ever say that the thing was doing lookup? I only said it was producing a probability distribution. Is your claim that all programs are just doing lookup? > If the response to a algebraic equation was based upon probability of tokens in a corpus... Ah, I see the confusion. When I say “probability distribution” I do not mean “for each option, an empirical fraction out of all the options, that this particular option appeared in the corpus”. Rather, by “probability distribution”, I mean (in the discrete case) “an assignment of a number which is at least zero and at most one, to each of the options, and such that the sum of the assigned values add up to 1”. I am allowing that this assignment of values is computed (from what is being conditioned on) in any way whatsoever . If the correct answer is a number, it may compute the entire correct number through some standard means, and then look at however many correct tokens from the number are already present, and assign a probability of 1 to the correct next one, and 0 to all other tokens. If conditioning on a partial answer that has parts wrong, it may use an arbitrary distribution.

drdeca 2y ago

> lookup likelihood is not indicative of understanding of concepts in any reasonable way. Where did I ever say that the thing was doing lookup? I only said it was producing a probability distribution. Is your claim that all programs are just doing lookup? > If the response to a algebraic equation was based upon probability of tokens in a corpus... Ah, I see the confusion. When I say “probability distribution” I do not mean “for each option, an empirical fraction out of all the options, that this particular option appeared in the corpus”. Rather, by “probability distribution”, I mean (in the discrete case) “an assignment of a number which is at least zero and at most one, to each of the options, and such that the sum of the assigned values add up to 1”. I am allowing that this assignment of values is computed (from what is being conditioned on) in any way whatsoever . If the correct answer is a number, it may compute the entire correct number through some standard means, and then look at however many correct tokens from the number are already present, and assign a probability of 1 to the correct next one, and 0 to all other tokens. If conditioning on a partial answer that has parts wrong, it may use an arbitrary distribution.