4 ms·

> The short answer is that of cause LLMs can come up with something new: Well, yes. They produce random output that adhere to statistical models. I had ChatGP

by e12e 3y ago

> The short answer is that of cause LLMs can come up with something new: Well, yes. They produce random output that adhere to statistical models. I had ChatGPT 3.5 produce a wonderful whimsical portrait interview of a couple breeding Giant Tibetian Hamsters for home protection in Norwegian. > They can learn to rationalise – the simplest example is learning to add numbers. Certainly not? Composite systems that leverage LLMs can do a lot of things - but AFAIU LLMs will likely never rationalize or be able to "add numbers" in the normal sense; they can count only in as much they know that 1,2,3,4 is more probably coming after "count to four" than 10,2,4.

famouswaffles 3y ago
>Well, yes. They produce random output that adhere to statistical models. Not really. Nothing random about the output of LLMs. https://www.nature.com/articles/s41587-022-01618-2 https://www.nature.com/articles/s41587-022-01618-2 >Certainly not? Composite systems that leverage LLMs can do a lot of things - but AFAIU LLMs will likely never rationalize or be able to "add numbers" in the normal sense; they can count only in as much they know that 1,2,3,4 is more probably coming after "count to four" than 10,2,4. LLMs can add numbers just fine. arithmetic is one of the easiest domains to test on data it'd never have seen in training.
- _ea1k 3y ago
  > Not really. Nothing random about the output of LLMs. https://www.nature.com/articles/s41587-022-01618-2 https://www.nature.com/articles/s41587-022-01618-2 I can see what you are getting at, but the underlying implementation of LLMs really is reliant on non-deterministic, random sampling of the underlying model. The random sampling is weighted to favor certain selections over others, but rare selections are possible. Its intrinsically built on a degree of randomness. Though it is also unfair to say something like "produce random output" as the probabilities themselves make it less than fully random.
HL33tibCe7 3y ago
> Certainly not? Composite systems that leverage LLMs can do a lot of things - but AFAIU LLMs will likely never rationalize or be able to "add numbers" in the normal sense; This implies we know what “rationalising” actually is, which we don’t. There’s no reason to believe that our brains don’t operate in fundamentally the same way as LLMs. There’s no reason to think that the LLM approach to reasoning is any less valid than the “normal” way, whatever that even means.
IanCal 3y ago
> but AFAIU LLMs will likely never rationalize or be able to "add numbers" in the normal sense; they can count only in as much they know that 1,2,3,4 is more probably coming after "count to four" than 10,2,4. You might be interested in this experiment - https://thegradient.pub/othello/ https://thegradient.pub/othello/ The question is does a model trained on move sequences in a game of othello just learn "play X(t) after seeing the sequence X(t-3), X(t-2), X(t-1)" or does it build a representation of the board and use that to choose the next move to play.
mo_42 3y ago
In ML 101, I learned that NNs are general function approximators. So given the right training data and algorithm, they should be able to learn to add numbers. Or any other computable function. Also, we don’t know how GPT computes numbers really. I guess no one traced all the layers and understood the activations. We also don’t do this with our children. We just assume that they understand it as soon as they don’t make many mistakes anymore.
- withinboredom 3y ago
  It depends on the activation functions. Some may never be able to compute addition.
- simiones 3y ago
  I don't know why this mistake gets repeated, but NNs are universal continous function approximators. They can't approximate non-continuous functions in general, and there are plenty of non-continuous computable functions. Also, this theorem is almost useless in practice: it only tells you that, for any function and desired error rate, there exists some NN which would approximate that function within that error rate. But there is no proof that there is some way to train the weights for that NN based on any known training mechanism, even if we magically knew the shape of that NN (which we don't). Obviously, since we don't know if an algorithm to train the NN even exists, we have even less idea of how long it might take, or how large the training set would have to be. So this theorem doesn't really help in any way answer whether our current AI techniques (of which training is a fundamental component) could be used to approximate human reason.
  - robert_no_robot 3y ago
    > They can't approximate non-continuous functions in general, and there are plenty of non-continuous computable functions. For a given error I guess. So it's a discussion about how good we can approximate such a function.
    - tsimionescu 3y ago
deleted 3y ago
[deleted]

by e12e 3y ago

famouswaffles 3y ago

>Well, yes. They produce random output that adhere to statistical models. Not really. Nothing random about the output of LLMs. https://www.nature.com/articles/s41587-022-01618-2 https://www.nature.com/articles/s41587-022-01618-2 >Certainly not? Composite systems that leverage LLMs can do a lot of things - but AFAIU LLMs will likely never rationalize or be able to "add numbers" in the normal sense; they can count only in as much they know that 1,2,3,4 is more probably coming after "count to four" than 10,2,4. LLMs can add numbers just fine. arithmetic is one of the easiest domains to test on data it'd never have seen in training.

_ea1k 3y ago
> Not really. Nothing random about the output of LLMs. https://www.nature.com/articles/s41587-022-01618-2 https://www.nature.com/articles/s41587-022-01618-2 I can see what you are getting at, but the underlying implementation of LLMs really is reliant on non-deterministic, random sampling of the underlying model. The random sampling is weighted to favor certain selections over others, but rare selections are possible. Its intrinsically built on a degree of randomness. Though it is also unfair to say something like "produce random output" as the probabilities themselves make it less than fully random.

HL33tibCe7 3y ago

> Certainly not? Composite systems that leverage LLMs can do a lot of things - but AFAIU LLMs will likely never rationalize or be able to "add numbers" in the normal sense; This implies we know what “rationalising” actually is, which we don’t. There’s no reason to believe that our brains don’t operate in fundamentally the same way as LLMs. There’s no reason to think that the LLM approach to reasoning is any less valid than the “normal” way, whatever that even means.

IanCal 3y ago

> but AFAIU LLMs will likely never rationalize or be able to "add numbers" in the normal sense; they can count only in as much they know that 1,2,3,4 is more probably coming after "count to four" than 10,2,4. You might be interested in this experiment - https://thegradient.pub/othello/ https://thegradient.pub/othello/ The question is does a model trained on move sequences in a game of othello just learn "play X(t) after seeing the sequence X(t-3), X(t-2), X(t-1)" or does it build a representation of the board and use that to choose the next move to play.

mo_42 3y ago

In ML 101, I learned that NNs are general function approximators. So given the right training data and algorithm, they should be able to learn to add numbers. Or any other computable function. Also, we don’t know how GPT computes numbers really. I guess no one traced all the layers and understood the activations. We also don’t do this with our children. We just assume that they understand it as soon as they don’t make many mistakes anymore.

withinboredom 3y ago
It depends on the activation functions. Some may never be able to compute addition.
simiones 3y ago
I don't know why this mistake gets repeated, but NNs are universal continous function approximators. They can't approximate non-continuous functions in general, and there are plenty of non-continuous computable functions. Also, this theorem is almost useless in practice: it only tells you that, for any function and desired error rate, there exists some NN which would approximate that function within that error rate. But there is no proof that there is some way to train the weights for that NN based on any known training mechanism, even if we magically knew the shape of that NN (which we don't). Obviously, since we don't know if an algorithm to train the NN even exists, we have even less idea of how long it might take, or how large the training set would have to be. So this theorem doesn't really help in any way answer whether our current AI techniques (of which training is a fundamental component) could be used to approximate human reason.
- robert_no_robot 3y ago
  > They can't approximate non-continuous functions in general, and there are plenty of non-continuous computable functions. For a given error I guess. So it's a discussion about how good we can approximate such a function.
  - tsimionescu 3y ago

deleted 3y ago

[deleted]