5 ms·

We absolutely have a way to get there. A sufficiently large neural network can approximate any function (the universal approximation theorem), which has been ma

We absolutely have a way to get there. A sufficiently large neural network can approximate any function (the universal approximation theorem), which has been mathematically proven. LLMs are trained to predict the next word in a sequence; doing this to a high enough level of accuracy requires building an internal model of the world; the ability to reason about it. Eventually we'll have LLMs trained on visual and audio input too (and be trained to predict what happens next), so they can receive an arbitrarily large amount of training samples. If the models keep growing and being trained on more and more data, we'd expect them to become at least as intelligent (good at modelling the world) as humans.

tedivm 3y ago
> LLMs are trained to predict the next word in a sequence; doing this to a high enough level of accuracy requires building an internal model of the world; the ability to reason about it. I don't see any evidence that this is true or proven in any way. Also, the ability to brute force something doesn't mean that brute forcing it is easy or even feasible. You can apply the same logic to "calculating a private key from a public key" that you are to human intelligence. Sure, a large enough neural network can do either, but that doesn't mean that building them is actually realistic.
- MostlyStable 3y ago
  It has been proven in much simpler toy models that LLMs just trained on token prediction can and do build "world" models[0]. That's not the same as evidence that they must build a world model, but it's proof that at least sometimes they do so even when just being trained on prediction. [0]https://thegradient.pub/othello/ https://thegradient.pub/othello/
tsimionescu 3y ago
This keeps popping up, but a sufficiently large neural network can approximate any continuous function. They can't approximate any discontinuous function. So, there is no absolute certainty that they must be able to approximate human cognition, since there is no reason to believe cognition is continuous. Also, there is no proof and no reason to believe that any current architecture and, more importantly, that the currently known training algorithms can achieve anything close to cognition. After all, animals and humans seem to learn much much much faster (much smaller training sets) than any algorithm we have so far.
- CamperBob2 3y ago
  This keeps popping up, but a sufficiently large neural network can approximate any continuous function. They can't approximate any discontinuous function Incorrect. That's what nonlinear activation functions are for. Researchers wasted many potentially-fruitful years because Minsky and other luminaries -- people who occupied the same position of authority that LeCun occupies today -- made a huge deal about how the original perceptron and subsequent multilayer variants "couldn't learn XOR." That was almost literally how they put it. At the first sign of difficulty they abandoned the approach that was, and remains, the most promising. We have to be careful to learn from that mistake.
- adroniser 3y ago
  You have to be a lot clearer about what you mean by continuous here. An LLM technically does not produce a continuous function on the reals because its inputs are floating point numbers with finite precision. Any function on discrete inputs like this has an extension to the reals which is continuous, just imagine joining all the discrete points up with lines. So then your claim wouldn't be about the limits of LLMs themselves, but on the limits of systems that do not take continuous inputs. The question then is do you think that humans take in continuous input?? Given that physics seems to be discrete at the low level, this suggests to me they don't, but I don't know enough to be sure.
  - lanstin 3y ago
    It is quantum which isn’t exactly discrete. You receive one photon or two, not 1.5, sure, but the energy of that photon, its frequency is a real. At least I am not aware of a quantum mechanics over Q. I think the math would be hard because you loose all the convergence properties. Maybe there is some Hilbert space over computable reals, Google is not finding it but I no longer find that to be indicative of anything.
  - tsimionescu 3y ago
    I mean continuous in the calculus sense, as in it doesn't have discontinuities, not as in continuous VS discrete. > Given that physics seems to be discrete at the low level, this suggests to me they don't, but I don't know enough to be sure. This is a misunderstanding of quantum mechanics. Only certain specific quantities come in discrete quanta (spin, charge, certain energy levels, etc). Other physical quantities are very much continuous - notably time and space. In fact, much of the mathematics of QM is not discretizable, it won't work if you try to make time or space discrete.
ryaniscool 3y ago
There is a leap from "predicting the next word" to "modelling the world" that you are sort of glossing over here. The idea that LLM's are actually "reasoning" rather than just performing a probability function seems a little bit...pseudo-religious. Like we've created a new life form.
RandomLensman 3y ago
Predictive models need not at all reflect what actually happens. Sometimes embracing less accurate models is actually need to enhance understanding. Epicycles in the geocentric model could be made very accurate as far as observations in the sky are concerned, but it wasn't a good world model.
kaba0 3y ago
That whitepaper operates with mathematical, infinite precision numbers. It’s just a categorical error to read too much into it - it’s not really a too interesting property to be able to get closer to something forever, without some sane growth function.

tedivm 3y ago

> LLMs are trained to predict the next word in a sequence; doing this to a high enough level of accuracy requires building an internal model of the world; the ability to reason about it. I don't see any evidence that this is true or proven in any way. Also, the ability to brute force something doesn't mean that brute forcing it is easy or even feasible. You can apply the same logic to "calculating a private key from a public key" that you are to human intelligence. Sure, a large enough neural network can do either, but that doesn't mean that building them is actually realistic.

MostlyStable 3y ago
It has been proven in much simpler toy models that LLMs just trained on token prediction can and do build "world" models[0]. That's not the same as evidence that they must build a world model, but it's proof that at least sometimes they do so even when just being trained on prediction. [0]https://thegradient.pub/othello/ https://thegradient.pub/othello/

tsimionescu 3y ago

This keeps popping up, but a sufficiently large neural network can approximate any continuous function. They can't approximate any discontinuous function. So, there is no absolute certainty that they must be able to approximate human cognition, since there is no reason to believe cognition is continuous. Also, there is no proof and no reason to believe that any current architecture and, more importantly, that the currently known training algorithms can achieve anything close to cognition. After all, animals and humans seem to learn much much much faster (much smaller training sets) than any algorithm we have so far.

CamperBob2 3y ago
This keeps popping up, but a sufficiently large neural network can approximate any continuous function. They can't approximate any discontinuous function Incorrect. That's what nonlinear activation functions are for. Researchers wasted many potentially-fruitful years because Minsky and other luminaries -- people who occupied the same position of authority that LeCun occupies today -- made a huge deal about how the original perceptron and subsequent multilayer variants "couldn't learn XOR." That was almost literally how they put it. At the first sign of difficulty they abandoned the approach that was, and remains, the most promising. We have to be careful to learn from that mistake.
adroniser 3y ago
You have to be a lot clearer about what you mean by continuous here. An LLM technically does not produce a continuous function on the reals because its inputs are floating point numbers with finite precision. Any function on discrete inputs like this has an extension to the reals which is continuous, just imagine joining all the discrete points up with lines. So then your claim wouldn't be about the limits of LLMs themselves, but on the limits of systems that do not take continuous inputs. The question then is do you think that humans take in continuous input?? Given that physics seems to be discrete at the low level, this suggests to me they don't, but I don't know enough to be sure.
- lanstin 3y ago
  It is quantum which isn’t exactly discrete. You receive one photon or two, not 1.5, sure, but the energy of that photon, its frequency is a real. At least I am not aware of a quantum mechanics over Q. I think the math would be hard because you loose all the convergence properties. Maybe there is some Hilbert space over computable reals, Google is not finding it but I no longer find that to be indicative of anything.
- tsimionescu 3y ago
  I mean continuous in the calculus sense, as in it doesn't have discontinuities, not as in continuous VS discrete. > Given that physics seems to be discrete at the low level, this suggests to me they don't, but I don't know enough to be sure. This is a misunderstanding of quantum mechanics. Only certain specific quantities come in discrete quanta (spin, charge, certain energy levels, etc). Other physical quantities are very much continuous - notably time and space. In fact, much of the mathematics of QM is not discretizable, it won't work if you try to make time or space discrete.

ryaniscool 3y ago

There is a leap from "predicting the next word" to "modelling the world" that you are sort of glossing over here. The idea that LLM's are actually "reasoning" rather than just performing a probability function seems a little bit...pseudo-religious. Like we've created a new life form.

RandomLensman 3y ago

Predictive models need not at all reflect what actually happens. Sometimes embracing less accurate models is actually need to enhance understanding. Epicycles in the geocentric model could be made very accurate as far as observations in the sky are concerned, but it wasn't a good world model.

kaba0 3y ago

That whitepaper operates with mathematical, infinite precision numbers. It’s just a categorical error to read too much into it - it’s not really a too interesting property to be able to get closer to something forever, without some sane growth function.