4 ms·

Really glad to see some academic research on this- it was quite obvious from interacting with LLMs that they form a world model and can, e.g. simulate simple ph

by _yb2s 2y ago

Really glad to see some academic research on this- it was quite obvious from interacting with LLMs that they form a world model and can, e.g. simulate simple physics experiments correctly that are not in the training set. I found it very frustrating to see people repeating the idea that “it can never do x” because it lacks a world model. Predicting text that represents events in the world requires modeling that world. Just because you can find examples where the predictions of a certain model are bad does not imply no model at all. At the limit of prediction becoming as good as theoretically possible given the input data and model size restrictions, the model also becomes as accurate and complete as possible. This process is formally described by the Solomonoff Induction theory.

slashdave 2y ago
> At the limit of prediction becoming as good as theoretically possible given the input data and model size restrictions You are treading on delicate ground here. Why do you believe that sequence models are capable of reaching theoretical maximums?
- _yb2s 2y ago
  I do not think any real systems can ever achieve theoretically perfect Solomonoff Induction- only that increasingly good AI systems can be thought of as increasingly good approximations of this process. I do not know if any particular modeling approach has a fundamental dead end that limits its potential or not. However, my main point is that people claiming that they are certain of a particular fundamental limitation are mistaken. Current LLMs aren’t very intelligent, yet can already do specific things that people like Noam Chomsky have argued are fundamentally theoretically impossible for them to ever do.
  - slashdave 2y ago
    > However, my main point is that people claiming that they are certain of a particular fundamental limitation are mistaken. No, they are correct. The architecture, by design and construction, is limited. This is simple math.
    - _yb2s 2y ago
      Limited how exactly? What limitation are you talking about, and what math proves it?

by _yb2s 2y ago

slashdave 2y ago

> At the limit of prediction becoming as good as theoretically possible given the input data and model size restrictions You are treading on delicate ground here. Why do you believe that sequence models are capable of reaching theoretical maximums?

_yb2s 2y ago
I do not think any real systems can ever achieve theoretically perfect Solomonoff Induction- only that increasingly good AI systems can be thought of as increasingly good approximations of this process. I do not know if any particular modeling approach has a fundamental dead end that limits its potential or not. However, my main point is that people claiming that they are certain of a particular fundamental limitation are mistaken. Current LLMs aren’t very intelligent, yet can already do specific things that people like Noam Chomsky have argued are fundamentally theoretically impossible for them to ever do.
- slashdave 2y ago
  > However, my main point is that people claiming that they are certain of a particular fundamental limitation are mistaken. No, they are correct. The architecture, by design and construction, is limited. This is simple math.
  - _yb2s 2y ago
    Limited how exactly? What limitation are you talking about, and what math proves it?