6 ms·

Let's say a model runs through a few iterations and finds a small, meaningful piece of information via "self-play" (iterating with itself without further prompt

Let's say a model runs through a few iterations and finds a small, meaningful piece of information via "self-play" (iterating with itself without further prompting from a human.) If the model then distills that information down to a new feature, and re-examines the original prompt with the new feature embedded in an extra input tensor, then repeats this process ad-infinitum, will the language model's "prime directive" and reasoning ability be sufficient to arrive at new, verifiable and provable conjectures, outside the realm of the dataset it was trained on? If GPT-4,5,...,n can progress in this direction, then we should all see the writing on the wall. Also, the day will come where we don't need to manually prepare an updated dataset and "kick off a new training". Self-supervised LLMs are going to be so shocking.

wbhart 3y ago
People have done experiments trying to get GPT-4 to come up with viable conjectures. So far it does such a woefully bad job that it isn't worth even trying. Unfortunately there are rather a lot of issues which are difficult to describe concisely, so here is probably not the best place. Primary amongst them is the fact that an LLM would be a horribly inefficient way to do this. There are much, much better ways, which have been tried, with limited success.
- gmerc 3y ago
  After a year the entire argument you make boils down to “so far”.
  - ra 3y ago
    Indeed. LLM is an application on a transformer trained with backpropagation. What stops you from adding a logic/mathematic "application" on the same transformer?
    - seanhunter 3y ago
      Nothing, and there are methods which allow these types of models to learn to use special purpose tools of this kind[1]. [1] https://arxiv.org/abs/2302.04761 https://arxiv.org/abs/2302.04761 Toolformer: Language Models Can Teach Themselves to Use Tools
  - Terr_ 3y ago
    Whereas your post sounds like "Just give the approach more time, it shall continue to incrementally improve until it finally works someday, cuz reasons." Early attempts at human flight approached it by strapping wings to people's arms and flapping: Do you think that would have eventually worked too, if only we had just given it a bit more time and faith?
    - xcv123 3y ago
      > Just give the approach more time, it shall continue to incrementally improve until it finally works someday, cuz reasons Yes. Because we haven't yet reached the limit of deep learning models. GPT-3.5 has 175 billion parameters. GPT-4 has an estimated 1.8 trillion parameters. That was nearly a year ago. Wait until you see what's next.
      - meheleventyone 3y ago
        Why would adding more parameters suddenly make it better at this sort of reasoning? It feels a bit of a “god of the gaps” where it’ll just stop being a stochastic parrot in just a few more million parameters.
        Al-Khwarizmi 3y ago
        I don't think it's guaranteed, but I do think it's very plausible because we've seen these models gain emerging abilities at every iteration, just from sheer scaling. So extrapolation tells us that they may keep gaining more capabilities (we don't know how exactly it does it, though, so of course it's all speculation). I don't think many people would describe GPT-4 as a stochastic parrot already... when the paper that coined (or at least popularized) the term came up in early 2021, the term made a lot of sense. In late 2023, with models that at the very least show clear signs of creativity (I'm sticking to that because "reasoning" or not is more controversial), it's relegated to reductionistic philosophical arguments, but not really a practical description anymore.
        meheleventyone 3y ago
jimmySixDOF 3y ago
Yes, it seems like this is a direction to replace RLHF so another way to scale without baremetal and if not this then still just a matter of time before some model optimization outperforms the raw epoch/parameters/token approach.

wbhart 3y ago

People have done experiments trying to get GPT-4 to come up with viable conjectures. So far it does such a woefully bad job that it isn't worth even trying. Unfortunately there are rather a lot of issues which are difficult to describe concisely, so here is probably not the best place. Primary amongst them is the fact that an LLM would be a horribly inefficient way to do this. There are much, much better ways, which have been tried, with limited success.

gmerc 3y ago
After a year the entire argument you make boils down to “so far”.
- ra 3y ago
  Indeed. LLM is an application on a transformer trained with backpropagation. What stops you from adding a logic/mathematic "application" on the same transformer?
  - seanhunter 3y ago
    Nothing, and there are methods which allow these types of models to learn to use special purpose tools of this kind[1]. [1] https://arxiv.org/abs/2302.04761 https://arxiv.org/abs/2302.04761 Toolformer: Language Models Can Teach Themselves to Use Tools
- Terr_ 3y ago
  Whereas your post sounds like "Just give the approach more time, it shall continue to incrementally improve until it finally works someday, cuz reasons." Early attempts at human flight approached it by strapping wings to people's arms and flapping: Do you think that would have eventually worked too, if only we had just given it a bit more time and faith?
  - xcv123 3y ago
    > Just give the approach more time, it shall continue to incrementally improve until it finally works someday, cuz reasons Yes. Because we haven't yet reached the limit of deep learning models. GPT-3.5 has 175 billion parameters. GPT-4 has an estimated 1.8 trillion parameters. That was nearly a year ago. Wait until you see what's next.
    - meheleventyone 3y ago
      Why would adding more parameters suddenly make it better at this sort of reasoning? It feels a bit of a “god of the gaps” where it’ll just stop being a stochastic parrot in just a few more million parameters.
      - Al-Khwarizmi 3y ago
        I don't think it's guaranteed, but I do think it's very plausible because we've seen these models gain emerging abilities at every iteration, just from sheer scaling. So extrapolation tells us that they may keep gaining more capabilities (we don't know how exactly it does it, though, so of course it's all speculation). I don't think many people would describe GPT-4 as a stochastic parrot already... when the paper that coined (or at least popularized) the term came up in early 2021, the term made a lot of sense. In late 2023, with models that at the very least show clear signs of creativity (I'm sticking to that because "reasoning" or not is more controversial), it's relegated to reductionistic philosophical arguments, but not really a practical description anymore.
        meheleventyone 3y ago

jimmySixDOF 3y ago

Yes, it seems like this is a direction to replace RLHF so another way to scale without baremetal and if not this then still just a matter of time before some model optimization outperforms the raw epoch/parameters/token approach.