3 ms·

And also whenever the next iteration bootstraps off the Wolfram outputs so it no longer has to call Wolfram & can better generalize/learn thanks to it...

by gwern 4y ago

superkuh 4y ago
Wolfram himself laid out why this is unlikely to happen, https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... >Or put another way, there’s an ultimate tradeoff between capability and trainability: the more you want a system to make “true use” of its computational capabilities, the more it’s going to show computational irreducibility, and the less it’s going to be trainable. And the more it’s fundamentally trainable, the less it’s going to be able to do sophisticated computation. >(For ChatGPT as it currently is, the situation is actually much more extreme, because the neural net used to generate each token of output is a pure “feed-forward” network, without loops, and therefore has no ability to do any kind of computation with nontrivial “control flow”.)
- sebzim4500 4y ago
  >and therefore has no ability to do any kind of computation with nontrivial “control flow” This isn't true if you let the model output tokens that aren't sent to the user. Basically the null api.
  - TOMDM 4y ago
    Exactly, giving ChatGPT a scratchpad it can use to think is trivial in this new plugin enabled ChatGPT world.
- calf 4y ago
  What is a feed forward network? Is that anything like the current-state and next-state functions for representing finite state machines?
  - telotortium 4y ago
    No, a feed forward network is just a neural network which doesn't contain cycles, in contrast to recurrent neural networks, which do. A system that can't perform cyclical behavior is not Turing complete, since we know that it will halt. Large language models are not recurrent networks because they don't contain cycles (the whole point of the transformer architecture is to mix position information with the tokens so that the model is sensitive to short and long range context, which obviates the need for recurrent networks for "memory" ), so they are not Turing complete - at least in a single iteration - and will probably never compute many functions efficiently. Nevertheless, because you can feed the generated text back into the LLM pipeline, they can operate on current-state and next-state as in your example of a finite state machine.
    - calf 4y ago
      Yeah that is a neat hack one of the comments mentioned. Also if a model of computation that is not Turing complete can apparently do English, then what does that say about the computational complexity of English, this piques my curiosity.
- gwern 4y ago
  Many of the examples are things that a feedforward model like GPT-4 has plenty of capacity for, could do (eg all the real-world facts), and each example helps induce the transition between memorization and generalization so it will do. In some cases like multiplying large numbers, it can't no matter how many examples it's given (with a realistic number, anyway) - but every query firms up the ability to swap out Wolfram Alpha for any other calculator... Wolfram is playing with fire here (https://gwern.net/complement https://gwern.net/complement) given that the output is going to, one way or another, assist training future AI models.

And also whenever the next iteration bootstraps off the Wolfram outputs so it no longer has to call Wolfram & can better generalize/learn thanks to it...

by gwern 4y ago

And also whenever the next iteration bootstraps off the Wolfram outputs so it no longer has to call Wolfram & can better generalize/learn thanks to it...

superkuh 4y ago

Wolfram himself laid out why this is unlikely to happen, https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... >Or put another way, there’s an ultimate tradeoff between capability and trainability: the more you want a system to make “true use” of its computational capabilities, the more it’s going to show computational irreducibility, and the less it’s going to be trainable. And the more it’s fundamentally trainable, the less it’s going to be able to do sophisticated computation. >(For ChatGPT as it currently is, the situation is actually much more extreme, because the neural net used to generate each token of output is a pure “feed-forward” network, without loops, and therefore has no ability to do any kind of computation with nontrivial “control flow”.)

sebzim4500 4y ago
>and therefore has no ability to do any kind of computation with nontrivial “control flow” This isn't true if you let the model output tokens that aren't sent to the user. Basically the null api.
- TOMDM 4y ago
  Exactly, giving ChatGPT a scratchpad it can use to think is trivial in this new plugin enabled ChatGPT world.
calf 4y ago
What is a feed forward network? Is that anything like the current-state and next-state functions for representing finite state machines?
- telotortium 4y ago
  No, a feed forward network is just a neural network which doesn't contain cycles, in contrast to recurrent neural networks, which do. A system that can't perform cyclical behavior is not Turing complete, since we know that it will halt. Large language models are not recurrent networks because they don't contain cycles (the whole point of the transformer architecture is to mix position information with the tokens so that the model is sensitive to short and long range context, which obviates the need for recurrent networks for "memory" ), so they are not Turing complete - at least in a single iteration - and will probably never compute many functions efficiently. Nevertheless, because you can feed the generated text back into the LLM pipeline, they can operate on current-state and next-state as in your example of a finite state machine.
  - calf 4y ago
    Yeah that is a neat hack one of the comments mentioned. Also if a model of computation that is not Turing complete can apparently do English, then what does that say about the computational complexity of English, this piques my curiosity.
gwern 4y ago
Many of the examples are things that a feedforward model like GPT-4 has plenty of capacity for, could do (eg all the real-world facts), and each example helps induce the transition between memorization and generalization so it will do. In some cases like multiplying large numbers, it can't no matter how many examples it's given (with a realistic number, anyway) - but every query firms up the ability to swap out Wolfram Alpha for any other calculator... Wolfram is playing with fire here (https://gwern.net/complement https://gwern.net/complement) given that the output is going to, one way or another, assist training future AI models.