3 ms·

> You would realize that all we are doing is using statistics to predict what the next item might be. Two questions: 1. How can humans come up with novel idea

> You would realize that all we are doing is using statistics to predict what the next item might be. Two questions: 1. How can humans come up with novel ideas then? Statistically predicting the next item means I am limited to the training set. How do humans formulate new concepts in this scenario? 2. Why can humans learn from their own output then? If all we do is statistically predicting the next token, then, by necessity, feeding our own output back into the model (aka. letting humans learn from it) should degrade human capabilities by overfitting to the training set. But we observe exactly the opposite.

micromacrofoot 3y ago
> How can humans come up with novel ideas then? hallucinations, emotions > Why can humans learn from their own output then? if we don't, we suffer
jameshart 3y ago
> Statistically predicting the next item means I am limited to the training set Not at all. Extrapolation is perfectly possible in a purely predictive model. It’s one of the things GPTs are best at. In the stream of tokens output by an LLM it’s completely possible for new concepts to emerge, and for it then to continue to use and build on them in the remainder of the stream. You see this simply executed in programming contexts with LLMs where it is able to declare a novel function and then use it.
- usrbinbash 3y ago
  > with LLMs where it is able to declare a novel function and then use it. Novel as in "implements a new algorithm that has never seen in any form and is actually an improvement over existing methodology"? Here is a little thought experiment: If all the training data in the set says that manned, powered flight is impossible, is a statistical prediction engine trained on that data capable of developing an airplane? In other words, could a very powerful stochastic parrot do what the Wright Brothers did? My best guess is that no, it cannot.
  - lovecg 3y ago
    Now I’m wondering if birds didn’t exist how much longer would it take us to catch on that flight is possible at all.
    - jameshart 3y ago
      Birds are just stochastic parrots after all.

micromacrofoot 3y ago

> How can humans come up with novel ideas then? hallucinations, emotions > Why can humans learn from their own output then? if we don't, we suffer

jameshart 3y ago

> Statistically predicting the next item means I am limited to the training set Not at all. Extrapolation is perfectly possible in a purely predictive model. It’s one of the things GPTs are best at. In the stream of tokens output by an LLM it’s completely possible for new concepts to emerge, and for it then to continue to use and build on them in the remainder of the stream. You see this simply executed in programming contexts with LLMs where it is able to declare a novel function and then use it.

usrbinbash 3y ago
> with LLMs where it is able to declare a novel function and then use it. Novel as in "implements a new algorithm that has never seen in any form and is actually an improvement over existing methodology"? Here is a little thought experiment: If all the training data in the set says that manned, powered flight is impossible, is a statistical prediction engine trained on that data capable of developing an airplane? In other words, could a very powerful stochastic parrot do what the Wright Brothers did? My best guess is that no, it cannot.
- lovecg 3y ago
  Now I’m wondering if birds didn’t exist how much longer would it take us to catch on that flight is possible at all.
  - jameshart 3y ago
    Birds are just stochastic parrots after all.