3 ms·

Wouldn’t it have to be continuously trained on new info moment by moment as economic news unfolds? Isn’t training the most computationally expensive part of the

Wouldn’t it have to be continuously trained on new info moment by moment as economic news unfolds? Isn’t training the most computationally expensive part of the whole process? Also, my understanding of LLMs (tiny) means they will only give you the most probable next sequence of words to your prompt based on the words in the training set. Has an LLM ever produced output that wasn’t buried in the training set somewhere? I.e. a novel answer to a novel question?

dragonwriter 4y ago
> Wouldn’t it have to be continuously trained on new info moment by moment as economic news unfolds? It would benefit from access to current news, but not continuous training (though more training is better). > Also, my understanding of LLMs (tiny) means they will only give you the most probable next sequence of words to your prompt based on the words in the training set. Has an LLM ever produced output that wasn’t buried in the training set somewhere? This is a misunderstanding. Its a pattern recognizing predictor, not a Markov chain, its not reproducing its training set, its predicting what a plausible document would look like from the universe from which the training set was sampled would look like if it started with the prompt, and the conpletion cna be vert different from anything in the training set if the prompt is, and (depending on tuning like “temperature”) potentially even if the prompt is exactly represented.
- vba616 4y ago
  >predicting what a plausible document would look like As people have pointed out, but I think should be repeated at every opportunity, this is the definition of bullshit. Superhumanly optimal bullshit. Which has its uses, yes, but it is what it is. Bullshit is not the totality of human existence.
  - Yiin 4y ago
    and yet that bullshit seems to be very valuable and useful, so not sure what is the point you're making?
    - vba616 4y ago
      "Bullshit has its uses" is kind of a way of saying "it may be very valuable and useful, but I don't particularly appreciate it". I feel like every time people equate this sort of thing with human consciousness, it is a way of saying "yes, bullshit is the totality of human existence".

dragonwriter 4y ago

> Wouldn’t it have to be continuously trained on new info moment by moment as economic news unfolds? It would benefit from access to current news, but not continuous training (though more training is better). > Also, my understanding of LLMs (tiny) means they will only give you the most probable next sequence of words to your prompt based on the words in the training set. Has an LLM ever produced output that wasn’t buried in the training set somewhere? This is a misunderstanding. Its a pattern recognizing predictor, not a Markov chain, its not reproducing its training set, its predicting what a plausible document would look like from the universe from which the training set was sampled would look like if it started with the prompt, and the conpletion cna be vert different from anything in the training set if the prompt is, and (depending on tuning like “temperature”) potentially even if the prompt is exactly represented.

vba616 4y ago
>predicting what a plausible document would look like As people have pointed out, but I think should be repeated at every opportunity, this is the definition of bullshit. Superhumanly optimal bullshit. Which has its uses, yes, but it is what it is. Bullshit is not the totality of human existence.
- Yiin 4y ago
  and yet that bullshit seems to be very valuable and useful, so not sure what is the point you're making?
  - vba616 4y ago
    "Bullshit has its uses" is kind of a way of saying "it may be very valuable and useful, but I don't particularly appreciate it". I feel like every time people equate this sort of thing with human consciousness, it is a way of saying "yes, bullshit is the totality of human existence".