3 ms·

Can a generator do things like 2 if it wasn't in the input text?

• lblume 1y ago
  Ambiguity is resolved via provided context, but just as with conversations, this context may be severely underspecified.
• antonvs 1y ago
  Yes, because generators generate at the token level, which is technically smaller than an individual word. They can easily generate unique sentences, and for example transfer learning allows them to apply knowledge obtained from some other training data to new domains. The idea that generators are some sort of parrot is very outdated. The 2021 paper that coined the term "stochastic parrot" was already wrong when it was published.
  • zahlman 1y ago
    > Yes, because generators generate at the token level, which is technically smaller than an individual word. They can easily generate unique sentences, and for example transfer learning allows them to apply knowledge obtained from some other training data to new domains. Sure. But can they read the original author's mind, and therefore generate the right unique sentence that expresses the actual intent?
  • nemomarx 1y ago
    Sure, but in the case of "he finally did X" without context passed in, how does the llm determine if it would be expanded as "this was a very anticipated change" or if the author is frustrated at how long it took? If the nuanced meaning isn't there in the input context. obviously it can generate the longer message, but is it going to go look up what the sentence refers to and infer extra meaning automatically...?