2 ms·
>there was simply text that is statistically likely to be arranged in that way This is wrong. An LLM can produce text that has never been arranged that way in
by danielmarkbruce 2y ago
>there was simply text that is statistically likely to be arranged in that way
This is wrong. An LLM can produce text that has never been arranged that way in training data.
- lolinder 2y agoYour statement doesn't contradict their statement. It produces text that is statistically likely to be arranged that way, but because we use non-deterministic sampling we get a wide variety of results that weren't necessarily in the training data.
- danielmarkbruce 2y agoNo, I don't think they are talking about the decoding process there. If they were, it's a non statement. They are almost certainly talking about training data.