3 ms·
How do you know you are not also picking the next word at random from a probability distribution? I mean, as I'm typing this sentence, I have some notion about
by cecilpl2 5y ago
How do you know you are not also picking the next word at random from a probability distribution?
I mean, as I'm typing this sentence, I have some notion about which word is going to come next, and a sense of where the sentence is going to go.
But maybe that is what it feels like to be a large neural network that has been trained on very large data sets for long periods of time.
- skybrian 5y agoOne skill we do have is the ability to realize that what we said is not what we meant, and make a correction. This is more goal-oriented than what a transformer is doing. Its only goal is to continue onward in a plausible way and it could be in any direction depending on a random choice. It doesn't care which way it goes.
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]