3 ms·
But somewhere in that massive corpus of text will be a description just like you've just given...
by londons_explore 5y ago
But somewhere in that massive corpus of text will be a description just like you've just given...
- tsimionescu 5y agoSure, but GPT-3 doesn't attach semantics to text, it just learns how to produce text patterns that are similar to text patterns it has seen before.
- FeepingCreature 5y ago"Similar" is an inherently semantic property.
- JonChesterfield 5y agoWords with the same length are similar to one another but not well correlated semantically. E.g. bog vs dog.
- tsimionescu 5y agoWould you say that something like string.GetSimilarity(string), which tells you by what number of characters two strings differ, is interpreting the text? Basically what GPT-3 does is to find a string X of a particular length such that it maximizes concat(userInput, X).GetSimilarity(someStringInTrainingSet). Edit: to be clear, I'm not suggesting it's looking up the training set at runtime, X.GetSimilarity(someStringInTrainingSet) is basically what got baked in during training.
- FeepingCreature 5y agoThat is not what GPT does.
- deleted 5y ago[deleted]