4 ms·
Arithmetic seems like an example where it would help to learn from the real world, not just from text. I learnt to add up by watching my teacher manipulate plas
by dash2 5y ago
Arithmetic seems like an example where it would help to learn from the real world, not just from text. I learnt to add up by watching my teacher manipulate plastic Lego-style blocks. Put 3 blocks with 2 blocks, and you have 1, 2, 3, 4, 5.
- londons_explore 5y agoBut somewhere in that massive corpus of text will be a description just like you've just given...
- tsimionescu 5y agoSure, but GPT-3 doesn't attach semantics to text, it just learns how to produce text patterns that are similar to text patterns it has seen before.
- FeepingCreature 5y ago"Similar" is an inherently semantic property.
- JonChesterfield 5y agoWords with the same length are similar to one another but not well correlated semantically. E.g. bog vs dog.
- tsimionescu 5y agoWould you say that something like string.GetSimilarity(string), which tells you by what number of characters two strings differ, is interpreting the text? Basically what GPT-3 does is to find a string X of a particular length such that it maximizes concat(userInput, X).GetSimilarity(someStringInTrainingSet). Edit: to be clear, I'm not suggesting it's looking up the training set at runtime, X.GetSimilarity(someStringInTrainingSet) is basically what got baked in during training.
- FeepingCreature 5y agoThat is not what GPT does.
- deleted 5y ago[deleted]