4 ms·
GPT3 and its cousins do things that no previous language model could do; it is qualitatively different from Eliza in its capabilities. As for your argument abou
by canjobear 5y ago
GPT3 and its cousins do things that no previous language model could do; it is qualitatively different from Eliza in its capabilities. As for your argument about random projections in the evaluation of GLoVE, comparisons with random projections are now routine. See for example https://aclanthology.org/N19-1419/ https://aclanthology.org/N19-1419/
- NoGravitas 5y agoWhy do you say it is qualitatively different from Eliza in its capabilities?
- PaulHoule 5y agoIt does something totally different. However that totally different still depends on people being desperate to see intelligence inside it. It’s like how people see a face in a cut stem or on Mars.
- NoGravitas 5y agoIs it because it does something totally different that you came to me?
- rytill 5y agoYou're trying to prove some kind of point where you respond as ELIZA would have to show how "even back then we could pass for conversation". The truth is that GPT-3 is actually, totally qualitatively different and if you played with it enough you'd realize.
- not2b 5y agoThe difference is quantitative, rather than qualitative, as compared to primitive Markov models that have been used in the past. It's just a numerical model with a very large number of parameters that extends a text token sequence. The parameter size is so large that it has in essence memorized its training data, so if the right answer was already present in the training data you'll get it, same if the answer is closely related to the training data in a way that lets the model predict it. If the wrong answer was present in the training data you may well get that.
- canjobear 5y agoAnd ice is the same thing as water, just at a quantitively different temperature.
- perl4ever 5y agoThere's a famous saying "quantity has a quality all its own". If GPT-3 is known to have some sort of intelligence, I think it logically follows that one can differentiate that from the reflected intelligence of all the humans that produced all the data it ingested. How do you see intelligence in GPT-3 and not see it in the data fed to it?
- NoGravitas 5y agoDoes it bother you that I'm trying to prove some kind of point?
- canjobear 5y agoWhat is your criterion for "truly" detecting intelligence? Do you have a test in mind that would succeed for humans and fail for GPT3?