3 ms·
This is basically true, but I think they underrate the improvements between GPT-2 and GPT-3. My mental model is, every once in a while these systems degenerate
by lacker 6y ago
This is basically true, but I think they underrate the improvements between GPT-2 and GPT-3. My mental model is, every once in a while these systems degenerate into surreal non sequitur nonsense. GPT-3 just does it a lot less than GPT-2. It still isn’t good enough to consistently answer casual questions in a human way, but the failure rate is going down, and perhaps straightforward improvements like GPT-4 will be able to fix this without fundamental architectural changes.
- kingkawn 6y ago“every once in a while these systems degenerate into surreal non sequitur nonsense.” Exactly as our minds do
- perl4ever 6y ago>Exactly as our minds do This rhetorically obscures the fact that when humans do produce similar stuff, it's a recognized sort of pathology that is obviously distinct from normal functioning. https://en.wikipedia.org/wiki/Derailment_(thought_disorder) https://en.wikipedia.org/wiki/Derailment_(thought_disorder) Example: "I think someone's infiltrated my copies of the cases. We've got to case the joint. I don't believe in joints, but they do hold your body together." https://en.wikipedia.org/wiki/Word_salad https://en.wikipedia.org/wiki/Word_salad Whatever the difference between this and normal language, call it "X", and whether or not it's amenable to implementing in software in principle, GPT-3 clearly does not have "X" at all. Maybe it would be fruitful to fund study of mental/neurological disorders more, just to understand the mind better.
- beefield 6y ago> when humans do produce similar stuff, it's a recognized sort of pathology Like, legalese? (Sorry, it was too easy a punch)
- deleted 6y ago[deleted]
- inakarmacoma 6y agoor poetry
- DonHopkins 6y agoCould somebody with GPT-3 access please ask it what words come after "person, woman, man, camera"?
- visarga 6y agoQ: What comes after "person, woman, man, camera" A: person, woman, man, camera, lens, light, film, lab, darkroom. A: person, woman, man, camera, dog, cat, horse A: person, woman, man, camera, camera, camera, camera
- perl4ever 6y agoI'm not sure what you would expect as a response. As far as I know, the reference was to a test for dementia in which some words were given at the beginning of the test and asked to be repeated at the end. Perhaps you could provide context. Maybe there were five words, so you could say "Donald Trump was asked to recall five words to test his memory. Four of them were "person, woman, man, camera". What word did he forget?"
- deleted 6y ago[deleted]
- lacker 6y agoI think this is precisely true. The difference is, I usually go days and say thousands of sentences without my words degenerating into nonsense. And usually if I say some nonsense it's because I'm sleepy or drunk, so if I'm trying hard to be not-nonsensical I can reduce my error level to under 0.1%. GPT-3's nonsense-rate is much higher. So, I think it is promising to simply scale up this model and see how much the error rate can be reduced. Perhaps a focus on simple quantitative improvement from here can lead to a much more human performance.