9 ms·
I don't think the problem is really with the data set, it's that GTP-3 doesn't actually have any understanding of the data. It's building a model of the text i
by simonh 6y ago
I don't think the problem is really with the data set, it's that GTP-3 doesn't actually have any understanding of the data. It's building a model of the text input to generate text output. It's not building a model of what the data means, or what it represents.
When GTP-3 writes a scientific paper it's not trying to test a premise or critically evaluate some data, it's trying to generate text that looks like that sort of thing.
It doesn't matter how many excellent quality papers you fed it, or how stringently you excluded low quality input data, it would still only be trying to produce facsimiles. It wouldn't be actually trying to do the things a real conscientious scientist is trying to do when they write a paper. Arguably at best it might be trying to do what a deceitful scientist trying to get credit for a paper with spurious results with no actual scientific merit might be doing when writing a paper that looks plausible, but thats actually a completely different activity.
The code generation examples are really interesting. Here it's being used to generate real working code that has actual value, and it appears to work pretty well. It's code often has bugs, but who's doesn't? It only works for fairly short precisely definable coding tasks though. I don't think scaling it up to more complex coding problems is going to work. Again it doesn't understand the meaning of anything. It's trying to produce code that looks like working code, not actually solve the programming task you're giving it. It doesn't even know what a program is or what a programming task is. It doesn't know what input and output are. For example you can't ask it to modify existing code to change it's behaviour. It doesn't know code has behaviour. It has no idea what that even means and has no way to find out or any route to gaining that capability, because that's not a text transformation task and all it does is transform text.
- SpicyLemonZest 6y agoHow can we tell whether or not GPT-3 has understanding of the data?
- deleted 6y ago[deleted]
- grugagag 6y agoI think the best way to tell whether GPT-3 has understanding of its data is by by asking questions related to the data but not explicit in the training dataset.
- polytely 6y agoMaybe by asking it to clean up it's own training data?
- mike_ivanov 6y agoBy asking counterfactual questions.
- stupidcar 6y agoI've seen this objection raised a lot, but I think it betrays a misunderstanding of what GPT-3 is capable of doing. The best, in fact the only way to generate truly convincing text output on most subjects is to understand, on some level, what you're writing about. In other words, to create a higher level abstraction than simply "statistically speaking, this word seems to follow that one". Once you start to encode that words map to concepts, you can use the resulting conceptual model to create output which is conceptually consistent, then map it backwards to words. There is what humans do with sensory data, and there is good evidence that GPT-3 is doing this too, to some degree. Take simple arithmetic, such as adding two and three digit numbers. GPT-2 could not do this very successfully. It did indeed look like it was treating it as a "find the textual pattern" problem. But GPT-3 is much more successful, including at giving correct answers to arithmetic problems that weren't in its training set. So what changed? We aren't sure, but the speculation is that in the process of training, GPT-3 found that the best strategy to correctly predicting the continuation of arithmetic expressions was to figure out the rules of basic arithmetic and encode them in some portion of its neural network, then apply them whenever the prompt suggested to do so. If this is the case, and it remains speculation at this point, would you still argue that GPT-3 doesn't "understand" arithmetic, on some level? I would argue that this abstraction, this mapping of words onto higher-level concepts, which can then be manipulated to solve more complex problems, is exactly what intelligence is, once you strip away biologically-biased assumptions. Certainly, at this point GPT-3's conceptual understanding remains somewhat primitive and unstable, but the fact that it exhibits it at all, and sometimes in spookily impressive ways, is what has people excited and worried. We have produced AIs that can perhaps think conceptually about relatively narrow topics like playing Go, but we have never before created one that can do so one such a wide range of topics. And there is no suggestion that GPT-3's level of ability represents a maximum. GPT-4 and beyond will be more powerful, meaning that it can mine more and more powerful conceptual understanding from their training data.
- simonh 6y agoThe example of simple arithmetic is interesting. I think you might be right, that is evidence that GPT-3 might be generating what we might consider models of it's input data. Very simple, primitive and fragile models, but yes that's a start. Thank you.
- johnc1 6y agoAnother way to think about it is comparing to how children learn. First, children spend inordinate amount of time just trying to make sense of words they hear. Once they develop their language models, adults can explain new concepts to them using the language. What'd be really exciting is being able to explain a new concept to GPT-n in words, and have it draw conclusions from it. Few-shots learning is a tiny step in that direction.
- tsimionescu 6y agoChildren don't spend inordinate amounts of time learning words. In fact, past the first months, children often learn words from hearing them a single time.
- coryfklein 6y agoI have a 4, 6, and 8 year old, and each of them are still learning words. Yeah they don’t spend 80% of each day learning words, but building up their vocabulary legit takes a looong time.
- tsimionescu 6y agoOh, absolutely. I'm 31 and I'm still learning words! But I don't think I've ever spent time to learn a particular word - it's almost always enough to hear it in context once, and maybe get a chance to actually use it yourself once or twice, and you'll probably remember it for life. If it's a word for a more complex concept (e.g. some mathematical construct), you may well need more time to actually understand the meaning, and you may also pretty easily forget the meaning in time, but you'll likely not forget the word itself.
- johnc1 6y agoThat's of course correct. Perhaps GPT-3 can do that too? I don't have access to it, but I wonder if it can be taught new words using few-shot learning. In fact, even GPT-2 gets close to that. Here's what I just got on Huggingface's Write With Transformer: Prompt: "Word dfjgasdjf means happiness. What is dfjgasdjf?" GPT-2: "dfjgasdjf is a very special word that you can use to express happiness, love or joy." What takes time is all the learning a child needs to go through before they can be taught new words on the spot.
- bo1024 6y agoI think this is a very good take.