3 ms·
How is it completing the task? The only possible way to perform that task reliably is to have a principled understanding of what the code actually does, and to
by cjbprime 4y ago
How is it completing the task? The only possible way to perform that task reliably is to have a principled understanding of what the code actually does, and to create a world model based on that understanding, and then extract consequences from that world model.
You can reassure yourself that GPT-4 is actually doing this by asking it to share its modeling of the code with you.
How is it able to do that? By being so grand in scale that it does not merely model the likelihood of a token based on the text it's seen before, but on the causes of that text having been written in the first place. It was a fallacy to expect that it would not do this if the parameters and dataset were sufficiently large. It's a fallacy that is still happening, including in popular New Yorker articles.
- strange_quark 4y agoIt’s also read probably countless CS textbooks and tutorials, all of StackOverflow, all of GitHub, etc. I think it’s really significantly less of a stretch that to state that it can do what you’re describing because a lot of code is similar and it’s seen gigabytes of it, rather than state it somehow learned how to reason in the human sense.
- cjbprime 4y agoTo be clear, this is not "what does this line of code do?", but "what does this many-line program do?". Even if you've seen all the lines individually before, you still have to work out what happens when you combine them together. If most of the combination was not in the training set, that's reasoning to me.
- mdp2021 4y agoBut it remains that we want to see how this works precisely enough to make it work reliably. (And "possibly" to learn from that - because, you know, otherwise we have a tool but no knowledge. Also consistently with the example of the fire.)