3 ms·
> unlike GPT-3 which might actually genuinely understand parts of language, computer code requires more abstracted mental model Shouldn't computer code be simp
by wongarsu 5y ago
> unlike GPT-3 which might actually genuinely understand parts of language, computer code requires more abstracted mental model
Shouldn't computer code be simpler to understand and express than natural language? The amount of knowledge about the world is much smaller and mostly concerns artificial systems. And of course programming languages are laughably simple and well defined compared to the constantly evolving mess that is the English language.
- js8 5y agoIn programming languages, it is much more common to define new words (symbols) than in natural language. The identifier (name) of the symbol (word) in such case is less informative about its nature than the actual definition, and I think this is the Achilles heel of the method.
- lambdaba 5y agoYes, programming languages are simple, programs are infinitely complex, they could express anything, and homoiconic languages can even define their own syntax.
- js8 5y agoAbsolutely. These days, I am exploring a concatenative language, Factor. I feel like if there ever was a consistent syntactic pattern in this language, then it would make sense to define it as a new word, rather than have a system which would repeatedly suggest it based on partial match. And that's what I am alluding to with my entropy comment. Factor's syntax has extremely high entropy (if we look at words as tokens), or rather, it is designed to allow extremely high entropy of code. Ultimately, I think it comes down to preference, what the tools should be doing for us programmers? Should the program source code (language syntax) have low entropy, and the tools should exploit that when we write the code (as Copilot does), or should we strive for as high entropy of the source code as possible, and have the tools explain the context for us (for example, in Factor, it would be what the expected types on the stack are in the certain place in the code, etc.)? I am in favor of the 2nd approach, I think that is going to be more productive.
- lambdaba 5y agoI agree, programs are and will always remain first for humans to read, which means they have to follow the shape of thought and not the other way around.
- lambdaba 5y agoWhat would it mean to "understand" it. Imagine a program written by such a system, it's easy to imagine an immensely complex question about the system that sounds simple, for instance asking about design choices, code architecture, aesthetic choices like expressivity. Even if it could produce some code, we couldn't say it understood it if it couldn't answer those questions, and those are very difficult questions which would require human-level intelligence, and even more a human psyche.
- throwawaygh 5y ago> Shouldn't computer code be simpler to understand and express than natural language? Yes, but the ideas that are being expressed in formal languages are much more complex and detailed than the ideas that are being expressed in natural language. Writing some plausibly on-topic blog posts is way easier than writing even a very simple program, especially if you don't really know what you're talking about.