4 ms·

I think Copilot is as helpful to programmers as ELIZA is helpful to psychiatrists. I don't think it really understands code, only very superficially (unlike GP

by js8 5y ago

I think Copilot is as helpful to programmers as ELIZA is helpful to psychiatrists. I don't think it really understands code, only very superficially (unlike GPT-3 which might actually genuinely understand parts of language, computer code requires more abstracted mental model). ELIZA was similar, some people also believed that it can understand what they are thinking. Turns out it was all just in our heads.

qsort 5y ago
I'm not even remotely worried for my job security, but I don't think the ELIZA comparison is apt. ELIZA is clearly a toy, Copilot is something that could become integrated with the workflow of a professional developer. Maybe not copilot specifically, but if Intellisense is in our IDEs already, something like it is not a stretch. > I don't think it really understands code It clearly doesn't, but (a) nobody actually claimed that outside of the usual media circus, and (b) that's a very high bar. It doesn't need to understand your code in order to be useful.
- js8 5y ago
  ELIZA wasn't "clearly a toy" when it appeared. It actually confused people. I don't deny that good suggestions are useful in coding. I am just not sure you can get these by mimicking other people's code without really understanding it. I think the reason why code is different than natural language is that in code, the meaning of identifiers (words) is very dependent on the context of the specific codebase, and not nearly as universal as in natural language. Sure, there are words like personal names in natural language that change meaning depending on wider context, but they are not as common in the natural text. So focusing on the names only, rather than for example understanding types of runtime objects/values, and how they change, can be actually actively harmful in trying to understand the code and making good suggestions. So I would believe that suggestions based on e.g. type inference would be more useful. There is another aspect to this, text has inherently low entropy, it's somewhat easy to predict the next symbol. But if code has low entropy (i.e. there are universal patterns that can be applied to any codebase in a particular programming language), I would argue that this is inefficient from the programmer's point of view, because this entropy could probably be abstracted away to a more condensed representation (which could be then intelligently suggested based on semantics rather than syntax). In other words, I think the approach assumes that the probability of the next word, based on syntax, is similar to probability of the next word, based on semantics, but I don't believe that's the case in programming languages.
- ska 5y ago
  > ELIZA is clearly a toy, That's a red herring. The "Eliza effect" has nothing to do with how toy-like ELIZA actual was, it is a lesson about how humans were very bad at evaluating its actual capability and insight. This continues to be true today, and is an effect with at least some impact on essentially any AI/ML system that people interact with.
- feoren 5y ago
  Copilot is clearly a toy.
  - strken 5y ago
    It's closer to a search engine. It's not competing with developers, it's competing with opening up your browser and searching for "astar algorithm implementation."
wongarsu 5y ago
> unlike GPT-3 which might actually genuinely understand parts of language, computer code requires more abstracted mental model Shouldn't computer code be simpler to understand and express than natural language? The amount of knowledge about the world is much smaller and mostly concerns artificial systems. And of course programming languages are laughably simple and well defined compared to the constantly evolving mess that is the English language.
- js8 5y ago
  In programming languages, it is much more common to define new words (symbols) than in natural language. The identifier (name) of the symbol (word) in such case is less informative about its nature than the actual definition, and I think this is the Achilles heel of the method.
  - lambdaba 5y ago
    Yes, programming languages are simple, programs are infinitely complex, they could express anything, and homoiconic languages can even define their own syntax.
    - js8 5y ago
      Absolutely. These days, I am exploring a concatenative language, Factor. I feel like if there ever was a consistent syntactic pattern in this language, then it would make sense to define it as a new word, rather than have a system which would repeatedly suggest it based on partial match. And that's what I am alluding to with my entropy comment. Factor's syntax has extremely high entropy (if we look at words as tokens), or rather, it is designed to allow extremely high entropy of code. Ultimately, I think it comes down to preference, what the tools should be doing for us programmers? Should the program source code (language syntax) have low entropy, and the tools should exploit that when we write the code (as Copilot does), or should we strive for as high entropy of the source code as possible, and have the tools explain the context for us (for example, in Factor, it would be what the expected types on the stack are in the certain place in the code, etc.)? I am in favor of the 2nd approach, I think that is going to be more productive.
falsaberN1 5y ago
I have played with GPT3 and similar lesser models for a while, and in my opinion it doesn't understand anything. It's good at correlation (it might figure out a pikachu is a pokemon or that The Wall is a music album) which is kinda amazing but ultimately seems to be really "useless"* and it's not deterministic. I think the current "AI" models will never understand a thing unless they become able to update their own models and have some guiding logic to them. Sure it has the context tokens so it might give a little illusion of following along, but the randomness of it all also means it will get wrong stuff that it got right before (and vice versa) and will clearly break the illusion when prodded ("pikachu is a purple dragon", "The Wall was composed by Metallica"). It's unfortunate. GPT3 and other text models have no sense of state. You can have the same instance work with 2 (or more) different persons and text provided by user A will have no bearing on anything user B does, since it's just trying to complete the provided text in form of tokens. As computationally intensive as it is, the whole thing is a lot simpler than it seems. It's like we managed to get the "learning" part of intelligence more or less right, but nothing else yet. I always wanted to have an AI "trusted assistant" kinda thing, but even the biggest GPT3 models don't give me much in terms of trust. Wouldn't trust it to turn a lamp on and off as it is. (I know GPT3 is just a text model but you know what I mean). And if it happens, it won't be run locally because the hardware requisites for this stuff are obscene on a good day, which opens a few cans of worms (some of which have been opened already). *useless as in getting anywhere close to intelligence. It can become decent entertainment and might have some interesting uses to correlate datum A to datum B/C/D, recognition and so, but so far it's just fancy ELIZA. An "educated guess engine" at best.

by js8 5y ago

qsort 5y ago

I'm not even remotely worried for my job security, but I don't think the ELIZA comparison is apt. ELIZA is clearly a toy, Copilot is something that could become integrated with the workflow of a professional developer. Maybe not copilot specifically, but if Intellisense is in our IDEs already, something like it is not a stretch. > I don't think it really understands code It clearly doesn't, but (a) nobody actually claimed that outside of the usual media circus, and (b) that's a very high bar. It doesn't need to understand your code in order to be useful.

js8 5y ago
ELIZA wasn't "clearly a toy" when it appeared. It actually confused people. I don't deny that good suggestions are useful in coding. I am just not sure you can get these by mimicking other people's code without really understanding it. I think the reason why code is different than natural language is that in code, the meaning of identifiers (words) is very dependent on the context of the specific codebase, and not nearly as universal as in natural language. Sure, there are words like personal names in natural language that change meaning depending on wider context, but they are not as common in the natural text. So focusing on the names only, rather than for example understanding types of runtime objects/values, and how they change, can be actually actively harmful in trying to understand the code and making good suggestions. So I would believe that suggestions based on e.g. type inference would be more useful. There is another aspect to this, text has inherently low entropy, it's somewhat easy to predict the next symbol. But if code has low entropy (i.e. there are universal patterns that can be applied to any codebase in a particular programming language), I would argue that this is inefficient from the programmer's point of view, because this entropy could probably be abstracted away to a more condensed representation (which could be then intelligently suggested based on semantics rather than syntax). In other words, I think the approach assumes that the probability of the next word, based on syntax, is similar to probability of the next word, based on semantics, but I don't believe that's the case in programming languages.
ska 5y ago
> ELIZA is clearly a toy, That's a red herring. The "Eliza effect" has nothing to do with how toy-like ELIZA actual was, it is a lesson about how humans were very bad at evaluating its actual capability and insight. This continues to be true today, and is an effect with at least some impact on essentially any AI/ML system that people interact with.
feoren 5y ago
Copilot is clearly a toy.
- strken 5y ago
  It's closer to a search engine. It's not competing with developers, it's competing with opening up your browser and searching for "astar algorithm implementation."

wongarsu 5y ago

> unlike GPT-3 which might actually genuinely understand parts of language, computer code requires more abstracted mental model Shouldn't computer code be simpler to understand and express than natural language? The amount of knowledge about the world is much smaller and mostly concerns artificial systems. And of course programming languages are laughably simple and well defined compared to the constantly evolving mess that is the English language.

js8 5y ago
In programming languages, it is much more common to define new words (symbols) than in natural language. The identifier (name) of the symbol (word) in such case is less informative about its nature than the actual definition, and I think this is the Achilles heel of the method.
- lambdaba 5y ago
  Yes, programming languages are simple, programs are infinitely complex, they could express anything, and homoiconic languages can even define their own syntax.
  - js8 5y ago
    Absolutely. These days, I am exploring a concatenative language, Factor. I feel like if there ever was a consistent syntactic pattern in this language, then it would make sense to define it as a new word, rather than have a system which would repeatedly suggest it based on partial match. And that's what I am alluding to with my entropy comment. Factor's syntax has extremely high entropy (if we look at words as tokens), or rather, it is designed to allow extremely high entropy of code. Ultimately, I think it comes down to preference, what the tools should be doing for us programmers? Should the program source code (language syntax) have low entropy, and the tools should exploit that when we write the code (as Copilot does), or should we strive for as high entropy of the source code as possible, and have the tools explain the context for us (for example, in Factor, it would be what the expected types on the stack are in the certain place in the code, etc.)? I am in favor of the 2nd approach, I think that is going to be more productive.

falsaberN1 5y ago

I have played with GPT3 and similar lesser models for a while, and in my opinion it doesn't understand anything. It's good at correlation (it might figure out a pikachu is a pokemon or that The Wall is a music album) which is kinda amazing but ultimately seems to be really "useless"* and it's not deterministic. I think the current "AI" models will never understand a thing unless they become able to update their own models and have some guiding logic to them. Sure it has the context tokens so it might give a little illusion of following along, but the randomness of it all also means it will get wrong stuff that it got right before (and vice versa) and will clearly break the illusion when prodded ("pikachu is a purple dragon", "The Wall was composed by Metallica"). It's unfortunate. GPT3 and other text models have no sense of state. You can have the same instance work with 2 (or more) different persons and text provided by user A will have no bearing on anything user B does, since it's just trying to complete the provided text in form of tokens. As computationally intensive as it is, the whole thing is a lot simpler than it seems. It's like we managed to get the "learning" part of intelligence more or less right, but nothing else yet. I always wanted to have an AI "trusted assistant" kinda thing, but even the biggest GPT3 models don't give me much in terms of trust. Wouldn't trust it to turn a lamp on and off as it is. (I know GPT3 is just a text model but you know what I mean). And if it happens, it won't be run locally because the hardware requisites for this stuff are obscene on a good day, which opens a few cans of worms (some of which have been opened already). *useless as in getting anywhere close to intelligence. It can become decent entertainment and might have some interesting uses to correlate datum A to datum B/C/D, recognition and so, but so far it's just fancy ELIZA. An "educated guess engine" at best.