3 ms·

I'm not even remotely worried for my job security, but I don't think the ELIZA comparison is apt. ELIZA is clearly a toy, Copilot is something that could becom

by qsort 5y ago

I'm not even remotely worried for my job security, but I don't think the ELIZA comparison is apt. ELIZA is clearly a toy, Copilot is something that could become integrated with the workflow of a professional developer. Maybe not copilot specifically, but if Intellisense is in our IDEs already, something like it is not a stretch. > I don't think it really understands code It clearly doesn't, but (a) nobody actually claimed that outside of the usual media circus, and (b) that's a very high bar. It doesn't need to understand your code in order to be useful.

js8 5y ago
ELIZA wasn't "clearly a toy" when it appeared. It actually confused people. I don't deny that good suggestions are useful in coding. I am just not sure you can get these by mimicking other people's code without really understanding it. I think the reason why code is different than natural language is that in code, the meaning of identifiers (words) is very dependent on the context of the specific codebase, and not nearly as universal as in natural language. Sure, there are words like personal names in natural language that change meaning depending on wider context, but they are not as common in the natural text. So focusing on the names only, rather than for example understanding types of runtime objects/values, and how they change, can be actually actively harmful in trying to understand the code and making good suggestions. So I would believe that suggestions based on e.g. type inference would be more useful. There is another aspect to this, text has inherently low entropy, it's somewhat easy to predict the next symbol. But if code has low entropy (i.e. there are universal patterns that can be applied to any codebase in a particular programming language), I would argue that this is inefficient from the programmer's point of view, because this entropy could probably be abstracted away to a more condensed representation (which could be then intelligently suggested based on semantics rather than syntax). In other words, I think the approach assumes that the probability of the next word, based on syntax, is similar to probability of the next word, based on semantics, but I don't believe that's the case in programming languages.
ska 5y ago
> ELIZA is clearly a toy, That's a red herring. The "Eliza effect" has nothing to do with how toy-like ELIZA actual was, it is a lesson about how humans were very bad at evaluating its actual capability and insight. This continues to be true today, and is an effect with at least some impact on essentially any AI/ML system that people interact with.
feoren 5y ago
Copilot is clearly a toy.
- strken 5y ago
  It's closer to a search engine. It's not competing with developers, it's competing with opening up your browser and searching for "astar algorithm implementation."

by qsort 5y ago

js8 5y ago

ELIZA wasn't "clearly a toy" when it appeared. It actually confused people. I don't deny that good suggestions are useful in coding. I am just not sure you can get these by mimicking other people's code without really understanding it. I think the reason why code is different than natural language is that in code, the meaning of identifiers (words) is very dependent on the context of the specific codebase, and not nearly as universal as in natural language. Sure, there are words like personal names in natural language that change meaning depending on wider context, but they are not as common in the natural text. So focusing on the names only, rather than for example understanding types of runtime objects/values, and how they change, can be actually actively harmful in trying to understand the code and making good suggestions. So I would believe that suggestions based on e.g. type inference would be more useful. There is another aspect to this, text has inherently low entropy, it's somewhat easy to predict the next symbol. But if code has low entropy (i.e. there are universal patterns that can be applied to any codebase in a particular programming language), I would argue that this is inefficient from the programmer's point of view, because this entropy could probably be abstracted away to a more condensed representation (which could be then intelligently suggested based on semantics rather than syntax). In other words, I think the approach assumes that the probability of the next word, based on syntax, is similar to probability of the next word, based on semantics, but I don't believe that's the case in programming languages.

ska 5y ago

> ELIZA is clearly a toy, That's a red herring. The "Eliza effect" has nothing to do with how toy-like ELIZA actual was, it is a lesson about how humans were very bad at evaluating its actual capability and insight. This continues to be true today, and is an effect with at least some impact on essentially any AI/ML system that people interact with.

feoren 5y ago

Copilot is clearly a toy.

strken 5y ago
It's closer to a search engine. It's not competing with developers, it's competing with opening up your browser and searching for "astar algorithm implementation."