30 ms·

Is GitHub Copilot a blessing, or a curse?

170 points by jph00 5y ago

drpixie 5y ago
Now, imagine you're sitting with your boss, the average typically incompetent project manager or lead programmer ... Them: type type type ... accept accept ... save (unaware or choosing not to understand the meaningless uncompilable garbage that's been generated) Them: "There, that's the xxxx function. You'll have the rest done in 2 days, won't you!"
- da39a3ee 5y ago
  Agree with your sentiment. But there is a solution: You: “Great, that might be good. Please stay here while we write and run the tests for that. (Our engineering handbook emphasizes how important tests are, as you know.)”
robertwt7 5y ago
Feels like its a magical thing for me honestly. although I'm never waiting for suggestion while coding, sometimes the suggestions just slipped through and there is the OMG moment when it completes your thought. 1 thing though, tab for auto complete still does not work for me.. ever
- da39a3ee 5y ago
  Yeah tab didn’t work for me either. But then I turned off tab for autocomplete in settings and that seemed to fix it. I didn’t understand, but try it.
- spookyuser 5y ago
  Same, I've enjoyed using it in a couple of one off scripts and honestly it made writing them more enjoyable because I was kind of excited to see what copilot would do. I asked it to write a js function to clean special characters from a string and it worked perfectly, I think, well it worked perfectly enough for the script to complete. I can't speak to whether or not using copilot in long lasting projects would be a good idea, but even if you just tagged the copilot generated code with a comment that said '#TODO: Verify this is correct' I still think it would help you get a lot of work done that you would normally have to switch to stackoverflow to do. One thing that I found very interesting from the article was the idea of anchoring bias, I definitely felt this a lot. When I used copilot to generate some nontrivial functions that were directly related to my problem, if the code did not work and I ended up erasing the generated code completely, I am just realising now that when I reimplemented it myself I was still essentially using the algorithm copilot had come up with.
NegativeLatency 5y ago
Well if they ever get it right I guess I could always become an electrician (this is not a dig on electricians, it seems like a fun trade, that or under water welding)
- bigyikes 5y ago
  Yeah, the real curse is on the programmers who are going to be out of jobs much sooner than people anticipate. All the talk about the low quality of code got me thinking: if humans aren’t reading the code, then the only thing that matters is correctness. Correctness may be an easier problem to solve than readability and refactorability.
  - methodin 5y ago
    They are intertwined, no? All the training inputs would be based on the code that was affected by the aforementioned traits.
  - robbedpeter 5y ago
    Media synthesis means artists of all sorts will take a hit. Gpt means writers, programmers, journalists, and others will take a hit. Prompt engineering and larger models could mean any procedural use of knowledge in any context could be subject to automation. Consumer level robotics, drones, and battery tech is close to a level that manual labor, delivery, and such tasks can be automated. Maybe we should be preparing for post-scarcity. At the very least, the idea of "jobs" is going to have to transform. The 40 hour work week isn't going to make much sense in the near future.
    - onion2k 5y ago
      Copilot (and all other AI tools, and non-AI tools) will lead to more developer jobs, not less. The thing holding most industries back at the moment is the lack of available developers at a cost-effective price. Literally every business has an opportunity to automate some process better than it does now, or to make an app to assist a business function, or to drive sales and growth through an application of new tech. The reason why they don't do that is cost and availability of skilled people to do the work. If a new AI tool removes the need to write code entirely and a developer just needs to work out the specifications and requirements then whole industries will start employing more developers. The part of dev that you get paid to do is not writing the code. It's the part where you define what the code should do. That's where all the value in development resides. Getting AI to write the syntax is 100% upside for the tech industry.
habibur 5y ago
His conclusion to the question in title : "I don't know."
eminence32 5y ago
> The code Copilot writes is not very good code. This has been my experience as well. Some of the early promo material for Copilot showed someone writing a function signature with docs, and having Copilot write the entire body. This rarely works for me, except for fairly trivial functions. This is also not how I normally write code. However, where Copilot has been rather good is offering intelligent "tab completion", where it makes a suggestion for the next line of code. In particular, it seems very good at figuring out the structure of my code and making sensible suggestions. I suspect that if anyone ends up using Copilot for the long term, they will (as a human operator) need to learn and gain intuition about how to use Copilot's strengths and avoid its weaknesses.
- skohan 5y ago
  Copilot is also almost certainly going to become better. For example, NVIDIA's DLSS was viewed as a failure at launch, but now it's almost magic in terms of offering better results at a lower performance cost.
  - outsomnia 5y ago
    Better at issuing haphazardly modified boilerplate that already exists... the real costs of writing code are about carefully choosing how pieces will fit together and deciding on minutae with that in mind. Copilot is the antithesis of designing good software.
    - skohan 5y ago
      AI is getting better than humans at a lot of complex task over time. Just because it can't code well now doesn't mean it won't in the future.
      - krageon 5y ago
        What it means that it can't code well, so there's nothing to be excited about. That we cannot predict the future is of course true but really has no bearing on what you're saying, except to lend credence to the idea that it will be able to some day (i.e. that you can predict the future). Which is in essence the opposite of what "we cannot predict the future" should mean. That makes the entire argument kind of disingenuous.
foxes 5y ago
How does copilot fair in a language where correctness or at least well soundness is more transparent, for example Haskell (eg at least making sure all the types are correct, actual logic on the other hand seems harder, but maybe the declarative style helps)? I imagine in a less strict language you can generate garbage more easily.
- toomanyducks 5y ago
  Some of the examples given wouldn't even run, as far as I know. Copilot writing Haskell/Rust/<insert your intricate type system lang here> would probably just fail to compile. I know very, very little about AI, but, like the OP said, it'll reproduce the traits of its training distribution. If the traits don't map on to the domain-specifics of the type ssytem, the traits don't map on, and the code fails. Generating garbage the compiles is harder, but generating garbage in general is just as easy.
  - truth_ 5y ago
    Yeah, I chanced upon an official livestream by GH showcasing Copilot, and it was performing very poorly on Rust.
- exdsq 5y ago
  Building on this idea, if you could design a type-level program with Haskell (including some liquid Haskell to add further type invariants) it'd be awesome for something like Copilot to fill in the blanks. Seems easier than program synthesis!
croes 5y ago
It will depend on the companies and the price of Copilot. Then they will replace expensive experienced programmers with cheaper inexperienced ones and Copilot.
- elevenoh 5y ago
  hmmm, remains to be seen! might just keep the experienced/quality ones & have less need for a high quantity of inexperienced programmers.
mbgerring 5y ago
So far it's been worse than useless for me, the suggestions it offers don't even make sense. The existing code completion suggestions in VSCode for TypeScript are better.
minimaxir 5y ago
Whenever a new generative language model is released and discussed on Hacker News, there's always the comment that "it's worthless because it doesn't always generate coherent text" which I've always disliked because it's unfairly reductive as progress in this field is iterative. However, in the case of Copilot, there has to be a much higher signal-to-noise ratio in terms of generated text than creative language models like GPT-2/GPT-3 to use it as a pair programmer where the overhead in working with the model is worth the time saved via the model.
- Barrin92 5y ago
  > it's unfairly reductive as progress in this field is iterative. is that true though? Progress in natural(!) language processing is very iterative, but coding and code correctness is a very precise thing. Both semantically as well as literally. Code that's a 'little bit' wrong technically does not compile, and code that's a 'little bit' wrong semantically might produce catastrophic results. Engineering doesn't become a natural language problem just because you throw a natural language model at it.
  - sanxiyn 5y ago
    Natural language syntax is precise too. It's not like it's personal idiosyncratic preference. People will mostly agree whether something is syntactically correct or incorrect.
ALittleLight 5y ago
There's an interesting section that talks about what you want from a pair programmer - questioning your assumptions, spotting errors, debating design with you, etc, and how Copilot doesn't do this, but instead just spits out the code it thinks you want. That made me think that, when using Copilot, the human is actually the copilot. You're the one spotting bugs, picking different designs, and so on. I also think the discussion around searching versus autocomplete is interesting. What we need is something that autogenerates a Google search based on your code context and shows you some promising results. I'm imagining a pane in VS Code getting populated with Stack Overflow answers whenever I slow down.
- carschno 5y ago
  Something like the Stackoverflow Importer for Python? https://github.com/drathier/stack-overflow-import https://github.com/drathier/stack-overflow-import
  - wildbook 5y ago
    It exists: https://github.com/MythicManiac/copilot-import https://github.com/MythicManiac/copilot-import
- jimmygrapes 5y ago
  100% agreed; the copilot is human.
- seumars 5y ago
  For what it’s worth, Copilot is not being marketed as anything else than code suggestion software - an alternative to google search, perhaps more of a “I’m feeling lucky” without having to actually type the search term. Knowing how to succinctly express a problem is half the problem after all.
  - ALittleLight 5y ago
    It's called "Copilot" and the tagline is "Your AI pair programmer." It's obviously being marketed as more than code suggestion software. It's being marketed as an AI pair programmer. I agree that it is basically just code suggestion software, or like a big autocomplete. But it's definitely being marketed as more.
imvetri 5y ago
Let your product owners capture the requirements in GHERKIN, feed it to copilot. Sit back and relax.
ignu 5y ago
Writing code is, uh, not remotely the hardest part of software development. Maintaining it is.
- grammarnazzzi 5y ago
  I agree. The challenging part of coding is converting domain knowledge into correct code. An AI has no high-level understanding of the domain and desired solution. It just guesses at what you might want and glosses over all the edge cases and exceptions that actually need to be considered and resolved. The issues that only come to light after a real intelligence studies the problem, and works through correct steps to implement a solution. It's an iterative process, because the solution is arrived at only after attempting a solution, learning what you don't know or haven't considered about the problem, then resolving those ambiguities, and producing a revised and more correct solution. When responsible programmers realize that they don't know what they don't know, they ask questions and apply the answers. AI never realizes when it makes mistakes because it has no domain knowledge. It just pulls a best guess out it's ass and says, "Here you go. Check my work."
- UncleMeat 5y ago
  That's why everybody programs in base notepad, with no syntax highlighting or auto complete or live error notifications or linting or any other quality of life feature. What you say here is true. Writing code is not remotely the hardest part of software engineering. But that does not mean that there is zero value in making it easier. Shifting gears is not remotely the hardest part of safely driving a car. Yet automatic transmissions are a nice feature for tons of people. I do not understand this dismissal based on the fact that copilot does not completely revolutionize software engineering in a way that no other product has ever come close to doing.
seibelj 5y ago
A brief tangent - the only useful bit of voice recognition for me has been telling Siri to remind me about something at a day and time. Calendar events don't work well (gets inserted on the wrong calendar, can't invite attendees properly), and every other use case for voice control has been a gimmick at best that I have shut off or not used. Whenever I call any phone number, I slam the 0 button and say "operator" to bypass the horrible menus, which usually accept digits anyway as the voice control is a piece of shit. The number of "Chat Bot AI" startups that have been funded, found no usage, pivoted to something else has been astronomical, and chat bots should seemingly be one of the first use cases of useful AI. With that said, I have found all "creative" types of AI to be similarly gimmicky and shallow. Co-pilot is like IntelliJ code-complete in more generic circumstances. It can't architect systems, interview and recruit good engineers, or do any of the things that separate true success from failure. It's yet another cool demo but ultimately useless for me.
zaptheimpaler 5y ago
Copilot was made from stealing code on Github, ignoring the licenses set on repos such as e.g GPLv2, using AI as a trick to license-launder code. Copilot has announced their plans to become a paid service. So this product that would not be possible without public, open-source code will itself be non-public, closed-source, closed-data. It is extracting value from the commons and funneling it to a private company.
- grp000 5y ago
  I think a lot of artists and content creators would also like a word.
- elviswolcott 5y ago
  While I agree with your concerns about licensing and copilot, this criticism doesn't seem particularly relevant to the article that was shared.
  - zaptheimpaler 5y ago
    IMO any place where Copilot is mentioned is a relevant place to put this. I don't know how anyone working in software can just turn a blind eye to shit like this. Anyone who uses Copilot is implicitly endorsing this theft. We have a duty as practitioners in the industry to call it out when we see something wrong. If even devs aren't calling bullshit on Copilot, the media won't care, courts won't care, and it will be declared legal, and future theft will be normalized. Its ridiculous how we all see the big tech companies doing various kinds of terrible shit and then the next new shiny thing comes along and everyone forgets all about it? Are you goldfish? What will it take to get someone to actually give a shit and stop supporting this kind of product/behavior?
    - fraktl 5y ago
      > Anyone who uses Copilot is implicitly endorsing this theft. Just like you're endorsing being sponsored and effectively stealing GitHub's bandwith? :) > We have a duty as practitioners in the industry to call it out when we see something wrong. It's wrong both ways. You accepted the service from GitHub, the free one where you get to open your account, host your code, share it etc. What exactly did you expect? To be served for free for your entire lifetime? Please, get off the moral highground. You ate the devil fruit, now you're whining about it. You should be smart enough to know that this kind of whining will get you nowhere. Just quit.
RandomBK 5y ago
Having used Copilot for a couple of hours on a Typescript/React codebase, I've encountered a very wide range of results. Sometimes, the model correctly guesses exactly what I'm trying to write, and gives me non-trivial contextually correct suggestions. More than once I've been impressed enough to pause and take a screen recording of what it generated. Other times, it's a complete mess that competes with the local language server for autocomplete corrections. Among the misses, the most glaring issues have been when it's suggested code that violates static typing constraints. The model outputs the right contextual code, but uses variables with slightly different spelling/wording/capitalization. This is to be expected of course, as Copilot doesn't integrate with the language server in VSCode. However it does show the limits of building the model without a native understanding of the AST it's working with.
- singularity2001 5y ago
  makes me wonder whether one can extend the system to filter/correct types/names
  - RandomBK 5y ago
    I sure hope so, and given Copilot currently integrates with VSCode and its amazing language server APIs, I imagine this could be a (relatively) simple win to make a significant bump in model accuracy. I do wonder if Copilot has any idea of the structure of what it's generating though. Does it know when something is a comment, a variable, or a piece of syntax?
svilen_dobrev 5y ago
what i don't get, is how a copy of (parts of) whatever's code, (or worse, a concoction of them), without any context, would solve anything but import the (unfound yet) errors as well, and introduce new errors. Hah, just make it autocommit, and will be much like with self-driving cars - human takes the blame for computer's mistakes (= algorithm/ programmer/ knowledgebase), as well as hir own, and can't/won't do anything about it..
bryanrasmussen 5y ago
In all literature there are strong themes of mistaking the blessing for a curse, and vice versa.
nxpnsv 5y ago
It is tempting to apply Betteridge's Law and answer no. But, perhaps the answer is yes. It probably can be a blessing, once you're used to it. And it certainly can be a curse, writing code that is incorrect, buggy, or just not understood by the programmer. It would be nice if it better could explain its reasons and add references relating to the snippets. Maybe one day. Until then, I think I'll just google stackexchange as I've been doing for so long...
- somehnacct3757 5y ago
  I think Betteridge's Law works just fine here. The answer to the article title is 'no' because Copilot is not just one thing used in one scenario. Depending on what situation you're in and how you use it, it could be a blessing or a curse.
thorum 5y ago
The article mentions TabNine as an alternative. I think TabNine’s approach of providing one-line autocomplete suggestions is a nice middle ground where you’re still in charge of writing code but can do it faster with AI assistance. Unlike Copilot you don’t have to give it instructions - it guesses what you’re trying to do from context. And since it works on a single line of code at a time, providing multiple completions which you can glance at and decide whether to use or ignore, you the programmer are still in charge of planning the function and making choices.
beltsazar 5y ago
> In this case, I felt like the Copilot approach would be worse for both experienced and beginner programmers. Experienced programmers would need to spend time studying the various options proposed, recognize that they don’t correctly solve the problem, and then would have to search online for solutions anyway. Beginner programmers would likely feel like they’ve solved the problem, wouldn’t actually learn what they need to understand about limitations and capabilities of regular expressions, and would end up with broken code without even realizing it. This is similar to what I thought—replace "experienced" and "beginner" with "curious" and "lazy"—when I first read the tagline of GitHub Copilot on its landing page: > Skip the docs and stop searching for examples. GitHub Copilot helps you stay focused right in your editor. We need docs and examples for things that we aren't familiar with. Skipping them means that we blindly trust Copilot to generate correct code. And even if the generated code is correct, we still need to understand why and how it's correct. One use case I see the best fit for Copilot is for generating boilerplates that we are already familiar with and understand.
wonnage 5y ago
It's mentioned towards the end of the blogpost, but I think the real blessing in disguise might be the ability to identify similar code from elsewhere on Github and linking to it. Too many libraries have you wading through pages of documentation in search of a single usage example. Some other ideas: - Look for libraries with similar code and suggest them - Deobfuscating minified variable names? Or just suggesting better ones for your source code - Speculatively running things. If the model predicts importing a certain library, start indexing it before I even type it - Estimate how long it thinks a block of code will take to write, based on the predicted code + past usage patterns (like the kindle reading progress estimate) Just spitballing but all of these things feel more helpful and pair programming-y than trying to fabricate code out of the blue.
- rightbyte 5y ago
  I would be happy of my IDE could throw me to the relevant topic on cppreference.com from a type or variable. Or still show the function signature after I started typing the function arguments. There are so much low hanging fruit it is silly.
elevenoh 5y ago
Does Copilot use a lotta RAM?
- jph00 5y ago
  No, it's done on the server side, so it doesn't use any more resources than regular autocomplete AFAICT.
jph00 5y ago
Hi all, Jeremy Howard here, author of the article. I'm happy to answer any questions you have about language models and program synthesis, Copilot, etc.
- tchalla 5y ago
  Thanks Jeremy! Where have language model(s) had the most useful for practical applications in terms of generated text? In terms of practical applications, I specifically mean the inclusion of a language model(s) has made life net better.
  - jph00 5y ago
    I'm actually most interested in language models that are fine tuned for other purposes, such as classification. IMO that's where the biggest immediate opportunities are. I've seen them used quite widely in industry, such as for legal discovery.
- YeGoblynQueenne 5y ago
  Hi Jeremy. Can you describe your knowledge of program synthesis? How long have you been studying the subject? Can you point to some interesting program synthesis work you have read lately?
peterthehacker 5y ago
> Given this, why does Copilot write such crappy code? > The reason is because of how language models work. They show how, on average, most people write. They don’t have any sense of what’s correct or what’s good. Most code on GitHub is (by software standards) pretty old, and (by definition) written by average programmers. Does Copilot apply any project quality filters before training? A small minority of public repos on GitHub are awesome projects that are popular and maintained, while the rest are forgotten experiments, side projects and interview answers.
nkmnz 5y ago
I’m not into copyleft, but is there already an analogue to AGPL that explicitly prohibits the usage of the code to train AI models? Shouldn’t be that hard to do, right?
moogly 5y ago
> blessing or a curse? It's just a footnote. I think it's getting talked about more than it deserves to be.
hatchnyc 5y ago
I have found that the cognitive load required to evaluate its suggestions is more of a distraction from what I'm doing--in sharp contrast to conventional autosuggest which is predictable. That said, I haven't turned it off yet. It's very fun on those occasions when it suggests correctly.
daitangio 5y ago
I do not know. I think best copilot are... - services - tested software libraries All the time I wrote the code in the last years, I ended up discovering someone ALREADY have done it, often in a better way (not always). There is a nice quote who says "Who did not know Unix is doomed to rewrite it..." I think the same is with Java/Python/... code.
cortexio 5y ago
Personally i dont believe in any AI project that requires "context". Making a program understand "context" is almost impossible, and we are not even close to making it happen. We will eventually have it, but it's going to take probably 10-15 more years in order to do it decently. A basic context based AI example would one that could say: this is a person in a picture shown on a tv in the reflection of a mirror. Without it being programmed to know what these reflection/tv/photo/etc concepts are, it has to somehow learn these concepts by itself by studying conversations and images and a dictionary/wiki. It's going to take a while before that happens.
Fnoord 5y ago
There's a need to answer the question (precedent, new technology, huge potential / market interest, and ethical extremes in one way or another) but there's an alternative to giving in to that urge. Perhaps we're better off leaving it as-is. We can evaluate about this in some later stage of the process.
dragosbulugean 5y ago
I like the idea, it's like a manual vs automatic clutch in cars. You are still the driver.
wilde 5y ago
I hope they release a fine tuner for Codex. Jeremy is right that whole problem solving is likely overreach right now. But can I train the model to do a Rails version upgrade?
fortyrod 5y ago
Copilot is going to really rock the numbers on lucrative unhose-your-sucky-codebase consulting gigs. I'd feel gleeful about that but those gigs are the definition of soul-sucking so I guess still no free lunch.
mark_l_watson 5y ago
I signed up for Copilot’s waiting list yesterday and I am looking forward to trying it, with some skepticism. That said, I find OpenAI’s beta APIs based on GPT-3 to be very useful (so useful that I immediately released new versions of my Common Lisp and Clojure books with examples). These APIs are so inexpensive to use and useful, that they really should be part of many developers’ standard stack: different than Copilot that seems like a gimmick (but to be fair, I haven’t tried it yet).
YeGoblynQueenne 5y ago
[Warning: long comment ahead] I haven't used Copilot yet (I'm not coding in any of the languages it covers) but I've tried to understand it from the few articles (most critical) posted about it on the internet and from my understanding of program synthesis and languae modelling. I don't fully trust lay users to be able to exercise a machine learning model comprehensively enough to precisely understand its pros and cons, but I think I've formed some intuition about how Copilot works from the demonstrations I've seen. Here's my thoughts: a) As TFA points out, Copilot was trained on _all_ code on github (for select languages). "All" includes good code and bad code, code with bugs and code with, er, fewer bugs, code that does what it should and code that doesn't. The result is that Copilot models _all_ that code, regardless of whether it's got bugs or not. b) Copilot is (based on) a language model that predicts the next sequence of tokens, not a discriminator of good/bad code. That means it can generate code but it can't generate "correct" code. It doesn't have a concept of correct/incorrect code. b.1) More generally, it's impossible to discriminate between "correct" and "incorrect" code from features (tokens, in the case of language models) alone. The correctness of code depends on the programer's intent. In program synthesis, the programmer's intent is encoded as a specification and the correctness of a synthesised program must be considered in the context of that specification. A language model could be trained to include a specification as a feature, or as an additional modality (similar to training on images and text together to generate captions for images, or images from captions). However, this is not how Copilot was trained. The upshot of all this is that Copilot can't generate "correct programs" because it doesn't know what "correct programs" are. b.2) But wait! Copilot can generate code from a specification! From a natural language specification, in the form of inline comments given as part of a prompt. This seems to be a favourite mode of testing Copilot and its results are impressive, but there are two problems: i) natural language specifications are imprecise and get more so the longer they grow; and, ii) Copilot still has no way to determine whether the code it generates satisfies a natural language specification, or not. b.3) For program synthesis with guarantees of correctness, a specification is not enough: a verification procedure is also necessary -and those tend to be very tricky because Halting Problem. c) Copilot has memorised at least part of its training set. This is evident in the demonstrations of verbatim generation of known code, e.g. the Inverse Square Root function (from Quake... III Arena I think?) etc. c.1) This observation is helpful in understanding language models in general. The assumption is often stated that a model can't possibly be memorising its training set because models are smaller on disk than their training sets. And yet it seems that large language models are memorising parts of their training sets- most likely, the ones that are more common, and therefore most likely. c.2) This in turn means that the programs that Copilot is most likely to generate are variations of programs included in its training set and that have been memorised. Why is this most likely? Because those programs are themselves "most likely" - they are programs with the highest prior probability in the space of programs represented by Copilot's model. c.3.1) That said, Copilot doesn't always generate memorised programs verbatim. It clearly has the ability to "mix and match" and generate programs that are not included in its training set. How does Copilot (or language models in general) achieve that? A language model is essentially a representation of a dense region of Cartesian space bounded by the elements (tokens, expressions, whole programs) of the training set. Anything within that dense geometric region, a language model can recognise and generate. Anything outside it, is down to luck. c.3.2) To generate programs not included in its training set, Copilot interpolates between the programs in its training set and other programs in its model ("interpolates" as in "finds a point on a gradient between two programs' representations"). However, this interpolation is stochastic. Fudging very, very much, we can imagine Copilot traversing the AST of a memorised program and applying slight peturbations to that AST, at random, though according to its learned prior probabilities. This is why Copilot will generate different code each time it's run even with the same prompt, and the generated code is closer or farther away from memorised code according to its "temperature" parameter (that controls the randomness of "perturbations"). c.4) Suppose you take some program P and walk down its AST, perturbing the nodes at random, without having any way to know whether a perturbation results in correct code, incorrect code, or pure garbage. What is the expected result? The expected result is correct code, incorrect code, or garbage. Remember: Copilot can't tell which is which. So we don't know which we'll get. d) All that said, I think that i) Copilot can make a great boilerplate generator but ii) only if people learn to use it right. So far I think most demonstrations I've seen are "holding it wrong". d.1) As to ii) above, I think the best use-case of Copilot is to generate short code snippets, spanning at most a couple of lines, rather than entire functions. I think it's also going to work a lot better if it's allowed to complete code rather than comments (with code). See point b.2 above. The demonstrations I've seen so far seem to try to get Copilot to act as an Autopilot (sorry, couldn't resist). In that, it fails in the ways it should be expected to fail . I suspect that as users get more familiar with Copilot's abilities the best way to use it will become evident and Copilot will take its rightful place as a useful tool in the programmer's toolbox that can simplify, but not replace, day-to-day programming work. e) Rumours of the imminent demise of the programming profession have, of course, been greately exaggerated. Wait another couple of generations, maybe. Human generations. f) If you're interested in program synthesis and the progress that has been achieved so far in this old, but still active field of research, the following is a good recent survey of the field: https://www.microsoft.com/en-us/research/wp-content/uploads/2017/10/program_synthesis_now.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...
markerickson 5y ago
I enjoyed the appropriately new phrase mentioned in the article “… gives a start of the art result…”
felgueres 5y ago
The value of copilot is in ideating solutions not code correctness. Given it's far from getting syntax and semantics right, it should probably be best to position this product as a tool that promotes creativity not productivity.
rurban 5y ago
Neither of both. It's completely irrelevant, as you cannot and may not use it. You cannot use it for production code as it's merely a toy for managers. And you may not use it as the license of the generated code is unclear at best, GPL at worst.

170 points by jph00 5y ago

Now, imagine you're sitting with your boss, the average typically incompetent project manager or lead programmer ... Them: type type type ... accept accept ... save (unaware or choosing not to understand the meaningless uncompilable garbage that's been generated) Them: "There, that's the xxxx function. You'll have the rest done in 2 days, won't you!"

da39a3ee 5y ago
Agree with your sentiment. But there is a solution: You: “Great, that might be good. Please stay here while we write and run the tests for that. (Our engineering handbook emphasizes how important tests are, as you know.)”

robertwt7 5y ago

Feels like its a magical thing for me honestly. although I'm never waiting for suggestion while coding, sometimes the suggestions just slipped through and there is the OMG moment when it completes your thought. 1 thing though, tab for auto complete still does not work for me.. ever

da39a3ee 5y ago
Yeah tab didn’t work for me either. But then I turned off tab for autocomplete in settings and that seemed to fix it. I didn’t understand, but try it.
spookyuser 5y ago
Same, I've enjoyed using it in a couple of one off scripts and honestly it made writing them more enjoyable because I was kind of excited to see what copilot would do. I asked it to write a js function to clean special characters from a string and it worked perfectly, I think, well it worked perfectly enough for the script to complete. I can't speak to whether or not using copilot in long lasting projects would be a good idea, but even if you just tagged the copilot generated code with a comment that said '#TODO: Verify this is correct' I still think it would help you get a lot of work done that you would normally have to switch to stackoverflow to do. One thing that I found very interesting from the article was the idea of anchoring bias, I definitely felt this a lot. When I used copilot to generate some nontrivial functions that were directly related to my problem, if the code did not work and I ended up erasing the generated code completely, I am just realising now that when I reimplemented it myself I was still essentially using the algorithm copilot had come up with.

NegativeLatency 5y ago

Well if they ever get it right I guess I could always become an electrician (this is not a dig on electricians, it seems like a fun trade, that or under water welding)

bigyikes 5y ago
Yeah, the real curse is on the programmers who are going to be out of jobs much sooner than people anticipate. All the talk about the low quality of code got me thinking: if humans aren’t reading the code, then the only thing that matters is correctness. Correctness may be an easier problem to solve than readability and refactorability.
- methodin 5y ago
  They are intertwined, no? All the training inputs would be based on the code that was affected by the aforementioned traits.
- robbedpeter 5y ago
  Media synthesis means artists of all sorts will take a hit. Gpt means writers, programmers, journalists, and others will take a hit. Prompt engineering and larger models could mean any procedural use of knowledge in any context could be subject to automation. Consumer level robotics, drones, and battery tech is close to a level that manual labor, delivery, and such tasks can be automated. Maybe we should be preparing for post-scarcity. At the very least, the idea of "jobs" is going to have to transform. The 40 hour work week isn't going to make much sense in the near future.
  - onion2k 5y ago
    Copilot (and all other AI tools, and non-AI tools) will lead to more developer jobs, not less. The thing holding most industries back at the moment is the lack of available developers at a cost-effective price. Literally every business has an opportunity to automate some process better than it does now, or to make an app to assist a business function, or to drive sales and growth through an application of new tech. The reason why they don't do that is cost and availability of skilled people to do the work. If a new AI tool removes the need to write code entirely and a developer just needs to work out the specifications and requirements then whole industries will start employing more developers. The part of dev that you get paid to do is not writing the code. It's the part where you define what the code should do. That's where all the value in development resides. Getting AI to write the syntax is 100% upside for the tech industry.

habibur 5y ago

His conclusion to the question in title : "I don't know."

eminence32 5y ago

> The code Copilot writes is not very good code. This has been my experience as well. Some of the early promo material for Copilot showed someone writing a function signature with docs, and having Copilot write the entire body. This rarely works for me, except for fairly trivial functions. This is also not how I normally write code. However, where Copilot has been rather good is offering intelligent "tab completion", where it makes a suggestion for the next line of code. In particular, it seems very good at figuring out the structure of my code and making sensible suggestions. I suspect that if anyone ends up using Copilot for the long term, they will (as a human operator) need to learn and gain intuition about how to use Copilot's strengths and avoid its weaknesses.

skohan 5y ago
Copilot is also almost certainly going to become better. For example, NVIDIA's DLSS was viewed as a failure at launch, but now it's almost magic in terms of offering better results at a lower performance cost.
- outsomnia 5y ago
  Better at issuing haphazardly modified boilerplate that already exists... the real costs of writing code are about carefully choosing how pieces will fit together and deciding on minutae with that in mind. Copilot is the antithesis of designing good software.
  - skohan 5y ago
    AI is getting better than humans at a lot of complex task over time. Just because it can't code well now doesn't mean it won't in the future.
    - krageon 5y ago
      What it means that it can't code well, so there's nothing to be excited about. That we cannot predict the future is of course true but really has no bearing on what you're saying, except to lend credence to the idea that it will be able to some day (i.e. that you can predict the future). Which is in essence the opposite of what "we cannot predict the future" should mean. That makes the entire argument kind of disingenuous.

foxes 5y ago

How does copilot fair in a language where correctness or at least well soundness is more transparent, for example Haskell (eg at least making sure all the types are correct, actual logic on the other hand seems harder, but maybe the declarative style helps)? I imagine in a less strict language you can generate garbage more easily.

toomanyducks 5y ago
Some of the examples given wouldn't even run, as far as I know. Copilot writing Haskell/Rust/<insert your intricate type system lang here> would probably just fail to compile. I know very, very little about AI, but, like the OP said, it'll reproduce the traits of its training distribution. If the traits don't map on to the domain-specifics of the type ssytem, the traits don't map on, and the code fails. Generating garbage the compiles is harder, but generating garbage in general is just as easy.
- truth_ 5y ago
  Yeah, I chanced upon an official livestream by GH showcasing Copilot, and it was performing very poorly on Rust.
exdsq 5y ago
Building on this idea, if you could design a type-level program with Haskell (including some liquid Haskell to add further type invariants) it'd be awesome for something like Copilot to fill in the blanks. Seems easier than program synthesis!

croes 5y ago

It will depend on the companies and the price of Copilot. Then they will replace expensive experienced programmers with cheaper inexperienced ones and Copilot.

elevenoh 5y ago
hmmm, remains to be seen! might just keep the experienced/quality ones & have less need for a high quantity of inexperienced programmers.

mbgerring 5y ago

So far it's been worse than useless for me, the suggestions it offers don't even make sense. The existing code completion suggestions in VSCode for TypeScript are better.

minimaxir 5y ago

Whenever a new generative language model is released and discussed on Hacker News, there's always the comment that "it's worthless because it doesn't always generate coherent text" which I've always disliked because it's unfairly reductive as progress in this field is iterative. However, in the case of Copilot, there has to be a much higher signal-to-noise ratio in terms of generated text than creative language models like GPT-2/GPT-3 to use it as a pair programmer where the overhead in working with the model is worth the time saved via the model.

Barrin92 5y ago
> it's unfairly reductive as progress in this field is iterative. is that true though? Progress in natural(!) language processing is very iterative, but coding and code correctness is a very precise thing. Both semantically as well as literally. Code that's a 'little bit' wrong technically does not compile, and code that's a 'little bit' wrong semantically might produce catastrophic results. Engineering doesn't become a natural language problem just because you throw a natural language model at it.
- sanxiyn 5y ago
  Natural language syntax is precise too. It's not like it's personal idiosyncratic preference. People will mostly agree whether something is syntactically correct or incorrect.

ALittleLight 5y ago

There's an interesting section that talks about what you want from a pair programmer - questioning your assumptions, spotting errors, debating design with you, etc, and how Copilot doesn't do this, but instead just spits out the code it thinks you want. That made me think that, when using Copilot, the human is actually the copilot. You're the one spotting bugs, picking different designs, and so on. I also think the discussion around searching versus autocomplete is interesting. What we need is something that autogenerates a Google search based on your code context and shows you some promising results. I'm imagining a pane in VS Code getting populated with Stack Overflow answers whenever I slow down.

carschno 5y ago
Something like the Stackoverflow Importer for Python? https://github.com/drathier/stack-overflow-import https://github.com/drathier/stack-overflow-import
- wildbook 5y ago
  It exists: https://github.com/MythicManiac/copilot-import https://github.com/MythicManiac/copilot-import
jimmygrapes 5y ago
100% agreed; the copilot is human.
seumars 5y ago
For what it’s worth, Copilot is not being marketed as anything else than code suggestion software - an alternative to google search, perhaps more of a “I’m feeling lucky” without having to actually type the search term. Knowing how to succinctly express a problem is half the problem after all.
- ALittleLight 5y ago
  It's called "Copilot" and the tagline is "Your AI pair programmer." It's obviously being marketed as more than code suggestion software. It's being marketed as an AI pair programmer. I agree that it is basically just code suggestion software, or like a big autocomplete. But it's definitely being marketed as more.

imvetri 5y ago

Let your product owners capture the requirements in GHERKIN, feed it to copilot. Sit back and relax.

ignu 5y ago

Writing code is, uh, not remotely the hardest part of software development. Maintaining it is.

grammarnazzzi 5y ago
I agree. The challenging part of coding is converting domain knowledge into correct code. An AI has no high-level understanding of the domain and desired solution. It just guesses at what you might want and glosses over all the edge cases and exceptions that actually need to be considered and resolved. The issues that only come to light after a real intelligence studies the problem, and works through correct steps to implement a solution. It's an iterative process, because the solution is arrived at only after attempting a solution, learning what you don't know or haven't considered about the problem, then resolving those ambiguities, and producing a revised and more correct solution. When responsible programmers realize that they don't know what they don't know, they ask questions and apply the answers. AI never realizes when it makes mistakes because it has no domain knowledge. It just pulls a best guess out it's ass and says, "Here you go. Check my work."
UncleMeat 5y ago
That's why everybody programs in base notepad, with no syntax highlighting or auto complete or live error notifications or linting or any other quality of life feature. What you say here is true. Writing code is not remotely the hardest part of software engineering. But that does not mean that there is zero value in making it easier. Shifting gears is not remotely the hardest part of safely driving a car. Yet automatic transmissions are a nice feature for tons of people. I do not understand this dismissal based on the fact that copilot does not completely revolutionize software engineering in a way that no other product has ever come close to doing.

seibelj 5y ago

A brief tangent - the only useful bit of voice recognition for me has been telling Siri to remind me about something at a day and time. Calendar events don't work well (gets inserted on the wrong calendar, can't invite attendees properly), and every other use case for voice control has been a gimmick at best that I have shut off or not used. Whenever I call any phone number, I slam the 0 button and say "operator" to bypass the horrible menus, which usually accept digits anyway as the voice control is a piece of shit. The number of "Chat Bot AI" startups that have been funded, found no usage, pivoted to something else has been astronomical, and chat bots should seemingly be one of the first use cases of useful AI. With that said, I have found all "creative" types of AI to be similarly gimmicky and shallow. Co-pilot is like IntelliJ code-complete in more generic circumstances. It can't architect systems, interview and recruit good engineers, or do any of the things that separate true success from failure. It's yet another cool demo but ultimately useless for me.

zaptheimpaler 5y ago

Copilot was made from stealing code on Github, ignoring the licenses set on repos such as e.g GPLv2, using AI as a trick to license-launder code. Copilot has announced their plans to become a paid service. So this product that would not be possible without public, open-source code will itself be non-public, closed-source, closed-data. It is extracting value from the commons and funneling it to a private company.

grp000 5y ago
I think a lot of artists and content creators would also like a word.
elviswolcott 5y ago
While I agree with your concerns about licensing and copilot, this criticism doesn't seem particularly relevant to the article that was shared.
- zaptheimpaler 5y ago
  IMO any place where Copilot is mentioned is a relevant place to put this. I don't know how anyone working in software can just turn a blind eye to shit like this. Anyone who uses Copilot is implicitly endorsing this theft. We have a duty as practitioners in the industry to call it out when we see something wrong. If even devs aren't calling bullshit on Copilot, the media won't care, courts won't care, and it will be declared legal, and future theft will be normalized. Its ridiculous how we all see the big tech companies doing various kinds of terrible shit and then the next new shiny thing comes along and everyone forgets all about it? Are you goldfish? What will it take to get someone to actually give a shit and stop supporting this kind of product/behavior?
  - fraktl 5y ago
    > Anyone who uses Copilot is implicitly endorsing this theft. Just like you're endorsing being sponsored and effectively stealing GitHub's bandwith? :) > We have a duty as practitioners in the industry to call it out when we see something wrong. It's wrong both ways. You accepted the service from GitHub, the free one where you get to open your account, host your code, share it etc. What exactly did you expect? To be served for free for your entire lifetime? Please, get off the moral highground. You ate the devil fruit, now you're whining about it. You should be smart enough to know that this kind of whining will get you nowhere. Just quit.

RandomBK 5y ago

Having used Copilot for a couple of hours on a Typescript/React codebase, I've encountered a very wide range of results. Sometimes, the model correctly guesses exactly what I'm trying to write, and gives me non-trivial contextually correct suggestions. More than once I've been impressed enough to pause and take a screen recording of what it generated. Other times, it's a complete mess that competes with the local language server for autocomplete corrections. Among the misses, the most glaring issues have been when it's suggested code that violates static typing constraints. The model outputs the right contextual code, but uses variables with slightly different spelling/wording/capitalization. This is to be expected of course, as Copilot doesn't integrate with the language server in VSCode. However it does show the limits of building the model without a native understanding of the AST it's working with.

singularity2001 5y ago
makes me wonder whether one can extend the system to filter/correct types/names
- RandomBK 5y ago
  I sure hope so, and given Copilot currently integrates with VSCode and its amazing language server APIs, I imagine this could be a (relatively) simple win to make a significant bump in model accuracy. I do wonder if Copilot has any idea of the structure of what it's generating though. Does it know when something is a comment, a variable, or a piece of syntax?

svilen_dobrev 5y ago

what i don't get, is how a copy of (parts of) whatever's code, (or worse, a concoction of them), without any context, would solve anything but import the (unfound yet) errors as well, and introduce new errors. Hah, just make it autocommit, and will be much like with self-driving cars - human takes the blame for computer's mistakes (= algorithm/ programmer/ knowledgebase), as well as hir own, and can't/won't do anything about it..

bryanrasmussen 5y ago

In all literature there are strong themes of mistaking the blessing for a curse, and vice versa.

nxpnsv 5y ago

It is tempting to apply Betteridge's Law and answer no. But, perhaps the answer is yes. It probably can be a blessing, once you're used to it. And it certainly can be a curse, writing code that is incorrect, buggy, or just not understood by the programmer. It would be nice if it better could explain its reasons and add references relating to the snippets. Maybe one day. Until then, I think I'll just google stackexchange as I've been doing for so long...

somehnacct3757 5y ago
I think Betteridge's Law works just fine here. The answer to the article title is 'no' because Copilot is not just one thing used in one scenario. Depending on what situation you're in and how you use it, it could be a blessing or a curse.

thorum 5y ago

The article mentions TabNine as an alternative. I think TabNine’s approach of providing one-line autocomplete suggestions is a nice middle ground where you’re still in charge of writing code but can do it faster with AI assistance. Unlike Copilot you don’t have to give it instructions - it guesses what you’re trying to do from context. And since it works on a single line of code at a time, providing multiple completions which you can glance at and decide whether to use or ignore, you the programmer are still in charge of planning the function and making choices.

beltsazar 5y ago

> In this case, I felt like the Copilot approach would be worse for both experienced and beginner programmers. Experienced programmers would need to spend time studying the various options proposed, recognize that they don’t correctly solve the problem, and then would have to search online for solutions anyway. Beginner programmers would likely feel like they’ve solved the problem, wouldn’t actually learn what they need to understand about limitations and capabilities of regular expressions, and would end up with broken code without even realizing it. This is similar to what I thought—replace "experienced" and "beginner" with "curious" and "lazy"—when I first read the tagline of GitHub Copilot on its landing page: > Skip the docs and stop searching for examples. GitHub Copilot helps you stay focused right in your editor. We need docs and examples for things that we aren't familiar with. Skipping them means that we blindly trust Copilot to generate correct code. And even if the generated code is correct, we still need to understand why and how it's correct. One use case I see the best fit for Copilot is for generating boilerplates that we are already familiar with and understand.

wonnage 5y ago

It's mentioned towards the end of the blogpost, but I think the real blessing in disguise might be the ability to identify similar code from elsewhere on Github and linking to it. Too many libraries have you wading through pages of documentation in search of a single usage example. Some other ideas: - Look for libraries with similar code and suggest them - Deobfuscating minified variable names? Or just suggesting better ones for your source code - Speculatively running things. If the model predicts importing a certain library, start indexing it before I even type it - Estimate how long it thinks a block of code will take to write, based on the predicted code + past usage patterns (like the kindle reading progress estimate) Just spitballing but all of these things feel more helpful and pair programming-y than trying to fabricate code out of the blue.

rightbyte 5y ago
I would be happy of my IDE could throw me to the relevant topic on cppreference.com from a type or variable. Or still show the function signature after I started typing the function arguments. There are so much low hanging fruit it is silly.

elevenoh 5y ago

Does Copilot use a lotta RAM?

jph00 5y ago
No, it's done on the server side, so it doesn't use any more resources than regular autocomplete AFAICT.

jph00 5y ago

Hi all, Jeremy Howard here, author of the article. I'm happy to answer any questions you have about language models and program synthesis, Copilot, etc.

tchalla 5y ago
Thanks Jeremy! Where have language model(s) had the most useful for practical applications in terms of generated text? In terms of practical applications, I specifically mean the inclusion of a language model(s) has made life net better.
- jph00 5y ago
  I'm actually most interested in language models that are fine tuned for other purposes, such as classification. IMO that's where the biggest immediate opportunities are. I've seen them used quite widely in industry, such as for legal discovery.
YeGoblynQueenne 5y ago
Hi Jeremy. Can you describe your knowledge of program synthesis? How long have you been studying the subject? Can you point to some interesting program synthesis work you have read lately?

peterthehacker 5y ago

> Given this, why does Copilot write such crappy code? > The reason is because of how language models work. They show how, on average, most people write. They don’t have any sense of what’s correct or what’s good. Most code on GitHub is (by software standards) pretty old, and (by definition) written by average programmers. Does Copilot apply any project quality filters before training? A small minority of public repos on GitHub are awesome projects that are popular and maintained, while the rest are forgotten experiments, side projects and interview answers.

nkmnz 5y ago

I’m not into copyleft, but is there already an analogue to AGPL that explicitly prohibits the usage of the code to train AI models? Shouldn’t be that hard to do, right?

moogly 5y ago

> blessing or a curse? It's just a footnote. I think it's getting talked about more than it deserves to be.

hatchnyc 5y ago

I have found that the cognitive load required to evaluate its suggestions is more of a distraction from what I'm doing--in sharp contrast to conventional autosuggest which is predictable. That said, I haven't turned it off yet. It's very fun on those occasions when it suggests correctly.

daitangio 5y ago

I do not know. I think best copilot are... - services - tested software libraries All the time I wrote the code in the last years, I ended up discovering someone ALREADY have done it, often in a better way (not always). There is a nice quote who says "Who did not know Unix is doomed to rewrite it..." I think the same is with Java/Python/... code.

cortexio 5y ago

Personally i dont believe in any AI project that requires "context". Making a program understand "context" is almost impossible, and we are not even close to making it happen. We will eventually have it, but it's going to take probably 10-15 more years in order to do it decently. A basic context based AI example would one that could say: this is a person in a picture shown on a tv in the reflection of a mirror. Without it being programmed to know what these reflection/tv/photo/etc concepts are, it has to somehow learn these concepts by itself by studying conversations and images and a dictionary/wiki. It's going to take a while before that happens.

Fnoord 5y ago

There's a need to answer the question (precedent, new technology, huge potential / market interest, and ethical extremes in one way or another) but there's an alternative to giving in to that urge. Perhaps we're better off leaving it as-is. We can evaluate about this in some later stage of the process.

dragosbulugean 5y ago

I like the idea, it's like a manual vs automatic clutch in cars. You are still the driver.

wilde 5y ago

I hope they release a fine tuner for Codex. Jeremy is right that whole problem solving is likely overreach right now. But can I train the model to do a Rails version upgrade?

fortyrod 5y ago

Copilot is going to really rock the numbers on lucrative unhose-your-sucky-codebase consulting gigs. I'd feel gleeful about that but those gigs are the definition of soul-sucking so I guess still no free lunch.

mark_l_watson 5y ago

I signed up for Copilot’s waiting list yesterday and I am looking forward to trying it, with some skepticism. That said, I find OpenAI’s beta APIs based on GPT-3 to be very useful (so useful that I immediately released new versions of my Common Lisp and Clojure books with examples). These APIs are so inexpensive to use and useful, that they really should be part of many developers’ standard stack: different than Copilot that seems like a gimmick (but to be fair, I haven’t tried it yet).

YeGoblynQueenne 5y ago

[Warning: long comment ahead] I haven't used Copilot yet (I'm not coding in any of the languages it covers) but I've tried to understand it from the few articles (most critical) posted about it on the internet and from my understanding of program synthesis and languae modelling. I don't fully trust lay users to be able to exercise a machine learning model comprehensively enough to precisely understand its pros and cons, but I think I've formed some intuition about how Copilot works from the demonstrations I've seen. Here's my thoughts: a) As TFA points out, Copilot was trained on _all_ code on github (for select languages). "All" includes good code and bad code, code with bugs and code with, er, fewer bugs, code that does what it should and code that doesn't. The result is that Copilot models _all_ that code, regardless of whether it's got bugs or not. b) Copilot is (based on) a language model that predicts the next sequence of tokens, not a discriminator of good/bad code. That means it can generate code but it can't generate "correct" code. It doesn't have a concept of correct/incorrect code. b.1) More generally, it's impossible to discriminate between "correct" and "incorrect" code from features (tokens, in the case of language models) alone. The correctness of code depends on the programer's intent. In program synthesis, the programmer's intent is encoded as a specification and the correctness of a synthesised program must be considered in the context of that specification. A language model could be trained to include a specification as a feature, or as an additional modality (similar to training on images and text together to generate captions for images, or images from captions). However, this is not how Copilot was trained. The upshot of all this is that Copilot can't generate "correct programs" because it doesn't know what "correct programs" are. b.2) But wait! Copilot can generate code from a specification! From a natural language specification, in the form of inline comments given as part of a prompt. This seems to be a favourite mode of testing Copilot and its results are impressive, but there are two problems: i) natural language specifications are imprecise and get more so the longer they grow; and, ii) Copilot still has no way to determine whether the code it generates satisfies a natural language specification, or not. b.3) For program synthesis with guarantees of correctness, a specification is not enough: a verification procedure is also necessary -and those tend to be very tricky because Halting Problem. c) Copilot has memorised at least part of its training set. This is evident in the demonstrations of verbatim generation of known code, e.g. the Inverse Square Root function (from Quake... III Arena I think?) etc. c.1) This observation is helpful in understanding language models in general. The assumption is often stated that a model can't possibly be memorising its training set because models are smaller on disk than their training sets. And yet it seems that large language models are memorising parts of their training sets- most likely, the ones that are more common, and therefore most likely. c.2) This in turn means that the programs that Copilot is most likely to generate are variations of programs included in its training set and that have been memorised. Why is this most likely? Because those programs are themselves "most likely" - they are programs with the highest prior probability in the space of programs represented by Copilot's model. c.3.1) That said, Copilot doesn't always generate memorised programs verbatim. It clearly has the ability to "mix and match" and generate programs that are not included in its training set. How does Copilot (or language models in general) achieve that? A language model is essentially a representation of a dense region of Cartesian space bounded by the elements (tokens, expressions, whole programs) of the training set. Anything within that dense geometric region, a language model can recognise and generate. Anything outside it, is down to luck. c.3.2) To generate programs not included in its training set, Copilot interpolates between the programs in its training set and other programs in its model ("interpolates" as in "finds a point on a gradient between two programs' representations"). However, this interpolation is stochastic. Fudging very, very much, we can imagine Copilot traversing the AST of a memorised program and applying slight peturbations to that AST, at random, though according to its learned prior probabilities. This is why Copilot will generate different code each time it's run even with the same prompt, and the generated code is closer or farther away from memorised code according to its "temperature" parameter (that controls the randomness of "perturbations"). c.4) Suppose you take some program P and walk down its AST, perturbing the nodes at random, without having any way to know whether a perturbation results in correct code, incorrect code, or pure garbage. What is the expected result? The expected result is correct code, incorrect code, or garbage. Remember: Copilot can't tell which is which. So we don't know which we'll get. d) All that said, I think that i) Copilot can make a great boilerplate generator but ii) only if people learn to use it right. So far I think most demonstrations I've seen are "holding it wrong". d.1) As to ii) above, I think the best use-case of Copilot is to generate short code snippets, spanning at most a couple of lines, rather than entire functions. I think it's also going to work a lot better if it's allowed to complete code rather than comments (with code). See point b.2 above. The demonstrations I've seen so far seem to try to get Copilot to act as an Autopilot (sorry, couldn't resist). In that, it fails in the ways it should be expected to fail . I suspect that as users get more familiar with Copilot's abilities the best way to use it will become evident and Copilot will take its rightful place as a useful tool in the programmer's toolbox that can simplify, but not replace, day-to-day programming work. e) Rumours of the imminent demise of the programming profession have, of course, been greately exaggerated. Wait another couple of generations, maybe. Human generations. f) If you're interested in program synthesis and the progress that has been achieved so far in this old, but still active field of research, the following is a good recent survey of the field: https://www.microsoft.com/en-us/research/wp-content/uploads/2017/10/program_synthesis_now.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...

markerickson 5y ago

I enjoyed the appropriately new phrase mentioned in the article “… gives a start of the art result…”

felgueres 5y ago

The value of copilot is in ideating solutions not code correctness. Given it's far from getting syntax and semantics right, it should probably be best to position this product as a tool that promotes creativity not productivity.

rurban 5y ago

Neither of both. It's completely irrelevant, as you cannot and may not use it. You cannot use it for production code as it's merely a toy for managers. And you may not use it as the license of the generated code is unclear at best, GPL at worst.