8 ms·

GitHub Copilot: First Impressions

125 points by vladiliescu 5y ago

lma21 5y ago
I love the fact that it can help write tedious, repetive, and simple logic code blocks! It'll save me time googling basic stuff that I tend to forget. Will it also learn (i.e. feed GPT) from the code we're writing which it is also helping to write? How do you think it'll learn to deprecate bad practices or evolutions observed in a language (think writing concurrency code in Java 5 vs. Java 9, or any other relevant Programming Language evolutions)?
- klohto 5y ago
  Lol yea, feed it even more licensed code. That’s exactly what we need even more of now.
- stared 5y ago
  > Will it also learn (i.e. feed GPT) from the code we're writing which it is also helping to write? Most likely not. And at least right now Tabnine's selling point is that you do inference locally (so no need to send code elsewhere) and can be trained on such (vide https://www.tabnine.com/tabnine-vs-github-copilot https://www.tabnine.com/tabnine-vs-github-copilot).
- dathinab 5y ago
  The problem is it produce "more tedious/repetitive but maybe subtitle slightly of code" where we should try to have "less and higher quality code".
- lttlrck 5y ago
  What kind of code? I'm having trouble thinking of anything for the kind of work I do, that isn't catered for by "intellisense". Anything beyond that seems to be impinging on why I actually enjoy programming.
obviyus 5y ago
I very recently got access to Copilot, while I was in the middle of learning and playing around with Clojure. It’s surprisingly useful when you’re not sure about how you want to proceed. E.g. While I was trying to make a simple function for printing all palindromic numbers under 10,000, Copilot inferred from the function name what I was trying and suggested a function using threading macros (something I hadn’t yet come across in Clojure). The result was a much neater affair than what I came up with on my own. I feel it could be a fantastic way to build familiarity with a new language.
- Waterluvian 5y ago
  I hear you. But I see this as a horrible thing. Now people are going to be further absolved of responsibility of thinking through the problem first and then implementing it after. Won’t this result in more junk code?
  - judofyr 5y ago
    Shouldn’t the question be whether it’s better or worse than copy-pasting random functions from StackOverflow you found when googling?
    - Waterluvian 5y ago
      Do people actually just copy paste? When I was much greener I never found that to work. The interface had to be thought about. And inevitably there’d be other details to modify. If anything this just makes that easier to do wrongly. It wants to offer you code that’s not yours that you didn’t conceive of thoughtfully.
      - deleted 5y ago
        [deleted]
      - stingraycharles 5y ago
        I’ve copy/pasted parts of code, but then I typically always link to the actual SO answer in the code. Sometimes it’s just the most pragmatic thing to do.
claviska 5y ago
> GitHub Copilot is a tool that helps you write better, faster, and most importantly, more code. I’ll agree with faster and more code, but from the many examples I’ve seen, it’s not better.
- dkersten 5y ago
  Its not even clear to me that writing faster and more code is necessarily a good thing, or, at least, that beneficial. I've said it in a previous Copilot discussion and I'll say it again, but actually writing code is a small part of what I do as a programmer. I spend much more time figuring out what the requirements even are, refining them, figuring out what that even means in terms of code, figuring out the overall architecture, how it fits in with other systems or existing code, what data formats it uses, how it handles faults, persistence, scale, security. How it interfaces with the outside world (UI or API). Besides the code itself, I also spend a lot of my time on writing tests (which I wouldn't want to pawn off on an AI outside of fuzzing or generating data for property-based tests; unit tests should mirror what the spec dictates and needs to test the correct things) and on writing documentation. Yes, the code does take up a good chunk of time, but really, its the easy part of my day! Also, speeding through the code means I'm not thinking about it very deeply. That's when I introduce the most bugs, design flaws or shortcomings that bite me later. I wonder if we'll end up with a situation like the old quote about code reviews: a ten line code review gets a hundred comments/suggestions/questions, a thousand line code review gets a ship it. If much of the code is written for us, will we have the attention span to scrutinize it and understand it deeply? Or will our eyes eventually just glaze over as we go yeah its probably fine, ship it.
  - b9a2cab5 5y ago
    This is exactly why I think Copilot (and other "AI writes code for you" solutions) are going to fail. It's harder to read code than it is to write it. That's the opposite of how English works.
the_lonely_road 5y ago
CoPilot for a few years while we train it and then in 2037 introducing Microsoft Pilot.
- esperent 5y ago
  Yes but 2027. If that.
ekster 5y ago
For me the AI part of this is all the bad parts. What would really be cool is better search, templating, and best practices at your fingertips in the IDE. Exact same problem to solve, because it is an important an interesting one, but a totally different approach. Maybe AI assisted somehow, but human curated. Otherwise, imagine how bad legacy codebases of the future will be when they are full of autocomplete code that nobody understands or cared enough to think through even originally.
- bsenftner 5y ago
  We should have AI tools to aid software engineers understanding of logic chains, and assorted visualizations like CAD, but for logic and the code creating that logic. And not UML nonsense, but some type of AI tool that begins where Doxygen ends and simply keeps going with various means of aiding the developer's understanding as they construct hierarchical logic systems.
  - ekster 5y ago
    Totally agree. Something that could interrogate and answer arbitrary questions about complex or new codebases would be incredible.
- vladiliescu 5y ago
  I think we'll have smarter AI shifting through those legacy codebases of the future, not sure if that'll be a big problem. The way I see this, GitHub Copilot and the like are true next-gen compilers, translating English into code. They'll only get better with time.
  - deleted 5y ago
    [deleted]
softwaredoug 5y ago
> We’ll also code reviews. Lots and lots of code reviews. Like, all the time. The algorithm will have to be kept in check. This is a repeated theme of the article. I think it’d be simpler to write the non boilerplaty code, no? Plus who’s going to be excited to have a job as “AI code reviewer”
- vladiliescu 5y ago
  You know that gif where Gary Oldman yells 'EVERYONE'? :P I've got a sneaking suspicion that this is what we'll be doing in the future, I see Copilot as a taste of what future compilers will be like. In a few years I expect to have most of an application built by an AI, with me developing the business/core logic by hand.
  - dathinab 5y ago
    > In a few years I expect to have most of an application built by an AI, with me developing the business/core logic by hand. I can see that too be preferably not with the approach done by GitHub.
- dathinab 5y ago
  It's worse then that, because the AI is likely to reproduce the kinds of bugs which where overlooked by the source it learned it from. At the same time code reviews are hard, much harder then writing code. So making it faster to write boiler code at the cost of harder code reviews seems to not be a good trade of for me.
Bitwit 5y ago
What if I slip in various 0-days? If crafted carefully, the sky is the limit. I'll try it and see what happens. I just have to know... This worries me.
- qayxc 5y ago
  This would be even more difficult to achieve than previous attempts (e.g. in the Linux kernel [0]) due to the fact that an attacker needs to corrupt thousands of repositories that are guaranteed to be part of the training set. Potential attackers would have two problems: 1) getting malicious checked into many repos and 2) making sure that these repos find their way into future deployed versions of GPT-3/Codex/CoPilot. CoPilot generates enough vulnerable code as-is [1], so the extra effort isn't even required. [0] https://www.bleepingcomputer.com/news/security/linux-bans-university-of-minnesota-for-committing-malicious-code/ https://www.bleepingcomputer.com/news/security/linux-bans-un... [1] https://cyber-reports.com/2021/07/14/devsecai-github-copilot-prone-to-writing-security-flaws/ https://cyber-reports.com/2021/07/14/devsecai-github-copilot...
- UncleMeat 5y ago
  The number of people who never write a vuln normally but would write a vuln if they were using machine synthesized code has got to be fewer than ten people on the planet.
- remram 5y ago
  Crafting might not be necessary. You might find a vulnerability in a commonly copiloted piece of code, and now you can exploit it in many projects. Better yet, those snippets cannot be updated even if Copilot improves, and there is nothing to file a CVE against either.
playcache 5y ago
> GitHub Copilot is a tool that helps you write better, faster, and most importantly, more code. I don't have a lot of faith in the author's code, if that is there opening statement ("better")
- vladiliescu 5y ago
  That makes two of us, I don't have a lot of faith in my code either.
blunte 5y ago
The AI coding approach is solving the wrong problem. The problem isn't with the low level detailed work. That problem should be solved by building composable, tested, audited libraries. Legos, if you will. If the problem is trivial enough that you can completely trust the AI coded solution, then you could have either done it yourself very easily or used a premade solution from a good library or toolkit. If the problem is not trivial, then you have the outsourcing challenges (which apply to a lot more scenarios than using AI to help you code). If you are not personally capable of judging the outsourced work, then whether you use AI or type it yourself, you will end up with errors or misfeatures. If you are capable of judging, then you must pay attention and read/review. So your job shifts from defining the problem and programming a solution to defining the problem and reviewing potential solution(s). Either way, you must focus and think. But again, perhaps you would be better off building a solution composed of known good blocks. <- This should be the future of software development... Sadly, I think that open source and freedom to (re)invent has worked against us in the long run. If instead of each of us going off and thinking, "I can make a better language/framework", we had built on existing technologies, I daresay we would be further along. To be fair of course, some level of dissatisfaction and divergence would be necessary or we would still be using assembly. Github does have one thing right though (from a business perspective) - they are making a remedy to a symptom, and in that they can expect longer term revenue than if they actually solved the core problem.
- sktrdie 5y ago
  I think you’re missing what the problem they’re trying to solve is. They’re solving the problem of “writing code faster” With that in mind I think this problem is totally real and worth solving. If copilot can save me those mundane moments during my day where I have to figure out “how to do this common thing that I already did 100 times” then it’s a win for everyone. It’s not trying to solve the whole of programming. It’s just a nice tool to let you actually concentrate on the non-automated parts of coding such as: actually translating requirements into code.
  - zabzonk 5y ago
    > how to do this common thing that I already did 100 times You don't do it 100 times - you do it once, test it thoroughly, and put it in a library.
drevil-v2 5y ago
> code reviews code reviews code reviews So basically shift the cognitive burden on to your coworkers? And what if they are also using copilot hoping that you will sanity check it’s output for them? Tit for tat prisoner’s dilemma and no one even realises they are playing the game..
- frabcus 5y ago
  Presumably copilot will easily do (bad!) code reviews as well, trained on all the world’s PR comments…
  - ptx 5y ago
    Maybe a future version could even write the requirements for the application. And when the users can't figure out how to use whatever comes out, we can have an AI do that part too.
- pdelgallego 5y ago
  That is what I like (at least in theory) of AWS CodeGuru. It helps you to detect bugs and common bad practices automatically in the code review.
kristiandupont 5y ago
So far, I am feeling positive about Copilot. I've used it for about a week and it has been useful at times. Like the author, I've mostly found it useful in situations where I am doing something repetitive. I definitely need to look over suggestions carefully though. I don't think it will go much further than that and I don't know what that would even look like anyway, unless I could actually start discussing architectural decisions with it like I do with a human pair programmer. I guess you could say that this is what the comments are for, so who knows.
losvedir 5y ago
I understand some of the legal implications to be regurgitating licensed code verbatim. But, what about this: what if the current Copilot is working out the kinks, and the real product is per-organization models with transfer learning using their repos' code? At work, we store all our code in our GitHub repo, some public, some private. As-is, I think there's a lot of legal ambiguity around using Copilot, but if all that code just served to teach the model structure of programs and common syntactical constructs, but then it had another layer with our code and its idioms, modules, names, then maybe it would regurgitate our code in a way that's useful and doesn't run afoul of licenses. I'm thinking of a fast.ai course I did where I took a base model trained on generic image data, and then did transfer learning on top where I fed it labeled images of Go games and Chess games, and with only maybe 100 of those images it learned to distinguish the two with shocking accuracy. As I understand it, the base model taught it how to look for things like lines, corners, contrast, etc, and then it could be easily specialized. Could something similar be the case here?
- vladiliescu 5y ago
  Yes, I think so too. Only not in the near future, as we don't really have enough computing power to make that feasible. I've written a few thoughts about this here https://vladiliescu.net/github-copilot-first-impressions/#potential https://vladiliescu.net/github-copilot-first-impressions/#po...
zaptheimpaler 5y ago
Microsoft just stole all the code on github to do this. Regardless of what the minutiae of the law say, no one really expected their work to be used this way. Open source code powers a huge chunk of the industry while capturing little value for the maintainers already. Github even explicitly supports a standard format for declaring the license of a repo, which was cleverly ignored. Here is the relevant section from Githubs privacy policy [1] > 6. Contributions Under Repository License > Whenever you add Content to a repository containing notice of a license, you license that Content under the same terms, and you agree that you have the right to license that Content under those terms. If you have a separate agreement to license that Content under different terms, such as a contributor license agreement, that agreement will supersede. From GPLv2, "When distributing derived works, the source code of the work must be made available under the same license." ------ This is not about technology, it is a legal endrun around using open source code without open sourcing derived work. It is using AI as a form of "license laundering". "OpenAI" is not open at all. Truly open AI means the code, the data and the model are all open. OpenAI sold the source to GPT-3 to Microsoft, received $1 billion from them in 2019 and does not make most of their work available except behind a highly exclusive, paid API - https://beta.openai.com/pricing/ https://beta.openai.com/pricing/. Its a joke to call that "open". I urge you to read up on OpenAI and look at what the have actually done. Their plan in the future is to sell access to Copilot, directly monetizing work they stole from others for free: > According to GitHub, “If the technical preview is successful, our plan is to build a commercial version of GitHub Copilot in the future.” I've deleted all my code from github and hope others do the same. Maybe if some bigger profile project starts doing this, we can start to organize around opposing Pilot and OpenAI. Others have also pointed out similar concerns - see https://news.ycombinator.com/item?id=27687450 https://news.ycombinator.com/item?id=27687450 for example. [1] https://docs.github.com/en/github/site-policy/github-terms-of-service#6-contributions-under-repository-license https://docs.github.com/en/github/site-policy/github-terms-o... [2] https://beta.openai.com/pricing/ https://beta.openai.com/pricing/
- 6gvONxR4sf7o 5y ago
  It’s a shame that copilot would not be possible without all the zillions of hours of work that went into writing that code, while the authors of that training data get zero compensation for their contribution to copilot (and zero ability to opt out).
ahofmann 5y ago
It is kind of impressive, that Copilot understood the misspelled "einz" and translated it as "one" ("eins" would have been correct).
- vladiliescu 5y ago
  Ouch, thanks for pointing this out, my German has a long way to go.
ipaddr 5y ago
This might work for some domains but I'm trying to write less code and abstract where possible. This writes more code and doesn't help design the application. It's a function autocompleter but it doesn't take my abstractions into account.
IfOnlyYouKnew 5y ago
It's good to read a less dramatic take on Copilot. The initial echo chamber of outrage felt rather strange, especially considering the usual attitude when people point out risks in AI systems. I guess everyone else's reading-file-line-by-line loops in python are the epitome of creativity, and being inspired by them is its own category of crime compared to the exact same thing happening in uncreative professions such as photography, music, or writing? And not being hired because the algorithm prefers people who played lacrosse in school is, like, your problem, because waiting to release models until they do not harm anyone would seriously mess with our agile process. As an aside, I really enjoyed the writing style. The subtle humour is better at signalling competence and friendliness than any CV ever could.
- vladiliescu 5y ago
  Thank you :)
- ptx 5y ago
  > I guess everyone else's reading-file-line-by-line loops in python are the epitome of creativity I would certainly hope not, since there is barely any code to write: for line in my_file: ... For even more convenience in common cases, there is the fileinput module[1] in the standard library. I can see how a boilerplate-generating AI could be helpful in a more boilerplate-heavy language like Java, but a better solution is to use a language that better suits your usecase and lets you express it without the boilerplate. [1] https://docs.python.org/3/library/fileinput.html https://docs.python.org/3/library/fileinput.html
  - vladiliescu 5y ago
    I'd say using a better language is an easy decision to make when you're in a team of one and not depending on any framework-specific features and somewhat harder when you're in a team of more, depending on some framework-specific features, or both.
DantesKite 5y ago
Not a single positive comment eh? Github Copilot and its iterations are the future. You can complain and whine about what problems are being solved, how it'll affect human developers (making them weaker instead of stronger over time). And to some extent, that's probably true. But it's still the future and it's coming. It's already here.
- solipsism 5y ago
  Is this your idea of a positive comment? "It's happening no matter what!"
  - alphachloride 5y ago
    I think he means a comment in support of Copilot's advancements and not one criticizing its kinks.
leksak 5y ago
One thing not mentioned in this article but that's on Github Copilots page is that they imagine it'll be useful for learning how to code. > ... or just learning to code I've done some teaching and my mental model for what is necessary to learn a language and, more generally, learning how to program and my view is that the rudimentary boilerplate-y type of stuff that this tool seems to excel at are mindless to most are essential for beginners as part of their learning. Any educators here with different ideas or thoughts?
arduinomancer 5y ago
To me it seems like copilot only helps with local code, which is absolutely not the bottleneck (it’s maybe 20% of my time) I spend much much more time figuring out how to convert requirements to code and how to structure it non-locally (as in how it fits into the whole codebase) Also if you’re concerned with writing clean code, copy-pasting boilerplate everywhere is not the right approach, you have to actually think about interfaces and abstractions
ellen364 5y ago
It’s early days for CoPilot, but I find myself wondering if it will eventually reduce idiomatic use of languages and increase the stickiness of old practices. In the article, the author included this example: alternate_word_mapping = {words[i]: words_in_english[i] for i in range(len(words))} That line is probably more readable than the Pythonic enumerate: alternate_word_mapping = {word: words_in_english[i] for i, word in enumerate(words)} But it superficially reminded me of a style that I see on Leetcode, which rarely uses Python features like enumerate or dict.items. CoPilot was trained on GitHub repos and many repos are mini-projects for learning a new language. Does that mean CoPilot will tend to suggest more generic, less language specific, implementations? If it does, will that change perceptions of what’s idiomatic? And will the volume of old code on GitHub influence CoPilot’s suggestions, making us slow to adopt new language features?
- falcor84 5y ago
  I agree in general, but just wanted to add that the pythonic approach would be to avoid the explicit index entirely, with something like: alternate_word_mapping = dict(zip(words, words_in_english))
  - vladiliescu 5y ago
    Thumbs up, I totally missed this. Didn’t encounter this approach in the suggestions either. That’s actually one reason why I don’t think it’s current incarnation is a good fit for learning new apis, it’s been trained on a lot of code, not all of it good. Seems like something a future version might be able to fix, perhaps by training a new layer using just demonstrably ‘good’ code?
  - ellen364 5y ago
    Agreed and an excellent point. Ironically, my example was not nearly as Pythonic as it could have been! (V late reply as still learning how to keep track of replies on HN. But perhaps you’ll see it anyway.)
- deleted 5y ago
  [deleted]
ipaddr 5y ago
Can I use this during my leetcode tests. I feel like it would be the perfect use-case
orange_puff 5y ago
I’m not a denier. I believe AI will vastly alter the way we write code. But, to be honest, it’s very depressing to me. I think I am someone who selfishly enjoys writing code, not necessarily getting software built. If my job became designing something at a very fine grained level, feeding it to an AI, having the AI write it and then code reviewing the AI, I’d just switch careers. Unfortunately for me, I’m super early in my career so I hope I can make enough money in the next 20 or so years such that I can retire young.

125 points by vladiliescu 5y ago

I love the fact that it can help write tedious, repetive, and simple logic code blocks! It'll save me time googling basic stuff that I tend to forget. Will it also learn (i.e. feed GPT) from the code we're writing which it is also helping to write? How do you think it'll learn to deprecate bad practices or evolutions observed in a language (think writing concurrency code in Java 5 vs. Java 9, or any other relevant Programming Language evolutions)?

klohto 5y ago
Lol yea, feed it even more licensed code. That’s exactly what we need even more of now.
stared 5y ago
> Will it also learn (i.e. feed GPT) from the code we're writing which it is also helping to write? Most likely not. And at least right now Tabnine's selling point is that you do inference locally (so no need to send code elsewhere) and can be trained on such (vide https://www.tabnine.com/tabnine-vs-github-copilot https://www.tabnine.com/tabnine-vs-github-copilot).
dathinab 5y ago
The problem is it produce "more tedious/repetitive but maybe subtitle slightly of code" where we should try to have "less and higher quality code".
lttlrck 5y ago
What kind of code? I'm having trouble thinking of anything for the kind of work I do, that isn't catered for by "intellisense". Anything beyond that seems to be impinging on why I actually enjoy programming.

obviyus 5y ago

I very recently got access to Copilot, while I was in the middle of learning and playing around with Clojure. It’s surprisingly useful when you’re not sure about how you want to proceed. E.g. While I was trying to make a simple function for printing all palindromic numbers under 10,000, Copilot inferred from the function name what I was trying and suggested a function using threading macros (something I hadn’t yet come across in Clojure). The result was a much neater affair than what I came up with on my own. I feel it could be a fantastic way to build familiarity with a new language.

Waterluvian 5y ago
I hear you. But I see this as a horrible thing. Now people are going to be further absolved of responsibility of thinking through the problem first and then implementing it after. Won’t this result in more junk code?
- judofyr 5y ago
  Shouldn’t the question be whether it’s better or worse than copy-pasting random functions from StackOverflow you found when googling?
  - Waterluvian 5y ago
    Do people actually just copy paste? When I was much greener I never found that to work. The interface had to be thought about. And inevitably there’d be other details to modify. If anything this just makes that easier to do wrongly. It wants to offer you code that’s not yours that you didn’t conceive of thoughtfully.
    - deleted 5y ago
      [deleted]
    - stingraycharles 5y ago
      I’ve copy/pasted parts of code, but then I typically always link to the actual SO answer in the code. Sometimes it’s just the most pragmatic thing to do.

claviska 5y ago

> GitHub Copilot is a tool that helps you write better, faster, and most importantly, more code. I’ll agree with faster and more code, but from the many examples I’ve seen, it’s not better.

dkersten 5y ago
Its not even clear to me that writing faster and more code is necessarily a good thing, or, at least, that beneficial. I've said it in a previous Copilot discussion and I'll say it again, but actually writing code is a small part of what I do as a programmer. I spend much more time figuring out what the requirements even are, refining them, figuring out what that even means in terms of code, figuring out the overall architecture, how it fits in with other systems or existing code, what data formats it uses, how it handles faults, persistence, scale, security. How it interfaces with the outside world (UI or API). Besides the code itself, I also spend a lot of my time on writing tests (which I wouldn't want to pawn off on an AI outside of fuzzing or generating data for property-based tests; unit tests should mirror what the spec dictates and needs to test the correct things) and on writing documentation. Yes, the code does take up a good chunk of time, but really, its the easy part of my day! Also, speeding through the code means I'm not thinking about it very deeply. That's when I introduce the most bugs, design flaws or shortcomings that bite me later. I wonder if we'll end up with a situation like the old quote about code reviews: a ten line code review gets a hundred comments/suggestions/questions, a thousand line code review gets a ship it. If much of the code is written for us, will we have the attention span to scrutinize it and understand it deeply? Or will our eyes eventually just glaze over as we go yeah its probably fine, ship it.
- b9a2cab5 5y ago
  This is exactly why I think Copilot (and other "AI writes code for you" solutions) are going to fail. It's harder to read code than it is to write it. That's the opposite of how English works.

the_lonely_road 5y ago

CoPilot for a few years while we train it and then in 2037 introducing Microsoft Pilot.

esperent 5y ago
Yes but 2027. If that.

ekster 5y ago

For me the AI part of this is all the bad parts. What would really be cool is better search, templating, and best practices at your fingertips in the IDE. Exact same problem to solve, because it is an important an interesting one, but a totally different approach. Maybe AI assisted somehow, but human curated. Otherwise, imagine how bad legacy codebases of the future will be when they are full of autocomplete code that nobody understands or cared enough to think through even originally.

bsenftner 5y ago
We should have AI tools to aid software engineers understanding of logic chains, and assorted visualizations like CAD, but for logic and the code creating that logic. And not UML nonsense, but some type of AI tool that begins where Doxygen ends and simply keeps going with various means of aiding the developer's understanding as they construct hierarchical logic systems.
- ekster 5y ago
  Totally agree. Something that could interrogate and answer arbitrary questions about complex or new codebases would be incredible.
vladiliescu 5y ago
I think we'll have smarter AI shifting through those legacy codebases of the future, not sure if that'll be a big problem. The way I see this, GitHub Copilot and the like are true next-gen compilers, translating English into code. They'll only get better with time.
- deleted 5y ago
  [deleted]

softwaredoug 5y ago

> We’ll also code reviews. Lots and lots of code reviews. Like, all the time. The algorithm will have to be kept in check. This is a repeated theme of the article. I think it’d be simpler to write the non boilerplaty code, no? Plus who’s going to be excited to have a job as “AI code reviewer”

vladiliescu 5y ago
You know that gif where Gary Oldman yells 'EVERYONE'? :P I've got a sneaking suspicion that this is what we'll be doing in the future, I see Copilot as a taste of what future compilers will be like. In a few years I expect to have most of an application built by an AI, with me developing the business/core logic by hand.
- dathinab 5y ago
  > In a few years I expect to have most of an application built by an AI, with me developing the business/core logic by hand. I can see that too be preferably not with the approach done by GitHub.
dathinab 5y ago
It's worse then that, because the AI is likely to reproduce the kinds of bugs which where overlooked by the source it learned it from. At the same time code reviews are hard, much harder then writing code. So making it faster to write boiler code at the cost of harder code reviews seems to not be a good trade of for me.

Bitwit 5y ago

What if I slip in various 0-days? If crafted carefully, the sky is the limit. I'll try it and see what happens. I just have to know... This worries me.

qayxc 5y ago
This would be even more difficult to achieve than previous attempts (e.g. in the Linux kernel [0]) due to the fact that an attacker needs to corrupt thousands of repositories that are guaranteed to be part of the training set. Potential attackers would have two problems: 1) getting malicious checked into many repos and 2) making sure that these repos find their way into future deployed versions of GPT-3/Codex/CoPilot. CoPilot generates enough vulnerable code as-is [1], so the extra effort isn't even required. [0] https://www.bleepingcomputer.com/news/security/linux-bans-university-of-minnesota-for-committing-malicious-code/ https://www.bleepingcomputer.com/news/security/linux-bans-un... [1] https://cyber-reports.com/2021/07/14/devsecai-github-copilot-prone-to-writing-security-flaws/ https://cyber-reports.com/2021/07/14/devsecai-github-copilot...
UncleMeat 5y ago
The number of people who never write a vuln normally but would write a vuln if they were using machine synthesized code has got to be fewer than ten people on the planet.
remram 5y ago
Crafting might not be necessary. You might find a vulnerability in a commonly copiloted piece of code, and now you can exploit it in many projects. Better yet, those snippets cannot be updated even if Copilot improves, and there is nothing to file a CVE against either.

playcache 5y ago

> GitHub Copilot is a tool that helps you write better, faster, and most importantly, more code. I don't have a lot of faith in the author's code, if that is there opening statement ("better")

vladiliescu 5y ago
That makes two of us, I don't have a lot of faith in my code either.

blunte 5y ago

The AI coding approach is solving the wrong problem. The problem isn't with the low level detailed work. That problem should be solved by building composable, tested, audited libraries. Legos, if you will. If the problem is trivial enough that you can completely trust the AI coded solution, then you could have either done it yourself very easily or used a premade solution from a good library or toolkit. If the problem is not trivial, then you have the outsourcing challenges (which apply to a lot more scenarios than using AI to help you code). If you are not personally capable of judging the outsourced work, then whether you use AI or type it yourself, you will end up with errors or misfeatures. If you are capable of judging, then you must pay attention and read/review. So your job shifts from defining the problem and programming a solution to defining the problem and reviewing potential solution(s). Either way, you must focus and think. But again, perhaps you would be better off building a solution composed of known good blocks. <- This should be the future of software development... Sadly, I think that open source and freedom to (re)invent has worked against us in the long run. If instead of each of us going off and thinking, "I can make a better language/framework", we had built on existing technologies, I daresay we would be further along. To be fair of course, some level of dissatisfaction and divergence would be necessary or we would still be using assembly. Github does have one thing right though (from a business perspective) - they are making a remedy to a symptom, and in that they can expect longer term revenue than if they actually solved the core problem.

sktrdie 5y ago
I think you’re missing what the problem they’re trying to solve is. They’re solving the problem of “writing code faster” With that in mind I think this problem is totally real and worth solving. If copilot can save me those mundane moments during my day where I have to figure out “how to do this common thing that I already did 100 times” then it’s a win for everyone. It’s not trying to solve the whole of programming. It’s just a nice tool to let you actually concentrate on the non-automated parts of coding such as: actually translating requirements into code.
- zabzonk 5y ago
  > how to do this common thing that I already did 100 times You don't do it 100 times - you do it once, test it thoroughly, and put it in a library.

drevil-v2 5y ago

> code reviews code reviews code reviews So basically shift the cognitive burden on to your coworkers? And what if they are also using copilot hoping that you will sanity check it’s output for them? Tit for tat prisoner’s dilemma and no one even realises they are playing the game..

frabcus 5y ago
Presumably copilot will easily do (bad!) code reviews as well, trained on all the world’s PR comments…
- ptx 5y ago
  Maybe a future version could even write the requirements for the application. And when the users can't figure out how to use whatever comes out, we can have an AI do that part too.
pdelgallego 5y ago
That is what I like (at least in theory) of AWS CodeGuru. It helps you to detect bugs and common bad practices automatically in the code review.

kristiandupont 5y ago

So far, I am feeling positive about Copilot. I've used it for about a week and it has been useful at times. Like the author, I've mostly found it useful in situations where I am doing something repetitive. I definitely need to look over suggestions carefully though. I don't think it will go much further than that and I don't know what that would even look like anyway, unless I could actually start discussing architectural decisions with it like I do with a human pair programmer. I guess you could say that this is what the comments are for, so who knows.

losvedir 5y ago

I understand some of the legal implications to be regurgitating licensed code verbatim. But, what about this: what if the current Copilot is working out the kinks, and the real product is per-organization models with transfer learning using their repos' code? At work, we store all our code in our GitHub repo, some public, some private. As-is, I think there's a lot of legal ambiguity around using Copilot, but if all that code just served to teach the model structure of programs and common syntactical constructs, but then it had another layer with our code and its idioms, modules, names, then maybe it would regurgitate our code in a way that's useful and doesn't run afoul of licenses. I'm thinking of a fast.ai course I did where I took a base model trained on generic image data, and then did transfer learning on top where I fed it labeled images of Go games and Chess games, and with only maybe 100 of those images it learned to distinguish the two with shocking accuracy. As I understand it, the base model taught it how to look for things like lines, corners, contrast, etc, and then it could be easily specialized. Could something similar be the case here?

vladiliescu 5y ago
Yes, I think so too. Only not in the near future, as we don't really have enough computing power to make that feasible. I've written a few thoughts about this here https://vladiliescu.net/github-copilot-first-impressions/#potential https://vladiliescu.net/github-copilot-first-impressions/#po...

zaptheimpaler 5y ago

Microsoft just stole all the code on github to do this. Regardless of what the minutiae of the law say, no one really expected their work to be used this way. Open source code powers a huge chunk of the industry while capturing little value for the maintainers already. Github even explicitly supports a standard format for declaring the license of a repo, which was cleverly ignored. Here is the relevant section from Githubs privacy policy [1] > 6. Contributions Under Repository License > Whenever you add Content to a repository containing notice of a license, you license that Content under the same terms, and you agree that you have the right to license that Content under those terms. If you have a separate agreement to license that Content under different terms, such as a contributor license agreement, that agreement will supersede. From GPLv2, "When distributing derived works, the source code of the work must be made available under the same license." ------ This is not about technology, it is a legal endrun around using open source code without open sourcing derived work. It is using AI as a form of "license laundering". "OpenAI" is not open at all. Truly open AI means the code, the data and the model are all open. OpenAI sold the source to GPT-3 to Microsoft, received $1 billion from them in 2019 and does not make most of their work available except behind a highly exclusive, paid API - https://beta.openai.com/pricing/ https://beta.openai.com/pricing/. Its a joke to call that "open". I urge you to read up on OpenAI and look at what the have actually done. Their plan in the future is to sell access to Copilot, directly monetizing work they stole from others for free: > According to GitHub, “If the technical preview is successful, our plan is to build a commercial version of GitHub Copilot in the future.” I've deleted all my code from github and hope others do the same. Maybe if some bigger profile project starts doing this, we can start to organize around opposing Pilot and OpenAI. Others have also pointed out similar concerns - see https://news.ycombinator.com/item?id=27687450 https://news.ycombinator.com/item?id=27687450 for example. [1] https://docs.github.com/en/github/site-policy/github-terms-of-service#6-contributions-under-repository-license https://docs.github.com/en/github/site-policy/github-terms-o... [2] https://beta.openai.com/pricing/ https://beta.openai.com/pricing/

6gvONxR4sf7o 5y ago
It’s a shame that copilot would not be possible without all the zillions of hours of work that went into writing that code, while the authors of that training data get zero compensation for their contribution to copilot (and zero ability to opt out).

ahofmann 5y ago

It is kind of impressive, that Copilot understood the misspelled "einz" and translated it as "one" ("eins" would have been correct).

vladiliescu 5y ago
Ouch, thanks for pointing this out, my German has a long way to go.

ipaddr 5y ago

This might work for some domains but I'm trying to write less code and abstract where possible. This writes more code and doesn't help design the application. It's a function autocompleter but it doesn't take my abstractions into account.

IfOnlyYouKnew 5y ago

It's good to read a less dramatic take on Copilot. The initial echo chamber of outrage felt rather strange, especially considering the usual attitude when people point out risks in AI systems. I guess everyone else's reading-file-line-by-line loops in python are the epitome of creativity, and being inspired by them is its own category of crime compared to the exact same thing happening in uncreative professions such as photography, music, or writing? And not being hired because the algorithm prefers people who played lacrosse in school is, like, your problem, because waiting to release models until they do not harm anyone would seriously mess with our agile process. As an aside, I really enjoyed the writing style. The subtle humour is better at signalling competence and friendliness than any CV ever could.

vladiliescu 5y ago
Thank you :)
ptx 5y ago
> I guess everyone else's reading-file-line-by-line loops in python are the epitome of creativity I would certainly hope not, since there is barely any code to write: for line in my_file: ... For even more convenience in common cases, there is the fileinput module[1] in the standard library. I can see how a boilerplate-generating AI could be helpful in a more boilerplate-heavy language like Java, but a better solution is to use a language that better suits your usecase and lets you express it without the boilerplate. [1] https://docs.python.org/3/library/fileinput.html https://docs.python.org/3/library/fileinput.html
- vladiliescu 5y ago
  I'd say using a better language is an easy decision to make when you're in a team of one and not depending on any framework-specific features and somewhat harder when you're in a team of more, depending on some framework-specific features, or both.

DantesKite 5y ago

Not a single positive comment eh? Github Copilot and its iterations are the future. You can complain and whine about what problems are being solved, how it'll affect human developers (making them weaker instead of stronger over time). And to some extent, that's probably true. But it's still the future and it's coming. It's already here.

solipsism 5y ago
Is this your idea of a positive comment? "It's happening no matter what!"
- alphachloride 5y ago
  I think he means a comment in support of Copilot's advancements and not one criticizing its kinks.

leksak 5y ago

One thing not mentioned in this article but that's on Github Copilots page is that they imagine it'll be useful for learning how to code. > ... or just learning to code I've done some teaching and my mental model for what is necessary to learn a language and, more generally, learning how to program and my view is that the rudimentary boilerplate-y type of stuff that this tool seems to excel at are mindless to most are essential for beginners as part of their learning. Any educators here with different ideas or thoughts?

arduinomancer 5y ago

To me it seems like copilot only helps with local code, which is absolutely not the bottleneck (it’s maybe 20% of my time) I spend much much more time figuring out how to convert requirements to code and how to structure it non-locally (as in how it fits into the whole codebase) Also if you’re concerned with writing clean code, copy-pasting boilerplate everywhere is not the right approach, you have to actually think about interfaces and abstractions

ellen364 5y ago

It’s early days for CoPilot, but I find myself wondering if it will eventually reduce idiomatic use of languages and increase the stickiness of old practices. In the article, the author included this example: alternate_word_mapping = {words[i]: words_in_english[i] for i in range(len(words))} That line is probably more readable than the Pythonic enumerate: alternate_word_mapping = {word: words_in_english[i] for i, word in enumerate(words)} But it superficially reminded me of a style that I see on Leetcode, which rarely uses Python features like enumerate or dict.items. CoPilot was trained on GitHub repos and many repos are mini-projects for learning a new language. Does that mean CoPilot will tend to suggest more generic, less language specific, implementations? If it does, will that change perceptions of what’s idiomatic? And will the volume of old code on GitHub influence CoPilot’s suggestions, making us slow to adopt new language features?

falcor84 5y ago
I agree in general, but just wanted to add that the pythonic approach would be to avoid the explicit index entirely, with something like: alternate_word_mapping = dict(zip(words, words_in_english))
- vladiliescu 5y ago
  Thumbs up, I totally missed this. Didn’t encounter this approach in the suggestions either. That’s actually one reason why I don’t think it’s current incarnation is a good fit for learning new apis, it’s been trained on a lot of code, not all of it good. Seems like something a future version might be able to fix, perhaps by training a new layer using just demonstrably ‘good’ code?
- ellen364 5y ago
  Agreed and an excellent point. Ironically, my example was not nearly as Pythonic as it could have been! (V late reply as still learning how to keep track of replies on HN. But perhaps you’ll see it anyway.)
deleted 5y ago
[deleted]

ipaddr 5y ago

Can I use this during my leetcode tests. I feel like it would be the perfect use-case

orange_puff 5y ago

I’m not a denier. I believe AI will vastly alter the way we write code. But, to be honest, it’s very depressing to me. I think I am someone who selfishly enjoys writing code, not necessarily getting software built. If my job became designing something at a very fine grained level, feeding it to an AI, having the AI write it and then code reviewing the AI, I’d just switch careers. Unfortunately for me, I’m super early in my career so I hope I can make enough money in the next 20 or so years such that I can retire young.