25 ms·
I've been using the alpha for the past 2 weeks, and I'm blown away. Copilot guesses the exact code I want to write about one in ten times, and the rest of the t
by fzaninotto 5y ago
I've been using the alpha for the past 2 weeks, and I'm blown away. Copilot guesses the exact code I want to write about one in ten times, and the rest of the time it suggests something rather good, or completely off. But when it guesses right, it feels like it's reading my mind.
It's really like pair programming, even though I'm coding alone. I have a better understanding of my own code, and I tend to give better names and descriptions to my methods. I write better code, documentation, and tests.
Copilot has made me a better programmer. No kidding. This is a huge achievement. Kudos to the GitHub Copilot team!
- arcturus17 5y ago> and the rest of the time it suggests something rather good, or completely off In what, proportion roughly?
- fzaninotto 5y agoHard to say, really. When writing React components, Jest tests and documentation, it's often not very far. I found it off when writing HTML markup (which is hard to describe with words).
- handrous 5y agoSeems like it's best classed (for now) as an automated tool to generate and apply all the boilerplate snippets I would be creating & using manually, if I weren't too lazy, and/or too often switching between projects, to set all those up and remember how to use them (and that they exist).
- fabiospampinato 5y agoHow big/complicated are the functions Copilot is autocompleting for you? I'm thinking perhaps reading 10 potential candidates is actually slower and less instructive than trying to write the thing yourself.
- fzaninotto 5y agoIt shows the suggestions line by line, and only shows the best guess. It's not more intrusive than Intellisense. You can actually see all the code blocks Copilot is thinking about if you want to, but that is indeed a distraction.
- fabiospampinato 5y agoThe problem I see with that is that's not possible for it to understand well which code is the best, GPT-3 is trying to mimic human writing in general, the thing is most human code is garbage, if this system was able to understand how to make code better you could keep training it until you had perfect code, which is not what the current system is giving you (a lot of the times anyway).
- fzaninotto 5y agoI guess you miss the point. It's not trying to suggest the perfect code. Only you know it. It's saving you time by writing a good (sometimes perfect) first solution based on method/argument names, context, comments, and inline doc. And that is already a huge boost in productivity and coding pleasure (as you only have to focus on the smart part).
- fabiospampinato 5y agoMaybe you are right, in my experience either the code you have easily available to you (either because another person or a computer wrote it) is perfect for your use case (to the best of your knowledge anyway) or rewriting it from scratch is usually better than morphing what you have into what you need.
- drusepth 5y ago>if this system was able to understand how to make code better you could keep training it until you had perfect code Based on the FAQ, it looks like some information about how you interact with the suggestions is fed back to the Copilot service (and theoretically OpenAI) to better improve the model. So, while it may not understand "how to make code better" on its own, it can learn a bit from seeing how actual devs do make code better and theoretically improve from usage.
- mdellavo 5y agodo you still go over the generated code line by line and touchup in places where it did not do a good job?
- fzaninotto 5y agoIt suggests code line by line, so yes
- mdellavo 5y agoI guess I dont see the point if only 10% of the time it's exactly what you want and the rest of the time you have to go back and touch up the line. Does it train a programmer for accepting less than ideal code because it was suggested? Similar to how some programmers blindly copy code from StackOverflow without modification. Seems like there is a potential downside that's being ignored.
- staticassertion 5y ago> Does it train a programmer for accepting less than ideal code because it was suggested? Similar to how some programmers blindly copy code from StackOverflow without modification. Maybe juniors, but I don't see this being likely for anyone else. I've been using TabNine for ages and it's closer to just a fancy autocomplete than a code assistant. Usually it writes what I would write, and if I don't see what I would write I just write it myself (until either I wrote the whole thing or it suggests the right thing). Of course, barring some variable names or whatever. I don't have it "write code for me" - that happens in my head. It just does the typing.
- corobo 5y ago> I guess I dont see the point if only 10% of the time it's exactly what you want and the rest of the time you have to go back and touch up the line. Think of it as tab complete. If it's wrong, don't hit tab.
- verst 5y agoThis is not true - and I've been using copilot for many months :) It suggests entire blocks of code - but not in every context.
- RivieraKid 5y agoWhat's your estimate of the productivity boost expressed as a percentage? I.e. if it takes you 100 hours to complete a project without Copilot, how many hours will it be with Copilot?
- read_if_gay_ 5y agoI tried TabNine and it wasn’t a huge improvement because what costs the most time isn’t typing stuff but thinking about what to type.
- the_arun 5y agoIn addition, I would like to see GitHub reviewing my code & giving me suggestions on how I could improve. That will be more educative & a tool to ensure consistency across code.
- onlyrealcuzzo 5y agoI'm surprised this doesn't exist. Google, FB, and Apple (and I imagine Microsoft) have a lot of this stuff built in that is light-years better than any open source solution I'm aware of. Given that MS owns GitHub and how valuable this is - I imagine it will be coming soon.
- kyawzazaw 5y agoSonarQube and IntelliJ does it in some way for me.
- tomnipotent 5y ago+1 for SonarQube. Very easy way to add value to a project without a lot of overhead.
- fzaninotto 5y agoI'm not sure I spend less time actually coding stuff (because I have to review the Copilot code). But the cost of the code I write is definitely reduced, because: - the review from my peers is faster (the code is more correct) - I come back less to the code (because I have thought about all the corner cases when checking the copilot code) - As I care more about naming & inline docs (it helps copilot), the code is actually cheaper to maintain.
- gmaijoe 5y agodo you have an invite? very interested to check it out
- hobofan 5y agoHave you used any intelligent code completion in the past? E.g. I'd really be interested how it compares to TabNine[0], which already gives pretty amazing single line suggestions (haven't tried their experimental multi-line suggestions yet). [0]: https://www.tabnine.com https://www.tabnine.com
- fzaninotto 5y agoI have used IDEs with good knowledge of the types and libraries I'm using (e.g. VSCode with TypeScript). They offer good suggestions once you start typing a function name. But nothing gets close to Copilot. It "understands" what you're trying to do, and writes the code for you. It makes type-based autocompletions useless.
- grandchild 5y agotabnine works quite similarly to copilot. it's not a thing that "knows about types and libraries", it's a similar predictive machine learning method as copilot seems to use.
- pgib 5y agoI've been using TabNine for a couple years – constantly impresses me, especially how quickly it picks up new patterns. I wouldn't say it's doing my job for me, but definitely saves me a lot of time.
- ayush--s 5y agoI'm curious as to how relevant Copilot would be when autocompleting code that is specific to my codebase in particular, like Tabnine completes most used filters as soon as I type the db table name for the query. I'm a big tabnine fan because it provides this feature. I'm much more often looking to be suggested a line than an entire function because I'm mostly writing business logic. also tabnine is useless in multi-lines completes. which is where co-pilot should be strong.
- meowface 5y agoYeah, I've been very happy with Tabnine for a while, but the prospect of good multi-line completions is appealing. I might try running both Tabnine and Copilot simultaneously for a bit to A/B test.
- softwaredoug 5y agoIt seems to replace/shorten the loop of “Google for snippet that does X” copy, paste, tweak, no? Which of course is super cool for many tasks!
- fzaninotto 5y agoIt's smarter than that. It suggests things that have never been written. It actually creates code based on the context, just like GPT-3 can create new text documents based on previous inputs. Edit: Check this screencast for instance: https://twitter.com/francoisz/status/1409908666166349831 https://twitter.com/francoisz/status/1409908666166349831
- semi-extrinsic 5y agoAnyone tried this on any sort of numeric computation? Numpy, Julia, Pandas, R, whatever? I definitely see the utility in the linked screencast. But I am left to wonder whether this effectiveness is really a symptom of the extreme propensity for boilerplate code that seems to come with anything web-related.
- aqme28 5y agoI'm not convinced that code snippet in the screencast had never been written. It's fairly generic React, no?
- Jeff_Brown 5y agoIs the model similar to GPT-3? That seems extremely resource-intensive, particularly given how fast the suggestions appear.
- feross 5y agoI’ve also been using the Alpha for around two weeks. I'm impressed by how GitHub Copilot seems to know exactly what I want to type next. Sometimes it even suggests code I was about to look up, such as a snippet to pick a random hex color or completing an array with all the common image mime-types. Copilot is particularly helpful when working on React components where it makes eerily accurate predictions. I see technology like Copilot becoming an indispensable part of the programmer toolbelt similar to IDE autocomplete for many people. I also see it changing the way that programmers document their code. With Copilot if you write a really good descriptive comment before jumping into the implementation, it does a much better job of suggesting the right code, sometimes even writing the entire function for you.
- pfraze 5y agoThey finally did it. They finally found a way to make me write comments.
- oars 5y agoThe real purpose of this tool.
- spec-obs 5y agospot the Github PR folks!
- qaq 5y agoThey were replaced by GPT too :)
- cercatrova 5y agoMore like GPT three :)
- FreeSpeech 5y agoHas anyone used Copilot with a more succinct language? It appears to only automate boilerplate and rudimentary patterns, which while useful in repetitive low signal to noise ratio languages like React or Java, sounds less appealing if you're writing Clojure.
- asdfman123 5y agoPack it all up, boys, programming's over. Hello, AI. Anyone want to hire me to teach your grandma how to use the internet?
- anthropodie 5y agoFew days back, Sam Altman tweeted this "Prediction: AI will cause the price of work that can happen in front of a computer to decrease much faster than the price of work that happens in the physical world. This is the opposite of what most people (including me) expected, and will have strange effects" And I was like yeah I gotta start preparing for next decade.
- deleted 5y ago[deleted]
- jostmey 5y agoI am blown away but not scared for my job... yet. I suspect the AI is only as good as the training examples from Github. If so, then this AI will never generate novel algorithms. The AI is simply performing some really amazing pattern matching to suggest code based on other pieces of code. But over the coming decades AI could dominate coding. I now believe in my lifetime it will be possible for an AI to win almost all coding competitions!
- asdfman123 5y agoI guess it's worth pointing out that the human brain is just an amazing pattern matcher. They feed you all these algorithms in college and your brain suggests new algorithms based on those patterns.
- jostmey 5y agoHumans are more than pattern matchers because we do not passively receive and imitate information. We learn cause and effect by perturbing our environment, which is not possible by passively examining data. An AI agent can interact with an environment and learn from its environment by reinforcement learning. It is important to remember that pattern matching is different from higher forms of learning, like reinforcement learning. To summarize, I think there are real limitations with this AI, but these limitations are solvable problems, and I anticipate significant future progress
- blueblisters 5y agoWith VSCode, Github and a perhaps a little bit of help from OpenAI, Microsoft is poised to dominate the developer productivity tools market in the near future. I wouldn't be surprised to see really good static analysis and automated code review tools coming out of these teams very soon.
- croes 5y agoAnd still Windows is a mess.
- _fat_santa 5y agoI'd bet money that the VSCode and Windows teams are basically on different planets and Microsoft.
- jrockway 5y agoI bet there are people that use Windows to develop VSCode and use VSCode to develop Windows, so some people probably know each other internally. I think what escapes HN is how massively successful Microsoft is. Sure, the search built into Windows sucks. There are many, many more complicated components of a platform and OS than that, and those seem to work as well as any other platform and OS.
- bostonsre 5y agoCompared to what other operating system(s)? wsl on windows 10 has been amazing to develop and work on.
- croes 5y agoEvery other OS. It's full of legacy APIs and scrapped new APIs. Every release is like two steps one step forward, one step back and one two the side. Just because thousands of companies have written software and drivers for it, it's still existing. If it were released today it wouldn't stand a chance.
- fzaninotto 5y agoSide note: I recently suffered from a tennis elbow due to sub optimal desktop setup when working from home. Copilot has drastically reduced my keystrokes, and therefore the strain on my tenders. It's good for our health, too!
- yodon 5y agoI watched the same thing happen with the introduction of Intellisense. Pre-Intellisense I had tons of RSI problems and had to use funky ergonomic keyboards like the Kinesis keyboard to function as a dev. Now I just hop on whatever laptop is in front of me and code. Same reason - massive reduction in the number of keys I have to touch to produce a line of code.
- croes 5y agoMaybe it's just because humans are not as creative as they think. Whatever you do, thousands of others have done the same already. So no need to pay a high level programmer, just a mediocre one and the right AI assistant gives the same results.
- jqbd 5y ago> So no need to pay a high level programmer, just a mediocre one and the right AI assistant gives the same results. I think of it as not needing juniors for boring work, all you need as a company is seniors and AI.
- sterlind 5y agowithout juniors, how do you get more seniors?
- spaced-out 5y agoMaybe programmers will adopt a similar model to artists and musicians. Do a lot of work for free/practically nothing hoping that some day you can make it "big" and land a full time role.
- kyawzazaw 5y agowe already have that with unpaid interns and PhD students
- Jeff_Brown 5y agoAnd most people who consider the job are driven away by the economics of it.
- ganzuul 5y agoThat is already the recommendation. Pick an open source project and contribute.
- 9dev 5y ago
- gentleman11 5y agoIt could start to replace us in 20 years. Or reduce. It is exciting for now
- handrous 5y agoUntil it automatically knows when and how it's wrong, you'll still need a human to figure that out, and that human will need to actually know how to program, without the overgrown auto-complete. May or may not reduce the demand for programmers, though. We'll see.
- megablast 5y agoThanks captain obvious.
- maficious 5y agoBut that's exactly what you are doing as a programmer who uses it. If you autocomplete using it and then fix the code, you are literally telling it what it got wrong.
- handrous 5y agoI was responding to "It could start to replace us in 20 years." I think if it can do that, it'll basically be AGI and a lot of things will change in a hurry. I don't think that a tool that can do that is even all that similar to this tool.
- iandanforth 5y agoWhat is the licensing for code generated in this way? GPT-3 has memorized hundreds of texts verbatim and can be prompted to regurgitate that text. Has this model only been trained on code that doesn't require attribution as part of the license?
- richardanaya 5y agoYou're using the word "memorized" in a very loose way.
- delaaxe 5y agoHis point still holds, GPT-3 can output large chunks of licensed code, verbatim
- tsbinz 5y agoHow is it loose? Both in the colloquial sense and in the sense it is used in machine learning it is fitting. https://bair.berkeley.edu/blog/2020/12/20/lmmem/ https://bair.berkeley.edu/blog/2020/12/20/lmmem/ is a post demonstrating it.
- mempko 5y agoFinally, a faster way to spread bugs than copy/paste.
- Abimelex 5y agoNot sure how this is handled in US, but in Germany a few lines of code have in general not enough uniqueness to be licensed.
- hakre 5y agoSo the corpus has been compiled under license and the derivative work is eligible for distribution?
- billti 5y agoThe landing page for it states the below, so hopefully not too much of an issue (though I guess some folks may find a 0.1% risk high). > GitHub Copilot is a code synthesizer, not a search engine: the vast majority of the code that it suggests is uniquely generated and has never been seen before. We found that about 0.1% of the time, the suggestion may contain some snippets that are verbatim from the training set.
- BrandonJung 5y agofor those of you like this concept but 1. did not get into the alpha (or want something that has lots of great reviews) 2. need to run locally (security or connectivity) 3. want to use any IDE... please try Tabnine
- chimtim 5y agoUnfortunately, one in 10 times is far from good enough (and this is with good prompt engineering which after using large language models for a while, one starts to do). I feel like the current generation of AI is bringing us close enough to something that works once in a while but requires constant human expertise ~50% of the time. The self-driving industry is in a similar situation of despair where millions have been spent in labelling and training but something fundamental is amiss in the ML models.
- miohtama 5y agoYou are correct. I feel this is why the service is called Copilot, not Pilot :)
- Jeff_Brown 5y agoI keep wondering how much time it could possibly save you, given that you're obligated to read the code and make sure it makes sense. Given that, the testimonials here are very surprising to me.
- ganzuul 5y agoI think 1/10 is incredible. If it holds up it means they may have found the right prior for a path of development that can actually lead to artificial general intelligence. With exponential improvement; humans learning to hack the AI and the AI learning better suggestions, this may in theory happen very quickly. We live in a very small corner of the space of possible universes, which is why finding a prior in program space within it is a big deal.
- mssundaram 5y agoIt sounds similar to the editor plugin called TabNine
- nihaals 5y agoHow did you manage to get early access? Is there some kind of GitHub Early Access programme (or something similar part of Enterprise)?