4 ms·

This is absolutely insane. I had no idea Copilot was this good. The negativity here just seems like sour grapes or weird goal posts. Sure, it makes mistakes a

This is absolutely insane. I had no idea Copilot was this good. The negativity here just seems like sour grapes or weird goal posts. Sure, it makes mistakes and needs verification. But know what also makes mistakes and needs verification? All the code I already manually write as I tediously ratchet towards a solution. Removing some cycles from that process is a win. Just stubbing out close-enough boilerplate is a win by itself, like setting up an NLP pipeline or figuring out which menagerie of classes need to be instantiated and hooked up together to do basic things in some verbose libs/langs.

Mountain_Skies 5y ago
Even though it is based on mostly human written code, Copilot makes mistakes that are different from the type human coders typically make. It will take a different skill set to detect and correct the errors made by systems like Copilot. The same is true for self-driving cars. This doesn't mean that we shouldn't use these technologies, just that there will be adaptations to our behavior we'll need to make if we want to make use of them.
- fn1 5y ago
  > Even though it is based on mostly human written code, Copilot makes mistakes that are different from the type human coders typically make. Can you give an example for this?
  - Satam 5y ago
    Probably not what OP intended but here's what I encountered. When autocompleting code for a Pythonic database query it would finish something like "query.all()" as "query.all().delete()". Meanwhile, as a programmer, I'm usually quite vary of deleting all the records from a database unless I'm very sure that's what I want.
ShamelessC 5y ago
> The negativity here just seems like sour grapes or weird goal posts. Indeed. Every negative comment I have seen here has been a shallow dismissal by someone who clearly hasn't engaged with the tool. I'm not sure why people here are so primed to shit all over anything potentially innovative, seemingly even without background knowledge. Like, is there something inherently offensive to coders about a model that threatens to do their job? Or is it just years and years of people getting burned by previous "AI" projects without knowing that this one is actually rather impressive and comes from good research? Keep shallow dismissals to yourselves people. It's in the site's rules.
- dash2 5y ago
  You might think it's awesome and well-executed, and still think that an automated toxicity classifier is a terrible idea.
  - ShamelessC 5y ago
    It's simply a fun toy tutorial and likely good introductory material for people trying to learn language modeling and classification, two important topics in the broader NLP/machine learning scene. The article isn't making any suggestions about whether or not they are good or bad ideas.
- nimih 5y ago
  This is definitely a very cool tech demo, but I got the same feeling reading this as I do when I read a blog post years ago where a guy walked through using very rigid green-red-green TDD to solve a hairy algorithmic problem[0]: it sort of seems like the person already had the shape of the solution in their head before they started writing the code. Which is maybe the point! As the article points out, remembering the correct incantation to get matplotlib to spit out a bar chart is hard[1]; I certainly have to look it up literally every time (well, these days, I just use tools which have more intuitive APIs, but that's maybe besides the point). I don't really know what it means to "binarize" a dataset, but apparently the language model did, and apparently seeing the giant stack trace when trying to plot a precision-recall curve was enough to prompt the article writer to realize such an operation might be useful. When you're doing exploratory analysis like this, keeping a train of thought going is extremely important, so avoiding paging back and forth to the scikit-learn documentation is obviously a huge win. But, on the other hand, this isn't a "no-code" solution in any real sense, because for all intents and purposes the author really did all the difficult parts which would've been necessary for a "fully coded" solution: they knew the technical outcome they wanted and had very good domain knowledge to guide the solution, and, shoot, they still ended up needing to understand semantics of the programming language and abstractions they were working with in that stacktrace at the end. It's still extremely neat (and, presumably, useful) to see the computer was able to correctly guess at all the syntax and API interfaces for the most part[2], but I don't really think you can fault people for wanting to push back against the idea that this is somehow fundamentally transformative, since I think it's pretty obvious that the human is (still) doing the hard and interesting parts and the computer is (still) doing the tedious and boring parts. Maybe people shouldn't be getting flustered about a click-baity title over-promising a hip new technology, but as you say: > Or is it just years and years of people getting burned by previous "AI" projects without knowing that this one is actually rather impressive and comes from good research? There's definitely some of this. --- [0] I wish I could find the link for this, but I'm very bad at google these days. [1] To risk ascribing agency to a statistical model of github commits, it is sort of funny that the co-pilot pulled in seaborn as a dependency but then did everything directly with calls to plt and DataFrame.plot. [2] I don't really have the expertise myself to tell you whether that scikit pipeline is at all reasonable, I suppose. It sure sounds fancy, though.
csomar 5y ago
Copilot is insanely brilliant and good. Its only issue is that it takes too little context (up to what your cursor is pointing on the file, at least on vim). If it did take all context (your whole project, maybe your shell history, your data files, the imported libraries code, GitHub repo issues/PRs, etc...) and it had some LSP checker for errors, add all of that to GPT-4; and maybe we'll have something that can do complex coding stuff auto-magically.
skhr0680 5y ago
> Sure, it makes mistakes and needs verification. But know what also makes mistakes and needs verification? The problem is when it makes something that looks OK but does the opposite of what you want it to. See: machine translation
deleted 5y ago
[deleted]

Mountain_Skies 5y ago

Even though it is based on mostly human written code, Copilot makes mistakes that are different from the type human coders typically make. It will take a different skill set to detect and correct the errors made by systems like Copilot. The same is true for self-driving cars. This doesn't mean that we shouldn't use these technologies, just that there will be adaptations to our behavior we'll need to make if we want to make use of them.

fn1 5y ago
> Even though it is based on mostly human written code, Copilot makes mistakes that are different from the type human coders typically make. Can you give an example for this?
- Satam 5y ago
  Probably not what OP intended but here's what I encountered. When autocompleting code for a Pythonic database query it would finish something like "query.all()" as "query.all().delete()". Meanwhile, as a programmer, I'm usually quite vary of deleting all the records from a database unless I'm very sure that's what I want.

ShamelessC 5y ago

> The negativity here just seems like sour grapes or weird goal posts. Indeed. Every negative comment I have seen here has been a shallow dismissal by someone who clearly hasn't engaged with the tool. I'm not sure why people here are so primed to shit all over anything potentially innovative, seemingly even without background knowledge. Like, is there something inherently offensive to coders about a model that threatens to do their job? Or is it just years and years of people getting burned by previous "AI" projects without knowing that this one is actually rather impressive and comes from good research? Keep shallow dismissals to yourselves people. It's in the site's rules.

dash2 5y ago
You might think it's awesome and well-executed, and still think that an automated toxicity classifier is a terrible idea.
- ShamelessC 5y ago
  It's simply a fun toy tutorial and likely good introductory material for people trying to learn language modeling and classification, two important topics in the broader NLP/machine learning scene. The article isn't making any suggestions about whether or not they are good or bad ideas.
nimih 5y ago
This is definitely a very cool tech demo, but I got the same feeling reading this as I do when I read a blog post years ago where a guy walked through using very rigid green-red-green TDD to solve a hairy algorithmic problem[0]: it sort of seems like the person already had the shape of the solution in their head before they started writing the code. Which is maybe the point! As the article points out, remembering the correct incantation to get matplotlib to spit out a bar chart is hard[1]; I certainly have to look it up literally every time (well, these days, I just use tools which have more intuitive APIs, but that's maybe besides the point). I don't really know what it means to "binarize" a dataset, but apparently the language model did, and apparently seeing the giant stack trace when trying to plot a precision-recall curve was enough to prompt the article writer to realize such an operation might be useful. When you're doing exploratory analysis like this, keeping a train of thought going is extremely important, so avoiding paging back and forth to the scikit-learn documentation is obviously a huge win. But, on the other hand, this isn't a "no-code" solution in any real sense, because for all intents and purposes the author really did all the difficult parts which would've been necessary for a "fully coded" solution: they knew the technical outcome they wanted and had very good domain knowledge to guide the solution, and, shoot, they still ended up needing to understand semantics of the programming language and abstractions they were working with in that stacktrace at the end. It's still extremely neat (and, presumably, useful) to see the computer was able to correctly guess at all the syntax and API interfaces for the most part[2], but I don't really think you can fault people for wanting to push back against the idea that this is somehow fundamentally transformative, since I think it's pretty obvious that the human is (still) doing the hard and interesting parts and the computer is (still) doing the tedious and boring parts. Maybe people shouldn't be getting flustered about a click-baity title over-promising a hip new technology, but as you say: > Or is it just years and years of people getting burned by previous "AI" projects without knowing that this one is actually rather impressive and comes from good research? There's definitely some of this. --- [0] I wish I could find the link for this, but I'm very bad at google these days. [1] To risk ascribing agency to a statistical model of github commits, it is sort of funny that the co-pilot pulled in seaborn as a dependency but then did everything directly with calls to plt and DataFrame.plot. [2] I don't really have the expertise myself to tell you whether that scikit pipeline is at all reasonable, I suppose. It sure sounds fancy, though.

csomar 5y ago

Copilot is insanely brilliant and good. Its only issue is that it takes too little context (up to what your cursor is pointing on the file, at least on vim). If it did take all context (your whole project, maybe your shell history, your data files, the imported libraries code, GitHub repo issues/PRs, etc...) and it had some LSP checker for errors, add all of that to GPT-4; and maybe we'll have something that can do complex coding stuff auto-magically.

skhr0680 5y ago

> Sure, it makes mistakes and needs verification. But know what also makes mistakes and needs verification? The problem is when it makes something that looks OK but does the opposite of what you want it to. See: machine translation

deleted 5y ago

[deleted]