11 ms·

ML code generation vs. coding by hand: what we think programming will look like

141 points by matijash 4y ago

weego 4y ago
Trying to solve the lack of progress in simplifying the complexity of standard development scenarios needs to be the next step in framework development. I just don't understand how or why it needs to be polluted by the unnecessary added hidden complexity of ML. You're just shifting your unknowns into a black box and it's magic output.
- generalizations 4y ago
  What you said actually sounds pretty similar to what I can imagine was said 'back in the day' when compilers were first developed and languages like C were invented. > Trying to solve the lack of progress in unifying computer architectures needs to be the next step in computer science. I just don't understand how or why it needs to be polluted by the unnecessary added hidden complexity of GCC. You're just shifting your assembly language generation into a black box and its magic output. (I don't actually know how to write assembly, so pardon any technical faux pas. But the comparison seemed apt. ML as an intent->code compiler just seems like an inevitable step forward.)
  - gugagore 4y ago
    GCC is less opaque (less of a black box) than these language models.
  - mywittyname 4y ago
    I can see why you would make this analogy but I personally don't think of code generation and code compilation to be analogous.
- lagrange77 4y ago
  It's what they call 'democratization of code' imo. ML generated code won't be used for high tech software at first, but for the mass of not so critical software like web frontend scripting.
  - Kaze404 4y ago
    I have zero hopes for ML getting web frontend right when real life humans have such a hard time doing so.
- turns0ut 4y ago
  ML is not a “black box”. It’s formalized statistics translated to code by community. Regex and grep are just as “black box magic” to those who read a blog post on how to scan some text files. ML is statistics on arrays, whereas Unix tools are stats for a Unix system. It’s the same abstraction at its core, new implementation.
  - acoard 4y ago
    In a recently reported issue with an Apple Car almost running over a pedestrian, someone on HN asked why can't we simply add the code: if (personInFrontOfCar) { halt(); } The reason, of course, is the self-driving algorithms are all done with machine learning. The model is trained with a corpus of information and the result is a "black box." The programmer could not insert the above `if` statement if they wanted to, they could not modify the source code of the model as if it was normal business logic. Instead, they would have to re-train the model on new data. This is what people mean when they say it is a "black box." There is no source code to edit, only training data that outputs an as-is model. It's similar to getting binary blobs, for eg Linux drivers. In contrast, I could fork regex if I wanted and add an `if` statement in the middle of it, and re-compile it. It is pure source code. Same with grep.
    - MathYouF 4y ago
      From my admittedly limited knowledge of how self driving policy models are implemented, I believe the main black box AI is determining the part `if personInFrontOfCar:` (using object detection, masking, 3D pose estimation, et/or cetera) And actually defined policies on how to handle certain situations given that inference are possible, in the case specifically of self driving vehicles. My assumptions come from when I'd asked someone at NeurIPS in 2019 about what reinforcement learning methods they use as they said "I don't think any self driving companies are using reinforcement learning, it is too risky". I don't mean to imply this is hard to find information either, reading a few papers would be all it takes to clear up the degree to which most self driving policies are "controllable" in the way you describe, or at least to what degree. My main point is that I think it is the inferring of the environment (current state) rather than the chosen policy at each time step which is more of an error prone black box.
oefrha 4y ago
> a huge win for the developer community > allows us to write even less code and care about fewer implementation details Remember Bjarne Stroustrup: “I Did It For You All…”? More code, more complexity — more job security.
- verdverm 4y ago
  it's a lot harder to fix code you didn't write, seems like even more job security for those who can clean up after the "ai"
  - paulmooreparks 4y ago
    Early in my career I worked for a contracting firm that established a secure position in a host company by being willing to take on maintenance of a code base that had been created by a code generator. It wasn't AI-generated, but it was a similar situation where none of the captive employees wanted to slog through the generated code, and my bosses were more than happy to take huge wads of cash to do it. So, the crusty old contractor in me kind of says, "Bring it on."
    - verdverm 4y ago
      Great attitude! As I'm working on a non-AI code generator, I will be relieved to know you, and those like you, are out there to clean up my downstream messes ;]
colordrops 4y ago
The article presupposes that ML is only for code generation but not maintenance. Why couldn't this change in the future? Perhaps you'd have an english language document that describes the functionality, and as you update it, the code gets re-generated or updated.
- matijash 4y ago
  This is an interesting thought. Kinda analogous to when you e.g. change your C++ code and the new assembly code is generated. If there was high-level intent (e.g. something like spec language) that was preserved and versioned, that might be possible I guess.
- marginalia_nu 4y ago
  Wouldn't that be strictly worse than specifying the functionality in code? The reason we've invented programming languages is because natural language is ambiguous and imprecise. Consider the instructions "take a list, add the number three and five, return the list" This compiles to both f(list): 3+5; return list; and f(list): list.add(3&5); return list; and f(list): list.add(3); list.add(5); return list; Decoding this type of vague descriptions is something human programmers struggle with, often resulting in a discussion with whoever wrote the requirements about just what the requirements say (sometimes they don't know and it needs to be worked out).
  - spc476 4y ago
    > This compiles to both [three code segments] Did you intentionally do an off-by-one error?
    - marginalia_nu 4y ago
      Not really, but it furthers my point about natural language being imprecise. Even the grammatical rules are ambiguous and frequently broken. It's as they say: Time flies like an arrow, fruit flies like a banana.
  - 4y ago
mort96 4y ago
Okay but I can't avoid noticing the bug in the copilot-generated code. The generated code is: async function isPositive(text: string): Promise<boolean> { const response = await fetch('https://text-processing.com/api/sentiment', { method: "POST", body: `text=${text}`, headers: { "Content-Type": "application/x-www-form-urlencoded", }, }); const json = await response.json(); return json.label === "pos"; } This code doesn't escape the text, so if the text contains the letter '&' or other characters with special meanings in form URL encoding, it will break. Moreover, these kinds of errors can cause serious security issues; probably not in this exact case, the worst an attacker could do is change the sentiment analysis language, but this class of bug in general is rife with security implications. This isn't the first time I've seen this kind of bug either -- and this class of bug is always shown by people trying to showcase how amazing Copilot is, so it seems like an inherent flaw. Is this really the future of programming? Is programming going to go from a creative endeavor to make the machine do what you want, to a job which mostly consists of reviewing and debugging auto-generated code?
- hourago 4y ago
  Is it cost effective? If companies think that this way of working is cutting development time and/or cost then it's the future. At least in the professional world. Even bugs are acceptable if the advantages of auto generated code are large enough.
  - ModernMech 4y ago
    Depends on the bug and how much damage it causes, I guess. The more nefarious and subtle the bug, the more intimate you have to be with the code to understand it. For developers, this intimacy comes with actually writing the code. I know that when I write code manually, I'm not just tapping keys on a keyboard but building a mental model in my brain. When there's a bug, I can usually intuit where it's located if that model is detailed enough. Maybe developers of the future will just be code reviewers for ML models. Yuck, who wants to do that? So then will we just have ML models reviewing ML generated code? My concern is that ML code generation will create very shallow mental models of code in developers' minds. If everyone in the organization is building code this way, then who has the knowledge and skills to debug it? I foresee showstopping bugs from ML-generated that bring entire organizations to a grinding halt in the future. I remember back in the day when website generators were in vogue, it became a trend for many webdevs to call their sites "handmade" if they typed all the code themselves. I predict we'll be seeing the same in the future in other domains to contrast "handwritten" source with ML generated source. What's that famous quote? "Debugging is twice as hard as writing the code in the first place." Well where does that leave us when you don't even have to write the code in the first place? I guess then we will have to turn to ML to handle that for us as well.
aaaaaaaaaaab 4y ago
>[...] thank you for your generous comments, ideas and suggestions! You made this post better and made sure I don't go overboard with memes :). Yeah man, I'm not sure about the latter. Not sure...
- matijash 4y ago
  Haha you should have seen the first draft! Still appreciate the feedback
tehsauce 4y ago
“Although ML code generation helps with getting the initial code written, it cannot do much beyond that - if that code is to be maintained and changed in the future (and if anyone uses the product, it is), the developer still needs to fully own and understand it.” I think this is a misconception. It’s true for these first prototype code generation tools, but there’s no reason to think that in the future these models won’t be adapted to modify/maintain code too.
spicyusername 4y ago
Machine Learning algorithms are only as good as the data they are trained on. For tasks like self-driving or spotting cancer in x-rays, they are producing novel result because these kinds of tasks are amenable to reinforcement. The algorithm crashed the car, or it didn't. The patient had cancer, or they didn't. For tasks like reproducing visual images or reproducing text, it _seems_ like these algorithms are starting to get "creative", but they are not. They are still just regurgitating versions of the data they've been fed. You will never see a truly new style or work of art from DALL-E, because DALL-E will never create something new. Only new flavors of something old or new flavors of the old relationships between old things. Assuming that it is even possible to describe novel software engineering problems in a way that a machine could understand (i.e. in some complex structured data format), software engineering is still mostly a creative field. So software engineering isn't going to performed by machine learning for the same reason that truly interesting novels or legitimately new musical styles won't be created by machine learning. Creating something new relies on genuine creativity and new ideas and these models can only make something "new" out of something old.
- lazide 4y ago
  Kinda - if you have a problem amenable to some kind of evolutionary training model/feedback, you could certainly improve it via ML. Anything from compression to data processing can and has been improved through evolutionary training processes. If you trained an ML model on recognizing patterns of older less efficient algorithms and swapping in new ones, that could be useful. It probably isn’t that much harder of a problem than knowledge graph building and objecg recognition. Also, having an ML model trained to recognize certain types of problem patterns in whatever source product/model/codebase or whatever and then spit out code to solve them is certainly not a theoretically impossible or low value task. That said, ML as an art or science is no where near being able to do that usefully near as I can tell. Hell even basic chatbots still suck. Hell, even getting Hibernate to usefully reverse engineer a pretty normal schema still sucks.
- joe_the_user 4y ago
  Machine Learning algorithms are only as good as the data they are trained on. Machine Learning algorithms are never better than the data they're trained-on. But they can easily be "worse". Specifically, an ML algorithm trained on one data set can have a hard time operating on a different data set with some similarities and some fundamental differences. This is related to algorithms generally not having an idea how accurate their predictions/guesses are. This in turn relates to current ML as being something like statistical prediction with worries about bias tossed-out (which isn't to dismiss it but to illuminate it's limitations). For tasks like self-driving or spotting cancer in x-rays, they are producing novel result because these kinds of tasks are amenable to reinforcement. The algorithm crashed the car, or it didn't. The patient had cancer, or they didn't. Ironically, both those applications have been failures so-far. Self-driving is far more complex than a binary crash-or-not scenario (usually the road is predictable but just about anything can wander into the roadway occasionally and you need to deal with this "long tail"). It also requires extreme safety to reach human levels. Diagnosis by image has problems of consistence, of doctors considering more than just an image and other things possibly not understood.
just_boost_it 4y ago
There's only 2 options, either you train on the full mediocrity of Github, or you over-train on highly rated projects, effectively just copy/pasting those projects and riding roughshod over any licensing the authors think they have. At least the mediocrity of the suggested code suggests they're trying to do the former.
cassac 4y ago
The problem with this is that, just like visual programming, it doesn’t really do the hard parts. I remember hearing “a million monkeys with a million type writers writing for a million years could write Shakespeare”. Sure, but how would they know it? How could they recognize when they have achieved their goal? ML will lower the bar for the trivial tedious things and let people believe they are more capable than they are. Even with ML you will have to know what to ask and what the answer should be. That is always the hard part.
- potatoman22 4y ago
  Automating the easy parts does give you more time to work on hard things, though.
dgb23 4y ago
Garbage in garbage out.
datpuz 4y ago
I've been using a Copilot for a while. It's neat, saves me some typing sometimes, but it's not even remotely close to doing even a small portion of my work for me.
- nsxwolf 4y ago
  I couldn't find any use for it. It just bloated my IDE and added latency, and never helped me accomplish a single thing. I deleted the plugin. I guess if it did one thing it was reinforce something I already knew about my job - writing code is always the easy part.
- dimgl 4y ago
  Yeah Copilot has been pretty useless for me. I'm not really sure what this is indicative of. Maybe my coding style is not that common? I'm unsure. Mostly used it with Go and configuration files.
deleted 4y ago
[deleted]
jrm4 4y ago
The momentum behind this whole thing feels strongly like a handwavy attempt to subvert the GPL and similar.
- luma 4y ago
  How does one differentiate? Say I spend some time looking at GPL code, learning the lessons presented, then writing my own code using what I’ve learned. Is this OK if I am working on non-GPL software? Is the answer different if an algorithm does the same thing?
  - jrm4 4y ago
    Yes, because an algorithm isn't a person, and necessarily got to where it got to through what the law clearly defines as "copying," whereas what the human does is expressly not "copying." Again, always good to remember the thing in the background: This is not a philosophical debate, it's a legal one. Intellectual property, for better or worse, is a made-up concept by humans to try to encourage more creation.
    - Siira 4y ago
      Intellectual Property is marketed as encouragement for more creation. At this point, its primary purpose is rent-seeking, like most old institutions, I guess.
      - jrm4 4y ago
        I mostly agree, but when I begin to game theory it out* in my own head, I still think it's something worth reforming, not abolishing. *for the internet, it would simply transfer literally all the power to the youtubes and spotifys.
      - snickerbockers 4y ago
        I agree with you, but as long as megacorps are abusing it to protect their pseudo-monopolies, they should be held up to their own standards. Look at how pissy they get when people reverse-engineer their proprietary systems and then implement unofficial replacements.
marmada 4y ago
I think it is strongly likely that Copilot will be 1 million times better by the end of the decade. Increasing compute, new techniques, etc The top comment on this post is complaining about a bug in Copilot generated code. 1 million times better Copilot won't do that.
- truculent 4y ago
  If you use a formal system with a checker/compiler, there are certain properties that you can guarantee with essentially total certainty. I’m not sure a stochastic system like ML can ever do that. Perhaps there is research on providing mathematical guarantees and constraints on complex models like neural networks? If not, it feels like it would be harder to give a model a high degree of control. Although embedding the model in a system that did provide guarantees (e.g. using a compiler) might be a pragmatic solution?
  - matijash 4y ago
    yep that would be really interesting! If e.g. the target generated code was some language with high type safety or a DSL whose compilation already guarantees certain properties.
  - flooo 4y ago
    Stochastic methods may not be able to do so in isolation, but they can be used in tandem with other approaches such as in “A Deep Reinforcement Learning Approach to First-Order Logic Theorem Proving” https://ojs.aaai.org/index.php/AAAI/article/view/16780/16587 https://ojs.aaai.org/index.php/AAAI/article/view/16780/16587
- trention 4y ago
  Given the fact that those models seem to improve logarithmically and that they pretty much trained codex on ~all available public code (maybe divide by 2) meaning the training data will at most scale with the growth rate of public repos in Github, I doubt that whole "1 million times better". Even the number of parameters won't improve with a factor of 1 million (it will not improve by a factor of a 1000 and may even not reach a factor of 100) by the end of the decade.
truculent 4y ago
I think it’s hard to view ML generated code as the future when you see how powerful type-driven development can be with the likes of Idris and others. Perhaps the best approach could use ML within the constraints provided by a type system. Does research like this exist?
zaptheimpaler 4y ago
The problem with all of these arguments is we are just shooting in the dark. The day before Copilot launched, if someone had told me code generation at the fairly decent quality Copilot achieves was already possible I probably wouldn't believe it. I could happily rattle off a couple of arguments for why it might be better than autocomplete but could never write a full function. Then it did. Who can say how far it's come since then? I think only the Copilot team knows. I wish we could hear from them, or some ML experts who might know?
- YeGoblynQueenne 4y ago
  Generating entire functions has been possible for a long time before Copilot. The discipline that studies algorithms that generate code or programs is called Program Synthesis. For a recent review see Gulwani et al: Program Synthesis https://www.semanticscholar.org/paper/Program-Synthesis-Gulwani-Polozov/f0a79cb16c3e53e9c536b65ec7b2157db1d352f1 https://www.semanticscholar.org/paper/Program-Synthesis-Gulw... Copilot and similar systems (like DeepMind's AlphaCode) are a step backwards compared with earlier systems most of which are capable of not only generating code, but generating correct programs, where correctness is measured with respect to a specification, which can be complete or incomplete (as in input/output examples etc). Systems like Copilot instead use a large language model to generate code by completing an initial "prompt" and have no way to control what kind of code is generated or whether it does what the programmer expects it to do. The task of checking correctness is left to the user. The result is code that may or may not contain bugs to an extent difficult to predict beforehand. Specifically about Copilot, it's based on Codex, a large language model that's a variant of GPT-3 fine-tuned on all of github. In formal testing, OpenAI found that Codex could generate the correct program about 30% of the time, which is pretty low: Evaluating Large Language Models Trained on Code https://arxiv.org/abs/2107.03374 https://arxiv.org/abs/2107.03374 Interestingly, if Codex was allowed 100 "guesses", the correct program was in there 75% of the time, instead. But that only goes to show that these kinds of systems are code firehoses without any ability to tell a correct program from garbage.
layer8 4y ago
If we imagine that at some point in the future, the majority of code would be produced by ML code generation, then where will the training data for future ML models come from? With the current way of ML code generation, it seems that there will always need to be a critical mass of coders who produce code on their own, to be able to serve as new input for the ML models, to be able to learn about new use cases, new APIs, new programming languages, and so on. ML code generation may serve as a multiplier, but it raises questions about the creation and flow of new knowledge and best practices.
dustingetz 4y ago
AI coding is not constrained to javascript, which is a function of human knowledge inertia and hiring practices, all irrelevant to AI-assisted coding. Ideally syntax becomes a non issue and we can start valuing correctness and maximizing business leverage per LOC. However we don’t know how long the transition window will be. Everything is accelerating, so quickly I hope!
FredPret 4y ago
My work is 90% business and 10% code. I do not see how we can have AI writing useful code on its own without also understanding lots of context about human society. When I code something out, I have to first know which problem to solve, and why it’s a problem. I then have to understand the problem in minute detail. This usually involves a slew of human factors, desires, and interfacing with idiosyncratic systems that evolved a certain way because of human factors and desires.
- iandanforth 4y ago
  This is a fair point, but you may lose 10% of your job. By the time one of our designers hands off their interactive Figma mocks to me this has already been figured out. I think there is plenty of specificity that is required to build things of value, but sufficient specificity is available from the people who don't code to allow a slightly better version of Copilot to do the implementation.
snickerbockers 4y ago
No, it's still a terrible idea even if it's used as an auto-complete. As the article correctly states, the primary job of a software engineer is to maintain existing code, not to write new code. You're not going to be doing yourself any favors by committing code that you didn't even read (unless you're one of those people who can get away with shoving all your bugs onto coworkers, in which case you're an asshole).
- LAC-Tech 4y ago
  So devils advocate: - the average developer tenure now is very short - most software companies just want features, features and more features due to market pressure - writing new systems is much more popular than maintaining or upgrading old ones. So maybe this is the future. But it's kind of a sad one.
LAC-Tech 4y ago
It's kind of sad that the future of programming isn't more expressive languages. It's auto-generating code in the languages we have.
kaetemi 4y ago
So far my most practical use of Copilot is prompting it with `// Handle errors`.
pabs3 4y ago
Wonder if anyone is working on using ML for machine code generation.
cdrini 4y ago
I think the biggest misconception with a lot of the comments here is that they are assuming that the code generated by copilot is final. Copilot, as the name suggests, is a human-in-the-loop system. It requires a developer to give it prompts and to massage the code it generates. It's not intended to or will it ever feasibly be able to generate code entirely independently. The point is that this massaging is a lot faster than writing everything from 0 in most cases. It doesn't have to be bug free. It doesn't have to be perfect quality. It doesn't have to do "the hard parts" of coding. It just has to be good enough to be sufficiently faster for the developer than starting from 0, and it is.
AllegedAlec 4y ago
> For the purposes of this post, I will not delve into the questions of code quality, security, legal & privacy issues, pricing, and others of similar character that are often brought up in these early days of ML code generation. Let’s just assume all this is sorted out and see what happens next. "Let's assume magic ponies exist, are commonplace and would love to give us all magic rainbow rides"

141 points by matijash 4y ago

Trying to solve the lack of progress in simplifying the complexity of standard development scenarios needs to be the next step in framework development. I just don't understand how or why it needs to be polluted by the unnecessary added hidden complexity of ML. You're just shifting your unknowns into a black box and it's magic output.

generalizations 4y ago
What you said actually sounds pretty similar to what I can imagine was said 'back in the day' when compilers were first developed and languages like C were invented. > Trying to solve the lack of progress in unifying computer architectures needs to be the next step in computer science. I just don't understand how or why it needs to be polluted by the unnecessary added hidden complexity of GCC. You're just shifting your assembly language generation into a black box and its magic output. (I don't actually know how to write assembly, so pardon any technical faux pas. But the comparison seemed apt. ML as an intent->code compiler just seems like an inevitable step forward.)
- gugagore 4y ago
  GCC is less opaque (less of a black box) than these language models.
- mywittyname 4y ago
  I can see why you would make this analogy but I personally don't think of code generation and code compilation to be analogous.
lagrange77 4y ago
It's what they call 'democratization of code' imo. ML generated code won't be used for high tech software at first, but for the mass of not so critical software like web frontend scripting.
- Kaze404 4y ago
  I have zero hopes for ML getting web frontend right when real life humans have such a hard time doing so.
turns0ut 4y ago
ML is not a “black box”. It’s formalized statistics translated to code by community. Regex and grep are just as “black box magic” to those who read a blog post on how to scan some text files. ML is statistics on arrays, whereas Unix tools are stats for a Unix system. It’s the same abstraction at its core, new implementation.
- acoard 4y ago
  In a recently reported issue with an Apple Car almost running over a pedestrian, someone on HN asked why can't we simply add the code: if (personInFrontOfCar) { halt(); } The reason, of course, is the self-driving algorithms are all done with machine learning. The model is trained with a corpus of information and the result is a "black box." The programmer could not insert the above `if` statement if they wanted to, they could not modify the source code of the model as if it was normal business logic. Instead, they would have to re-train the model on new data. This is what people mean when they say it is a "black box." There is no source code to edit, only training data that outputs an as-is model. It's similar to getting binary blobs, for eg Linux drivers. In contrast, I could fork regex if I wanted and add an `if` statement in the middle of it, and re-compile it. It is pure source code. Same with grep.
  - MathYouF 4y ago
    From my admittedly limited knowledge of how self driving policy models are implemented, I believe the main black box AI is determining the part `if personInFrontOfCar:` (using object detection, masking, 3D pose estimation, et/or cetera) And actually defined policies on how to handle certain situations given that inference are possible, in the case specifically of self driving vehicles. My assumptions come from when I'd asked someone at NeurIPS in 2019 about what reinforcement learning methods they use as they said "I don't think any self driving companies are using reinforcement learning, it is too risky". I don't mean to imply this is hard to find information either, reading a few papers would be all it takes to clear up the degree to which most self driving policies are "controllable" in the way you describe, or at least to what degree. My main point is that I think it is the inferring of the environment (current state) rather than the chosen policy at each time step which is more of an error prone black box.

oefrha 4y ago

> a huge win for the developer community > allows us to write even less code and care about fewer implementation details Remember Bjarne Stroustrup: “I Did It For You All…”? More code, more complexity — more job security.

verdverm 4y ago
it's a lot harder to fix code you didn't write, seems like even more job security for those who can clean up after the "ai"
- paulmooreparks 4y ago
  Early in my career I worked for a contracting firm that established a secure position in a host company by being willing to take on maintenance of a code base that had been created by a code generator. It wasn't AI-generated, but it was a similar situation where none of the captive employees wanted to slog through the generated code, and my bosses were more than happy to take huge wads of cash to do it. So, the crusty old contractor in me kind of says, "Bring it on."
  - verdverm 4y ago
    Great attitude! As I'm working on a non-AI code generator, I will be relieved to know you, and those like you, are out there to clean up my downstream messes ;]

colordrops 4y ago

The article presupposes that ML is only for code generation but not maintenance. Why couldn't this change in the future? Perhaps you'd have an english language document that describes the functionality, and as you update it, the code gets re-generated or updated.

matijash 4y ago
This is an interesting thought. Kinda analogous to when you e.g. change your C++ code and the new assembly code is generated. If there was high-level intent (e.g. something like spec language) that was preserved and versioned, that might be possible I guess.
marginalia_nu 4y ago
Wouldn't that be strictly worse than specifying the functionality in code? The reason we've invented programming languages is because natural language is ambiguous and imprecise. Consider the instructions "take a list, add the number three and five, return the list" This compiles to both f(list): 3+5; return list; and f(list): list.add(3&5); return list; and f(list): list.add(3); list.add(5); return list; Decoding this type of vague descriptions is something human programmers struggle with, often resulting in a discussion with whoever wrote the requirements about just what the requirements say (sometimes they don't know and it needs to be worked out).
- spc476 4y ago
  > This compiles to both [three code segments] Did you intentionally do an off-by-one error?
  - marginalia_nu 4y ago
    Not really, but it furthers my point about natural language being imprecise. Even the grammatical rules are ambiguous and frequently broken. It's as they say: Time flies like an arrow, fruit flies like a banana.
- 4y ago

mort96 4y ago

Okay but I can't avoid noticing the bug in the copilot-generated code. The generated code is: async function isPositive(text: string): Promise<boolean> { const response = await fetch('https://text-processing.com/api/sentiment', { method: "POST", body: `text=${text}`, headers: { "Content-Type": "application/x-www-form-urlencoded", }, }); const json = await response.json(); return json.label === "pos"; } This code doesn't escape the text, so if the text contains the letter '&' or other characters with special meanings in form URL encoding, it will break. Moreover, these kinds of errors can cause serious security issues; probably not in this exact case, the worst an attacker could do is change the sentiment analysis language, but this class of bug in general is rife with security implications. This isn't the first time I've seen this kind of bug either -- and this class of bug is always shown by people trying to showcase how amazing Copilot is, so it seems like an inherent flaw. Is this really the future of programming? Is programming going to go from a creative endeavor to make the machine do what you want, to a job which mostly consists of reviewing and debugging auto-generated code?

hourago 4y ago
Is it cost effective? If companies think that this way of working is cutting development time and/or cost then it's the future. At least in the professional world. Even bugs are acceptable if the advantages of auto generated code are large enough.
- ModernMech 4y ago
  Depends on the bug and how much damage it causes, I guess. The more nefarious and subtle the bug, the more intimate you have to be with the code to understand it. For developers, this intimacy comes with actually writing the code. I know that when I write code manually, I'm not just tapping keys on a keyboard but building a mental model in my brain. When there's a bug, I can usually intuit where it's located if that model is detailed enough. Maybe developers of the future will just be code reviewers for ML models. Yuck, who wants to do that? So then will we just have ML models reviewing ML generated code? My concern is that ML code generation will create very shallow mental models of code in developers' minds. If everyone in the organization is building code this way, then who has the knowledge and skills to debug it? I foresee showstopping bugs from ML-generated that bring entire organizations to a grinding halt in the future. I remember back in the day when website generators were in vogue, it became a trend for many webdevs to call their sites "handmade" if they typed all the code themselves. I predict we'll be seeing the same in the future in other domains to contrast "handwritten" source with ML generated source. What's that famous quote? "Debugging is twice as hard as writing the code in the first place." Well where does that leave us when you don't even have to write the code in the first place? I guess then we will have to turn to ML to handle that for us as well.

aaaaaaaaaaab 4y ago

>[...] thank you for your generous comments, ideas and suggestions! You made this post better and made sure I don't go overboard with memes :). Yeah man, I'm not sure about the latter. Not sure...

matijash 4y ago
Haha you should have seen the first draft! Still appreciate the feedback

tehsauce 4y ago

“Although ML code generation helps with getting the initial code written, it cannot do much beyond that - if that code is to be maintained and changed in the future (and if anyone uses the product, it is), the developer still needs to fully own and understand it.” I think this is a misconception. It’s true for these first prototype code generation tools, but there’s no reason to think that in the future these models won’t be adapted to modify/maintain code too.

spicyusername 4y ago

Machine Learning algorithms are only as good as the data they are trained on. For tasks like self-driving or spotting cancer in x-rays, they are producing novel result because these kinds of tasks are amenable to reinforcement. The algorithm crashed the car, or it didn't. The patient had cancer, or they didn't. For tasks like reproducing visual images or reproducing text, it _seems_ like these algorithms are starting to get "creative", but they are not. They are still just regurgitating versions of the data they've been fed. You will never see a truly new style or work of art from DALL-E, because DALL-E will never create something new. Only new flavors of something old or new flavors of the old relationships between old things. Assuming that it is even possible to describe novel software engineering problems in a way that a machine could understand (i.e. in some complex structured data format), software engineering is still mostly a creative field. So software engineering isn't going to performed by machine learning for the same reason that truly interesting novels or legitimately new musical styles won't be created by machine learning. Creating something new relies on genuine creativity and new ideas and these models can only make something "new" out of something old.

lazide 4y ago
Kinda - if you have a problem amenable to some kind of evolutionary training model/feedback, you could certainly improve it via ML. Anything from compression to data processing can and has been improved through evolutionary training processes. If you trained an ML model on recognizing patterns of older less efficient algorithms and swapping in new ones, that could be useful. It probably isn’t that much harder of a problem than knowledge graph building and objecg recognition. Also, having an ML model trained to recognize certain types of problem patterns in whatever source product/model/codebase or whatever and then spit out code to solve them is certainly not a theoretically impossible or low value task. That said, ML as an art or science is no where near being able to do that usefully near as I can tell. Hell even basic chatbots still suck. Hell, even getting Hibernate to usefully reverse engineer a pretty normal schema still sucks.
joe_the_user 4y ago
Machine Learning algorithms are only as good as the data they are trained on. Machine Learning algorithms are never better than the data they're trained-on. But they can easily be "worse". Specifically, an ML algorithm trained on one data set can have a hard time operating on a different data set with some similarities and some fundamental differences. This is related to algorithms generally not having an idea how accurate their predictions/guesses are. This in turn relates to current ML as being something like statistical prediction with worries about bias tossed-out (which isn't to dismiss it but to illuminate it's limitations). For tasks like self-driving or spotting cancer in x-rays, they are producing novel result because these kinds of tasks are amenable to reinforcement. The algorithm crashed the car, or it didn't. The patient had cancer, or they didn't. Ironically, both those applications have been failures so-far. Self-driving is far more complex than a binary crash-or-not scenario (usually the road is predictable but just about anything can wander into the roadway occasionally and you need to deal with this "long tail"). It also requires extreme safety to reach human levels. Diagnosis by image has problems of consistence, of doctors considering more than just an image and other things possibly not understood.

just_boost_it 4y ago

There's only 2 options, either you train on the full mediocrity of Github, or you over-train on highly rated projects, effectively just copy/pasting those projects and riding roughshod over any licensing the authors think they have. At least the mediocrity of the suggested code suggests they're trying to do the former.

cassac 4y ago

The problem with this is that, just like visual programming, it doesn’t really do the hard parts. I remember hearing “a million monkeys with a million type writers writing for a million years could write Shakespeare”. Sure, but how would they know it? How could they recognize when they have achieved their goal? ML will lower the bar for the trivial tedious things and let people believe they are more capable than they are. Even with ML you will have to know what to ask and what the answer should be. That is always the hard part.

potatoman22 4y ago
Automating the easy parts does give you more time to work on hard things, though.

dgb23 4y ago

Garbage in garbage out.

datpuz 4y ago

I've been using a Copilot for a while. It's neat, saves me some typing sometimes, but it's not even remotely close to doing even a small portion of my work for me.

nsxwolf 4y ago
I couldn't find any use for it. It just bloated my IDE and added latency, and never helped me accomplish a single thing. I deleted the plugin. I guess if it did one thing it was reinforce something I already knew about my job - writing code is always the easy part.
dimgl 4y ago
Yeah Copilot has been pretty useless for me. I'm not really sure what this is indicative of. Maybe my coding style is not that common? I'm unsure. Mostly used it with Go and configuration files.

deleted 4y ago

[deleted]

jrm4 4y ago

The momentum behind this whole thing feels strongly like a handwavy attempt to subvert the GPL and similar.

luma 4y ago
How does one differentiate? Say I spend some time looking at GPL code, learning the lessons presented, then writing my own code using what I’ve learned. Is this OK if I am working on non-GPL software? Is the answer different if an algorithm does the same thing?
- jrm4 4y ago
  Yes, because an algorithm isn't a person, and necessarily got to where it got to through what the law clearly defines as "copying," whereas what the human does is expressly not "copying." Again, always good to remember the thing in the background: This is not a philosophical debate, it's a legal one. Intellectual property, for better or worse, is a made-up concept by humans to try to encourage more creation.
  - Siira 4y ago
    Intellectual Property is marketed as encouragement for more creation. At this point, its primary purpose is rent-seeking, like most old institutions, I guess.
    - jrm4 4y ago
      I mostly agree, but when I begin to game theory it out* in my own head, I still think it's something worth reforming, not abolishing. *for the internet, it would simply transfer literally all the power to the youtubes and spotifys.
    - snickerbockers 4y ago
      I agree with you, but as long as megacorps are abusing it to protect their pseudo-monopolies, they should be held up to their own standards. Look at how pissy they get when people reverse-engineer their proprietary systems and then implement unofficial replacements.

marmada 4y ago

I think it is strongly likely that Copilot will be 1 million times better by the end of the decade. Increasing compute, new techniques, etc The top comment on this post is complaining about a bug in Copilot generated code. 1 million times better Copilot won't do that.

truculent 4y ago
If you use a formal system with a checker/compiler, there are certain properties that you can guarantee with essentially total certainty. I’m not sure a stochastic system like ML can ever do that. Perhaps there is research on providing mathematical guarantees and constraints on complex models like neural networks? If not, it feels like it would be harder to give a model a high degree of control. Although embedding the model in a system that did provide guarantees (e.g. using a compiler) might be a pragmatic solution?
- matijash 4y ago
  yep that would be really interesting! If e.g. the target generated code was some language with high type safety or a DSL whose compilation already guarantees certain properties.
- flooo 4y ago
  Stochastic methods may not be able to do so in isolation, but they can be used in tandem with other approaches such as in “A Deep Reinforcement Learning Approach to First-Order Logic Theorem Proving” https://ojs.aaai.org/index.php/AAAI/article/view/16780/16587 https://ojs.aaai.org/index.php/AAAI/article/view/16780/16587
trention 4y ago
Given the fact that those models seem to improve logarithmically and that they pretty much trained codex on ~all available public code (maybe divide by 2) meaning the training data will at most scale with the growth rate of public repos in Github, I doubt that whole "1 million times better". Even the number of parameters won't improve with a factor of 1 million (it will not improve by a factor of a 1000 and may even not reach a factor of 100) by the end of the decade.

truculent 4y ago

I think it’s hard to view ML generated code as the future when you see how powerful type-driven development can be with the likes of Idris and others. Perhaps the best approach could use ML within the constraints provided by a type system. Does research like this exist?

zaptheimpaler 4y ago

The problem with all of these arguments is we are just shooting in the dark. The day before Copilot launched, if someone had told me code generation at the fairly decent quality Copilot achieves was already possible I probably wouldn't believe it. I could happily rattle off a couple of arguments for why it might be better than autocomplete but could never write a full function. Then it did. Who can say how far it's come since then? I think only the Copilot team knows. I wish we could hear from them, or some ML experts who might know?

YeGoblynQueenne 4y ago
Generating entire functions has been possible for a long time before Copilot. The discipline that studies algorithms that generate code or programs is called Program Synthesis. For a recent review see Gulwani et al: Program Synthesis https://www.semanticscholar.org/paper/Program-Synthesis-Gulwani-Polozov/f0a79cb16c3e53e9c536b65ec7b2157db1d352f1 https://www.semanticscholar.org/paper/Program-Synthesis-Gulw... Copilot and similar systems (like DeepMind's AlphaCode) are a step backwards compared with earlier systems most of which are capable of not only generating code, but generating correct programs, where correctness is measured with respect to a specification, which can be complete or incomplete (as in input/output examples etc). Systems like Copilot instead use a large language model to generate code by completing an initial "prompt" and have no way to control what kind of code is generated or whether it does what the programmer expects it to do. The task of checking correctness is left to the user. The result is code that may or may not contain bugs to an extent difficult to predict beforehand. Specifically about Copilot, it's based on Codex, a large language model that's a variant of GPT-3 fine-tuned on all of github. In formal testing, OpenAI found that Codex could generate the correct program about 30% of the time, which is pretty low: Evaluating Large Language Models Trained on Code https://arxiv.org/abs/2107.03374 https://arxiv.org/abs/2107.03374 Interestingly, if Codex was allowed 100 "guesses", the correct program was in there 75% of the time, instead. But that only goes to show that these kinds of systems are code firehoses without any ability to tell a correct program from garbage.

layer8 4y ago

If we imagine that at some point in the future, the majority of code would be produced by ML code generation, then where will the training data for future ML models come from? With the current way of ML code generation, it seems that there will always need to be a critical mass of coders who produce code on their own, to be able to serve as new input for the ML models, to be able to learn about new use cases, new APIs, new programming languages, and so on. ML code generation may serve as a multiplier, but it raises questions about the creation and flow of new knowledge and best practices.

dustingetz 4y ago

AI coding is not constrained to javascript, which is a function of human knowledge inertia and hiring practices, all irrelevant to AI-assisted coding. Ideally syntax becomes a non issue and we can start valuing correctness and maximizing business leverage per LOC. However we don’t know how long the transition window will be. Everything is accelerating, so quickly I hope!

FredPret 4y ago

My work is 90% business and 10% code. I do not see how we can have AI writing useful code on its own without also understanding lots of context about human society. When I code something out, I have to first know which problem to solve, and why it’s a problem. I then have to understand the problem in minute detail. This usually involves a slew of human factors, desires, and interfacing with idiosyncratic systems that evolved a certain way because of human factors and desires.

iandanforth 4y ago
This is a fair point, but you may lose 10% of your job. By the time one of our designers hands off their interactive Figma mocks to me this has already been figured out. I think there is plenty of specificity that is required to build things of value, but sufficient specificity is available from the people who don't code to allow a slightly better version of Copilot to do the implementation.

snickerbockers 4y ago

No, it's still a terrible idea even if it's used as an auto-complete. As the article correctly states, the primary job of a software engineer is to maintain existing code, not to write new code. You're not going to be doing yourself any favors by committing code that you didn't even read (unless you're one of those people who can get away with shoving all your bugs onto coworkers, in which case you're an asshole).

LAC-Tech 4y ago
So devils advocate: - the average developer tenure now is very short - most software companies just want features, features and more features due to market pressure - writing new systems is much more popular than maintaining or upgrading old ones. So maybe this is the future. But it's kind of a sad one.

LAC-Tech 4y ago

It's kind of sad that the future of programming isn't more expressive languages. It's auto-generating code in the languages we have.

kaetemi 4y ago

So far my most practical use of Copilot is prompting it with `// Handle errors`.

pabs3 4y ago

Wonder if anyone is working on using ML for machine code generation.

cdrini 4y ago

I think the biggest misconception with a lot of the comments here is that they are assuming that the code generated by copilot is final. Copilot, as the name suggests, is a human-in-the-loop system. It requires a developer to give it prompts and to massage the code it generates. It's not intended to or will it ever feasibly be able to generate code entirely independently. The point is that this massaging is a lot faster than writing everything from 0 in most cases. It doesn't have to be bug free. It doesn't have to be perfect quality. It doesn't have to do "the hard parts" of coding. It just has to be good enough to be sufficiently faster for the developer than starting from 0, and it is.

AllegedAlec 4y ago

> For the purposes of this post, I will not delve into the questions of code quality, security, legal & privacy issues, pricing, and others of similar character that are often brought up in these early days of ML code generation. Let’s just assume all this is sorted out and see what happens next. "Let's assume magic ponies exist, are commonplace and would love to give us all magic rainbow rides"