8 ms·

This is utterly damning. I have already instructed my team that Copilot can never be used for our projects. Compromising the product because of unknowable lic

by louthy 5y ago

This is utterly damning. I have already instructed my team that Copilot can never be used for our projects. Compromising the product because of unknowable license demands isn't acceptable in the professional world of software engineering. But if we put the licensing to one side for a moment... 1/ Everything I've seen it generate so far is 'imperative hell'. It is practically a 'boilerplate generator'. That might be useful for pet projects, smaller code bases, or even unit-test writing. But large swathes of application code looking like the examples I've seen so far is hard to manage. 2/ The boilerplate is what bothers me the most (as someone who believes in the declarative approach to software engineering). The future for programming and programming languages should be an attempt to step up to a higher level of abstraction, that has been historically the way we step up to higher levels of productivity. As applications get larger and code-bases grow significantly we need abstraction, not more boilerplate. 3/ As someone who develops a functional framework for C# [1], I could see Copilot essentially side-lining my ideas and my approach to writing code in C#. Not just style, but choice of types, etc. I wonder if the fall out of what is Copilot's 'one true way' of generating code was ever considered? It appears to force a style that is at odds with many who are looking for more robust code. At worst it will homogenise code "people who wrote that, also wrote this" - stifling innovation and iterative improvements in the industry. 4/ Writing code is easy. Reading and understanding code written by another developer is hard. Will we spend most of our time as code-reviewers going forwards? Usually, you can ask the author what their intentions were, or why they think their approach is the correct one. Copilot (as far as I can tell) can't justify its decisions. So, beyond the simple boilerplate generation, will this destroy the art of programming? I can imagine many juniors using this as a crutch, and potentially never understanding the 'why'. I'm not against productivity tools per se; it's certainly a neat trick, and a very impressive feat of engineering in its own right. I am however dubious that this really adds value to professional code-bases, and actively may decrease code quality over time. Then there's the grey area of licensing, which I feel has been totally brushed to one side. [1] https://github.com/louthy/language-ext https://github.com/louthy/language-ext

ethangk 5y ago
This is a little off topic, but your framework looks really interesting! How come you opted for building a functional framework in C#, vs using F#? I couldn’t see anything in the README about what was specifically frustrating about F#? I ask because we’re looking at introducing it at my company.
- louthy 5y ago
  I cofounded a company in 2005, the primary product is a never-ending C# web-application project. As the code-base grew to many millions of lines of code I started to see the very real problems of software engineering in the OO paradigm, and had the functional programming enlightenment moment. We started building some services in F#, but still had a massive amount of C# - and so I wanted the inertia of my team to be in the direction of writing declarative code. There wasn't really anything (outside of LINQ) that did that, so I set about creating something. We don't write F# any more and find functional C# (along with the brilliant C# tooling) to be very effective for us (although we also now use PureScript and Haskell). I do have a stock wiki post on the repo for this though [1]. You might not be surprised to hear it isn't the first time I've been asked this :) [1] https://github.com/louthy/language-ext/wiki/%22Why-don't-you-just-use-F%23%3F%22 https://github.com/louthy/language-ext/wiki/%22Why-don't-you...
  - ethangk 5y ago
    Ha, it's good to see I'm full of original thoughts. That post in the wiki sums it up perfectly, much appreciated!
mimixco 5y ago
Awesome summary and thanks for trying it for the rest of us! Copilot sounded terrible in the press release. The idea that a computer is going to pick the right code for you (from comments, no less) is really just completely nuts. The belief that it could be better than human-picked code is really way off. You bring up a really important point. When you use a tool like Copilot (or copypasta of any kind), you are introducing the additional burden of understanding that other person's code -- which is worse than trying to understand your own code or write something correct from scratch. I think you've hit the nail on the head. Stuff like Copilot makes programming worse and more difficult, not better and easier.
- onion2k 5y ago
  Stuff like Copilot makes programming worse and more difficult, not better and easier. Copilot makes programming worse and more difficult if you're aiming for a specific set of coding values and style that Copilot doesn't generate (yet?). If Copilot generates the sort of code that you would write, and it does for a lot of people, then it's definitely no worse (or better) than copying something from SO. The author of a declarative, functional C# framework likely has very different ideas to what code should be than some PHP developer just trying to do their day-to-day job. We shouldn't abandon tools like Copilot just because they don't work out at the more rigorous ends of the development spectrum.
  - nightpool 5y ago
    The arguments that the GP makes are not based on a specific style or value of coding. Instead, they're based on the simple truth that it is harder to understand code that somebody else wrote. In some cases the benefits of doing so outweigh the costs (such as using a stack overflow answer that's stood the test of time for something you don't know how to do), but with Copilot you don't even get the benefit of upvotes, human intent, or crowdsourced peer review.
  - serf 5y ago
    >If Copilot generates the sort of code that you would write, and it does for a lot of people, then it's definitely no worse (or better) than copying something from SO. Disagree. Most SO copy-paste must be integrated into your project -- maybe it expects different inputs, maybe it expects or works with different variables -- whatever, it must be partially modified to work with the existing code-base that you're working with. Copilot does the integration tasks for you. When one might have had to read through the code from SO to understand it enough to integrate it, the person using Copilot need not even invest that much understanding. Because of these workflow differences, it seems to me as if Copilot enables an even more low-quality workflow than offered by copy-pasting from SO and patching together multiple code-styles and paradigms while hoping for the best; Copilot does that without even the wisdom that an SO user might have that 'this is a bad idea.'
bohemian99 5y ago
My question is would Copilot be useful if you could choose the codebase it would be drawing from? Almost as an internal company tool?
- saurik 5y ago
  If you find yourself copying code someone else in your organization wrote rather than abstracting it to a function in a shared library or building a more declarative framework to manage the problem, something horrible has happened.
  - maccard 5y ago
    Sometimes boilerplate is unavoidable. As an example, how do you send a GET request with libcurl in C with an authorization header? I can't tell you offhand, but I can tell you the file in my codebase that does have it, because I've duplicated the logic for two separate systems.
    - saurik 5y ago
      So you are saying you would rather every project in the world have at least one--if not, thanks to making it easier via Copilot, many--copies of this code rather than one shared library that provides a high-level abstraction for libcurl?... At least for your own code, how did you end up with two copies of duplicated logic rather than a shared library of functionality?
      - maccard 5y ago
        > So you are saying you would rather every project in the world have at least one--if not, thanks to making it easier via Copilot, many--copies of this code. Absolutely not, not at all. I'm suggesting that copying and pasting happens, particularly in the context of a single project. > At least for your own code, how did you end up with two copies of duplicated logic rather than a shared library of functionality? At what point is it worth introducing an abstraction rather than copying? Using my libcurl example, you can create an abstraction over the~ 10 lines of initialization, but if you need to change it to a POST, then you're just implemnenting an abstraction over libcurl, which is just silly.
deleted 5y ago
[deleted]
goodpoint 5y ago
5/ Boilerplate is easy to write but expensive to maintain in large quantities. Proper abstraction/templating requires careful thinking. Copilot encourages the first and discourages the second. 6/ Copilot learns from the past. It can only favor popularity and familiarity in code patterns over correctness and innovation.
reader_mode 5y ago
>2/ The boilerplate is what bothers me the most (as someone who believes in the declarative approach to software engineering). The future for programming and programming languages should be an attempt to step up to a higher level of abstraction, that has been historically the way we step up to higher levels of productivity. As applications get larger and code-bases grow significantly we need abstraction, not more boilerplate. Just the other day someone on copilot threads was arguing that this kind of boilerplate optimizes for readability... It's like Java Stockholm syndrome and the old myth of easy to approach = easy to read (how long it took them to introduce var). I've always viewed code generators as a symptom of language limitations (which is why they were so popular in Java land) that lead to unmaintainable code, this seems like a fancier version of that - with all the same drawbacks.
- ethbr0 5y ago
  Code generators = out of practice, out of mind
- slumdev 5y ago
  Tools like xsd or T4 (in the .NET ecosystem) are great time-savers, but you would never consider directly modifying the code they generate. You would leave the generated code untouched (in case it ever needed to be generated again) and subclass it to make whatever changes you intend. I think Copilot is so unfortunate because it's not building abstractions and expecting you to override parts of them. It's acting as an army of monkeys banging out Shakespeare on a typewriter. And the code it generates is going to require an army to maintain.
  - hu3 5y ago
    Linq2Db is a great example of T4 code generation that works. It creates partial classes from database schema. Together with C# I have strongly typed database access. https://github.com/linq2db/linq2db https://github.com/linq2db/linq2db
  - reader_mode 5y ago
    Even there I feel like code generators are just a band aid around the fact that metaprogramming facilities suck. If you would never modify the generated code why generate in the first place. You could argue that stack traces are easier to follow but TBH generated code is rarely pretty in that regard as well. For example I think F# idea of type providers > code generators.
GlennS 5y ago
I'm inclined to agree with you, and actually I'm rather mistrustful of even basic autocomplete ever since a colleague caught me using it without even looking at the screen! But I wonder... Is this a difference of programmer culture? I think there are people who write successful computer programs for successful businesses without delving into the details. Without considering all the things that might go wrong. Without mapping the code they're writing to concepts. Lots of people. What would they do with this?
- louthy 5y ago
  > What would they do with this? Not get a job working for me ;) More seriously, when I think back to when I was first learning programming - in the heady days of 1985 - I would often copy listings out of computing magazines, make a mistake whilst doing it, and then have no idea what was wrong. The only way was to check character by character. I didn't have the deeper understanding yet, and so I couldn't contribute to solving the problem in any real way. If they're at that level as a programmer, to the point where their code is being written for them and they don't really understand it, then they're going to make some serious mistakes eventually. If you want to step up as a dev, understanding is key. Programming is hard and gets harder as you step up and bite off bigger and more complex problems. If you're relying on the tools to write your code, then your job is one step away from being automated. That should be enough to light a fire under your ambition!
  - biztos 5y ago
    I also typed stuff in from magazines in the 80’s, and my fast but imperfect typing really helped me learn programming: I often had to stop, go back to the first page, and actually read the damned thing in order to make it work.
bostonsre 5y ago
I'm not sure we should throw the baby out with the bath water here due to the large blurbs it stubs in when when it doesn't have a lot to go on in mostly empty files. It is a preview release. They are working on proper attribution of suggested code and explainability [1]. Having a stochastic parrot that types faster than I do would be useful in a lot of cases. Yes, better layers of abstraction could make us more productive in the future, but we're not there yet. By all means, don't accept the larger blurbs it proposes, but there is productivity to be gained in the smaller suggestions. If it correctly intuits the exact rest of the line that you were thinking of, it will save time and not make you lose understanding of the program. In some areas complete understanding and complete code ownership is required but in a lot of places, it's not. If it produces the work of a moderately skilled developer it would be sufficient. I don't remember all code I write as time passes. If it produces work that I would have produced, then I don't see how that's any different that work that was produced by my past self. It may feel offensive but a lot of the comments against it sound like rage against the machine/industrialization opponents and the arguments sound pretty similar to those made in the past by those that had their jobs automated away. I'm not sure we're all as unique snowflakes as we like to think we are. Sure, there will be some code that requires an absolute master that is outside the capabilities of this tool. But I'd guess there is a massive amount of code that doesn't need that mastery. [1] https://docs.github.com/en/github/copilot/research-recitation https://docs.github.com/en/github/copilot/research-recitatio...
- vlovich123 5y ago
  I think it depends on how you look at it. For small snippets that have likely been already written by someone else, this probably works great. For those though, the time savings is probably at most 5-10 min down to 1 or less. The challenge is that that’s not where my time goes unless I’m working in an unfamiliar language. As someone who writes a lot of code quickly, I’m usually bottlenecked by reviews. For more complex changes I’m bottlenecked by understanding the problem and experimenting with solutions (and then reviews, domain-specific tests usually, fixing bugs etc). Writing code isn’t like waiting for code to compile since I’m not actually ending up task switching that frequently. This does sound like a fantastic tool when I’m not familiar with the language although I wonder if it actually generates useful stuff that integrates well as the code gets larger (eg can I say “write an async function that allocates a record handle in the file with ownership that doesn’t outlive the open file’s lifetime”). I’m sure though that this is what a lot of people are overindexing on. For things like that I expect normal evolution of the product will work well. For things like “cool, understand your snippets but also weight my own codebase higher and understand the specifics of my codebase”, I think there’s a lot of groundbreaking research that would be required. That is what I see as a true productivity boost - I’d make this 100% required for anyone joining the codebase. The more mentorship can be offloaded, the lower the cost is to growing teams. OSS projects can more easily scale similarly.
hallqv 5y ago
Neural net: ”It’s all in the training data, stupid.”
mdellavo 5y ago
Generating code has never been a problem for developers :) I'd be more interested in a tool that notices patterns and boilerplate. It could offer a chance for generalization, abstraction or use of a common pattern from the codebase. This is of course much harder.
0xbadcafebee 5y ago
> programming languages should be an attempt to step up to a higher level of abstraction Adding abstraction buries complexity. If all you do is keep adding more abstractions, you end up with an overcomplicated, inefficient mess. Which is part of why application sizes are so bloated today. People just keep adding layers, as long as they have room for more of them. Everything gets less efficient and definitely not better. The right way to design better is to iterate on a core design until it cannot be any simpler. All of the essential complexity of software systems today comes from 40 year old conventions. We need a redesign, not more layers. One example is version management. Most applications today can implement versioned functions and keep multiple versions in an application, and track dependencies between external applications. Make a simple DAG of the versions and let apps call the versions they were designed against, or express what versions are compatible with what, internally. This would make applications infinitely backwards-compatible. The functionality exists right now in GNU Libc. You can literally do it today. But rather than do that, we stumble around replacing entire environments of specific versions of applications and dependencies, because we can't seem to move the entire industry forward to new ideas. Redesign is hard, adding layers is easy.
- louthy 5y ago
  > Adding abstraction buries complexity. If all you do is keep adding more abstractions, you end up with an overcomplicated, inefficient mess. Which is part of why application sizes are so bloated today. People just keep adding layers, as long as they have room for more of them. Everything gets less efficient and definitely not better. Presumably you're writing code in binary then? This is a non-argument, because there's evidence that it's worked. Computers were first programmed with switches and punch cards, then tape, then assembly, then low level languages like C, then memory managed languages etc. Abstraction works when side-effects are controlled. Composition is what we're after, but we must compose the bigger bits from smaller bits that don't have surprises in. This works well in functional programming, a good example would be monadic composition: monads remove the boilerplate of dealing with asynchrony, value availability, list iteration, state management, environment management, etc. Languages that have first-class support for these tend to have significantly less boilerplate. The efficiency argument is also off too. Most software engineering teams would trade some efficiency for more reliable and bug free code. At some point (and I would argue we're way past it) programs become too complex for the human brain to comprehend, and that's where bugs come from. That's why we're overdue an abstraction lift. Tools like Copilot almost tacitly agree, because they're trying to provide a way of turning the abstract into the real, but then all you see is the real, not the abstract. Continuing the assault on our weak and feeble grey matter. I spent the early part of my career obsessing over performance on crippled architectures (Playstation 3D engine programmer). If I continued to write applications now like I did then, nothing would go out the door and my company wouldn't exist. Of course there are times when performance matters. But the vast majority of code needs to be correct first, not the most optimal it can be for the architecture.
code4you 5y ago
Great points. Really makes me question why so many developers were excited / worried about programming jobs being automated away by this technology. I really doubt that many jobs are going to be displaced by what is at best an improvement to autocomplete/intellisense and at worst an unreliable, copyright infringing boilerplate generator. Also agree with point #3 - I could see Copilot steering devs away from new code patterns toward whatever was most commonly seen in the existing codebases it was trained on. Doesn't seem good for innovation in that sense.

by louthy 5y ago

ethangk 5y ago

This is a little off topic, but your framework looks really interesting! How come you opted for building a functional framework in C#, vs using F#? I couldn’t see anything in the README about what was specifically frustrating about F#? I ask because we’re looking at introducing it at my company.

louthy 5y ago
I cofounded a company in 2005, the primary product is a never-ending C# web-application project. As the code-base grew to many millions of lines of code I started to see the very real problems of software engineering in the OO paradigm, and had the functional programming enlightenment moment. We started building some services in F#, but still had a massive amount of C# - and so I wanted the inertia of my team to be in the direction of writing declarative code. There wasn't really anything (outside of LINQ) that did that, so I set about creating something. We don't write F# any more and find functional C# (along with the brilliant C# tooling) to be very effective for us (although we also now use PureScript and Haskell). I do have a stock wiki post on the repo for this though [1]. You might not be surprised to hear it isn't the first time I've been asked this :) [1] https://github.com/louthy/language-ext/wiki/%22Why-don't-you-just-use-F%23%3F%22 https://github.com/louthy/language-ext/wiki/%22Why-don't-you...
- ethangk 5y ago
  Ha, it's good to see I'm full of original thoughts. That post in the wiki sums it up perfectly, much appreciated!

mimixco 5y ago

Awesome summary and thanks for trying it for the rest of us! Copilot sounded terrible in the press release. The idea that a computer is going to pick the right code for you (from comments, no less) is really just completely nuts. The belief that it could be better than human-picked code is really way off. You bring up a really important point. When you use a tool like Copilot (or copypasta of any kind), you are introducing the additional burden of understanding that other person's code -- which is worse than trying to understand your own code or write something correct from scratch. I think you've hit the nail on the head. Stuff like Copilot makes programming worse and more difficult, not better and easier.

onion2k 5y ago
Stuff like Copilot makes programming worse and more difficult, not better and easier. Copilot makes programming worse and more difficult if you're aiming for a specific set of coding values and style that Copilot doesn't generate (yet?). If Copilot generates the sort of code that you would write, and it does for a lot of people, then it's definitely no worse (or better) than copying something from SO. The author of a declarative, functional C# framework likely has very different ideas to what code should be than some PHP developer just trying to do their day-to-day job. We shouldn't abandon tools like Copilot just because they don't work out at the more rigorous ends of the development spectrum.
- nightpool 5y ago
  The arguments that the GP makes are not based on a specific style or value of coding. Instead, they're based on the simple truth that it is harder to understand code that somebody else wrote. In some cases the benefits of doing so outweigh the costs (such as using a stack overflow answer that's stood the test of time for something you don't know how to do), but with Copilot you don't even get the benefit of upvotes, human intent, or crowdsourced peer review.
- serf 5y ago
  >If Copilot generates the sort of code that you would write, and it does for a lot of people, then it's definitely no worse (or better) than copying something from SO. Disagree. Most SO copy-paste must be integrated into your project -- maybe it expects different inputs, maybe it expects or works with different variables -- whatever, it must be partially modified to work with the existing code-base that you're working with. Copilot does the integration tasks for you. When one might have had to read through the code from SO to understand it enough to integrate it, the person using Copilot need not even invest that much understanding. Because of these workflow differences, it seems to me as if Copilot enables an even more low-quality workflow than offered by copy-pasting from SO and patching together multiple code-styles and paradigms while hoping for the best; Copilot does that without even the wisdom that an SO user might have that 'this is a bad idea.'

bohemian99 5y ago

My question is would Copilot be useful if you could choose the codebase it would be drawing from? Almost as an internal company tool?

saurik 5y ago
If you find yourself copying code someone else in your organization wrote rather than abstracting it to a function in a shared library or building a more declarative framework to manage the problem, something horrible has happened.
- maccard 5y ago
  Sometimes boilerplate is unavoidable. As an example, how do you send a GET request with libcurl in C with an authorization header? I can't tell you offhand, but I can tell you the file in my codebase that does have it, because I've duplicated the logic for two separate systems.
  - saurik 5y ago
    So you are saying you would rather every project in the world have at least one--if not, thanks to making it easier via Copilot, many--copies of this code rather than one shared library that provides a high-level abstraction for libcurl?... At least for your own code, how did you end up with two copies of duplicated logic rather than a shared library of functionality?
    - maccard 5y ago
      > So you are saying you would rather every project in the world have at least one--if not, thanks to making it easier via Copilot, many--copies of this code. Absolutely not, not at all. I'm suggesting that copying and pasting happens, particularly in the context of a single project. > At least for your own code, how did you end up with two copies of duplicated logic rather than a shared library of functionality? At what point is it worth introducing an abstraction rather than copying? Using my libcurl example, you can create an abstraction over the~ 10 lines of initialization, but if you need to change it to a POST, then you're just implemnenting an abstraction over libcurl, which is just silly.

deleted 5y ago

[deleted]

goodpoint 5y ago

5/ Boilerplate is easy to write but expensive to maintain in large quantities. Proper abstraction/templating requires careful thinking. Copilot encourages the first and discourages the second. 6/ Copilot learns from the past. It can only favor popularity and familiarity in code patterns over correctness and innovation.

reader_mode 5y ago

>2/ The boilerplate is what bothers me the most (as someone who believes in the declarative approach to software engineering). The future for programming and programming languages should be an attempt to step up to a higher level of abstraction, that has been historically the way we step up to higher levels of productivity. As applications get larger and code-bases grow significantly we need abstraction, not more boilerplate. Just the other day someone on copilot threads was arguing that this kind of boilerplate optimizes for readability... It's like Java Stockholm syndrome and the old myth of easy to approach = easy to read (how long it took them to introduce var). I've always viewed code generators as a symptom of language limitations (which is why they were so popular in Java land) that lead to unmaintainable code, this seems like a fancier version of that - with all the same drawbacks.

ethbr0 5y ago
Code generators = out of practice, out of mind
slumdev 5y ago
Tools like xsd or T4 (in the .NET ecosystem) are great time-savers, but you would never consider directly modifying the code they generate. You would leave the generated code untouched (in case it ever needed to be generated again) and subclass it to make whatever changes you intend. I think Copilot is so unfortunate because it's not building abstractions and expecting you to override parts of them. It's acting as an army of monkeys banging out Shakespeare on a typewriter. And the code it generates is going to require an army to maintain.
- hu3 5y ago
  Linq2Db is a great example of T4 code generation that works. It creates partial classes from database schema. Together with C# I have strongly typed database access. https://github.com/linq2db/linq2db https://github.com/linq2db/linq2db
- reader_mode 5y ago
  Even there I feel like code generators are just a band aid around the fact that metaprogramming facilities suck. If you would never modify the generated code why generate in the first place. You could argue that stack traces are easier to follow but TBH generated code is rarely pretty in that regard as well. For example I think F# idea of type providers > code generators.

GlennS 5y ago

I'm inclined to agree with you, and actually I'm rather mistrustful of even basic autocomplete ever since a colleague caught me using it without even looking at the screen! But I wonder... Is this a difference of programmer culture? I think there are people who write successful computer programs for successful businesses without delving into the details. Without considering all the things that might go wrong. Without mapping the code they're writing to concepts. Lots of people. What would they do with this?

louthy 5y ago
> What would they do with this? Not get a job working for me ;) More seriously, when I think back to when I was first learning programming - in the heady days of 1985 - I would often copy listings out of computing magazines, make a mistake whilst doing it, and then have no idea what was wrong. The only way was to check character by character. I didn't have the deeper understanding yet, and so I couldn't contribute to solving the problem in any real way. If they're at that level as a programmer, to the point where their code is being written for them and they don't really understand it, then they're going to make some serious mistakes eventually. If you want to step up as a dev, understanding is key. Programming is hard and gets harder as you step up and bite off bigger and more complex problems. If you're relying on the tools to write your code, then your job is one step away from being automated. That should be enough to light a fire under your ambition!
- biztos 5y ago
  I also typed stuff in from magazines in the 80’s, and my fast but imperfect typing really helped me learn programming: I often had to stop, go back to the first page, and actually read the damned thing in order to make it work.

bostonsre 5y ago

I'm not sure we should throw the baby out with the bath water here due to the large blurbs it stubs in when when it doesn't have a lot to go on in mostly empty files. It is a preview release. They are working on proper attribution of suggested code and explainability [1]. Having a stochastic parrot that types faster than I do would be useful in a lot of cases. Yes, better layers of abstraction could make us more productive in the future, but we're not there yet. By all means, don't accept the larger blurbs it proposes, but there is productivity to be gained in the smaller suggestions. If it correctly intuits the exact rest of the line that you were thinking of, it will save time and not make you lose understanding of the program. In some areas complete understanding and complete code ownership is required but in a lot of places, it's not. If it produces the work of a moderately skilled developer it would be sufficient. I don't remember all code I write as time passes. If it produces work that I would have produced, then I don't see how that's any different that work that was produced by my past self. It may feel offensive but a lot of the comments against it sound like rage against the machine/industrialization opponents and the arguments sound pretty similar to those made in the past by those that had their jobs automated away. I'm not sure we're all as unique snowflakes as we like to think we are. Sure, there will be some code that requires an absolute master that is outside the capabilities of this tool. But I'd guess there is a massive amount of code that doesn't need that mastery. [1] https://docs.github.com/en/github/copilot/research-recitation https://docs.github.com/en/github/copilot/research-recitatio...

vlovich123 5y ago
I think it depends on how you look at it. For small snippets that have likely been already written by someone else, this probably works great. For those though, the time savings is probably at most 5-10 min down to 1 or less. The challenge is that that’s not where my time goes unless I’m working in an unfamiliar language. As someone who writes a lot of code quickly, I’m usually bottlenecked by reviews. For more complex changes I’m bottlenecked by understanding the problem and experimenting with solutions (and then reviews, domain-specific tests usually, fixing bugs etc). Writing code isn’t like waiting for code to compile since I’m not actually ending up task switching that frequently. This does sound like a fantastic tool when I’m not familiar with the language although I wonder if it actually generates useful stuff that integrates well as the code gets larger (eg can I say “write an async function that allocates a record handle in the file with ownership that doesn’t outlive the open file’s lifetime”). I’m sure though that this is what a lot of people are overindexing on. For things like that I expect normal evolution of the product will work well. For things like “cool, understand your snippets but also weight my own codebase higher and understand the specifics of my codebase”, I think there’s a lot of groundbreaking research that would be required. That is what I see as a true productivity boost - I’d make this 100% required for anyone joining the codebase. The more mentorship can be offloaded, the lower the cost is to growing teams. OSS projects can more easily scale similarly.

hallqv 5y ago

Neural net: ”It’s all in the training data, stupid.”

mdellavo 5y ago

Generating code has never been a problem for developers :) I'd be more interested in a tool that notices patterns and boilerplate. It could offer a chance for generalization, abstraction or use of a common pattern from the codebase. This is of course much harder.

0xbadcafebee 5y ago

> programming languages should be an attempt to step up to a higher level of abstraction Adding abstraction buries complexity. If all you do is keep adding more abstractions, you end up with an overcomplicated, inefficient mess. Which is part of why application sizes are so bloated today. People just keep adding layers, as long as they have room for more of them. Everything gets less efficient and definitely not better. The right way to design better is to iterate on a core design until it cannot be any simpler. All of the essential complexity of software systems today comes from 40 year old conventions. We need a redesign, not more layers. One example is version management. Most applications today can implement versioned functions and keep multiple versions in an application, and track dependencies between external applications. Make a simple DAG of the versions and let apps call the versions they were designed against, or express what versions are compatible with what, internally. This would make applications infinitely backwards-compatible. The functionality exists right now in GNU Libc. You can literally do it today. But rather than do that, we stumble around replacing entire environments of specific versions of applications and dependencies, because we can't seem to move the entire industry forward to new ideas. Redesign is hard, adding layers is easy.

louthy 5y ago
> Adding abstraction buries complexity. If all you do is keep adding more abstractions, you end up with an overcomplicated, inefficient mess. Which is part of why application sizes are so bloated today. People just keep adding layers, as long as they have room for more of them. Everything gets less efficient and definitely not better. Presumably you're writing code in binary then? This is a non-argument, because there's evidence that it's worked. Computers were first programmed with switches and punch cards, then tape, then assembly, then low level languages like C, then memory managed languages etc. Abstraction works when side-effects are controlled. Composition is what we're after, but we must compose the bigger bits from smaller bits that don't have surprises in. This works well in functional programming, a good example would be monadic composition: monads remove the boilerplate of dealing with asynchrony, value availability, list iteration, state management, environment management, etc. Languages that have first-class support for these tend to have significantly less boilerplate. The efficiency argument is also off too. Most software engineering teams would trade some efficiency for more reliable and bug free code. At some point (and I would argue we're way past it) programs become too complex for the human brain to comprehend, and that's where bugs come from. That's why we're overdue an abstraction lift. Tools like Copilot almost tacitly agree, because they're trying to provide a way of turning the abstract into the real, but then all you see is the real, not the abstract. Continuing the assault on our weak and feeble grey matter. I spent the early part of my career obsessing over performance on crippled architectures (Playstation 3D engine programmer). If I continued to write applications now like I did then, nothing would go out the door and my company wouldn't exist. Of course there are times when performance matters. But the vast majority of code needs to be correct first, not the most optimal it can be for the architecture.

code4you 5y ago

Great points. Really makes me question why so many developers were excited / worried about programming jobs being automated away by this technology. I really doubt that many jobs are going to be displaced by what is at best an improvement to autocomplete/intellisense and at worst an unreliable, copyright infringing boilerplate generator. Also agree with point #3 - I could see Copilot steering devs away from new code patterns toward whatever was most commonly seen in the existing codebases it was trained on. Doesn't seem good for innovation in that sense.