6 ms·

The place where Copilot shines most for me is boilerplate like: up = foo.north() right = and then it will correctly suggest up = foo.north()

The place where Copilot shines most for me is boilerplate like: up = foo.north() right = and then it will correctly suggest up = foo.north() right = foo.east() down = foo.south() left = foo.west() This requires a kind of linguistic understanding that is beyond basic intellisense. If you are just wanting it for basic intellisense autocomplete, it's not much better than JetBrains stuff. But for things that require linguistic comprehension, it's incredibly useful. For example, if I have a `Point` object with `x` and `y` fields, and I make a function called `lowest_point`, it knows I want to the point with the minimum `y` value. Because it knows the name `y` is most typically used for vertical direction, and the word `lowest` also implies vertical direction. This is kind of a linguistic/semantic knowledge that is not found in just the type system.

onemiketwelve 4y ago
Especially boilerplate in a language or framework you don't usually use. It's such a time saver.
- jlarocco 4y ago
  I've never used Copilot, but are you worried it will only spit out: up = foo.north() right = foo.east() down = foo.south() And you won't notice?
  - antihero 4y ago
    Not really no
    - richbell 4y ago
      Why not? From what I've seen GTP3 works exceptionally well until it doesn't, which can be easily missed.
      - chrisseaton 4y ago
        But I read the code, and I run tests.
        blibble 4y ago
        does the bot write most of the tests too?
        chrisseaton 4y ago
        Do you use it to write anything you want - even works on READMEs and comments! But like any good test - test it in a different way to how to implement it.
      - simonw 4y ago
        My rule using copilot is to use it as a fancy typing assistant which will frequently make mistakes - so you need to read everything it writes for you, and have solid test coverage too in order to avoid accidentally accepting a dumb mistake.
        Gigachad 4y ago
        This made it useless for me because writing code is easier and faster than reviewing someone else’s code. And less error prone.
        throwaway1851 4y ago
        And much more enjoyable. I haven’t tried Copilot yet, but the process of letting it generate something and then checking it for accuracy sounds mind-numbing and stressful. Kind of like supervising a car that’s on assisted cruise: I don’t enjoy that either.
      - antihero 4y ago
        I've simply never had an issue with this sort of thing in months of using co-pilot to be my fancy autocomplete. And if there was an issue I'd notice quickly enough, because I am still the primary pilot. It is also great as a starting point for error messages, comments, etc. I think the key is to still give a shit about the code you're writing.
  - baby 4y ago
    You have to think about it like auto complete. You don't blindly accept what auto complete tells you, your code would never compile otherwise, you read what it proposes to you and you modify what you don't like.
    - jlarocco 4y ago
      I get that, but I meant how often does it introduce bugs in the situation the OP mentioned, using it for "a language or framework you don't usually use", where I won't know if it looks right or not because I'm unfamiliar. Are they going back to fix the code often? Personally, I don't even rely on normal autocomplete very much when I code because I feel like if I don't know what I'm trying to call, then I don't know the language or library well enough, and it's best to go look at the documentation.
      - baby 4y ago
        I'm not sure, I use it A LOT for cryptographic code in Rust and I don't blindly rely on its output. I believe if there's a bug that was introduced it would be my fault with high probability, not Copilot's.
      - lolinder 4y ago
        I recently used Copilot while learning Rust and it was immensely helpful for the learning process. There were a lot of language features that I learned about by seeing Copilot use them. As for mistakes, I've got a good enough sense for reading code in any language to know if something is obviously wrong with an algorithm. The mistakes Copilot makes aren't usually to do with language-specific syntax, they're typically algorithmic and therefore easy to spot in any language. This makes sense if you think about GPT-3's output for English: it tends to be syntactically correct, but often completely wrong about facts. It helps that Rust's compiler is so picky and so helpful, because on the rare occasions when Copilot was completely wrong about the language the compiler could set me straight pretty quickly. Writing automated tests also helped and was made easy by Copilot. I could write a single test case showing how to use the API, and then just write function names to generate the rest of the tests (obviously I would sanity-check the assertions Copilot produced).
  - Gigachad 4y ago
    This is why I stopped using copilot. It was very good at spitting out code that looked correct enough that I didn’t spot the mistakes. And they were mistakes I’d never make myself if I wrote it manually.
    - motoxpro 4y ago
      Maybe don't use it for huge chunks > 10 lines? The point of copilot to me is avg typing speed is 40wpm and read is 250wpm. A 5.25x speedup considering even after I type code I still read it over. There will be errors in copilots code sometimes so even if you said its only 50% accurate thats still a 2.5x speedup on the boilerplate it's used for.
  - throwaway675309 4y ago
    No more worried than in the past if I made the same mistake myself by manually typing it or by copying / pasting and leaving something off. As with everything I always do visual sanity checks.
madeofpalk 4y ago
What's funny is that i often see Copilot give you code that Intellisense/a type system will tell you is obviously wrong. I wish Copilot was more integrated with type systems.
- phire 4y ago
  What's worse, is that if you make a mistake in the code leading up to it's completion (even one picked up by Intellisense), Copilot will now assume that you actually wanted the mistakes, and it's autocompletion will give you even more mistakes.
- neoberg 4y ago
  It produces very good results with typescript. When there is an interface and I'm creating an object that implements the interface; it produces a result that fits into that type.
  - madeofpalk 4y ago
    This has not been my experience with Copilot and Typescript.
    - adamddev1 4y ago
      Same here.
  - baby 4y ago
    Same with Rust. Sometimes I just have to write a comment saying "display implementation for X" and it will autocomplete with the trait implementation. This is extra magical because I don't need to remember the exact detail of the trait.
- dgunay 4y ago
  It is definitely hit or miss, but I don't find it too annoying to just clean up the small mismatches that crop up sometimes.
- gamegoblin 4y ago
croes 4y ago
>This requires a kind of linguistic understanding that is beyond basic intellisense Are you sure? If you would look at the statistics what comes after up = foo.north() right = I bet most of the time it's up = foo.north() right = foo.east() down = foo.south() left = foo.west()
- elcomet 4y ago
  but it contextualizes it with adapted variable names, case and everything. That's very hard to do with only statistical analysis.
  - gamegoblin 4y ago
    Exactly, for example if I give it: hn_example_up = foo.north_garbage_jlakd() hn_example_right = it correctly spits out hn_example_up = foo.north_garbage_jlakd() hn_example_right = foo.east_garbage_jlakd() hn_example_down = foo.south_garbage_jlakd() hn_example_left = foo.west_garbage_jlakd() Even though those identifiers including random letters are certainly not in the training data. It can do surprisingly good basic reasoning. Here is another example. I give it: hn_example_up = foo.x_0_y_1() hn_example_left = foo.x_neg_1_y_0() So my function names are like referring to unit distances on the x/y axes. Copilot correctly figures out what I'm doing and completes: hn_example_up = foo.x_0_y_1() hn_example_left = foo.x_neg_1_y_0() hn_example_down = foo.x_0_y_neg_1() hn_example_right = foo.x_1_y_0() A few weeks ago I coded a rubik's cube solver that involved some very nuanced and weird spatial reasoning and was blown away by Copilot's ability demonstrate it "understood" concepts such as turning the right side down is identical to turning the left side up when viewed from the back of the cube, or stuff like that.
  - sidlls 4y ago
    No it isn’t. Variable names are literally fungible in this context.
- phire 4y ago
  Yes, ultimately it's just a really good statistical model. But it does a really good job, and it's not just matching code it's seen before. You can use any variable name. You can do this in a weird programming language which uses weird symbols for assignment and calls. copilot will probably see though it and give you a good prediction. You can probably even swap west for not_east in code elsewhere in the file, and it will manage to predict that you also want not_east here. And I guaranteed nobody has written code like that before.
baby 4y ago
I've been writing a compiler, and I guess enough people have because it always knows exactly what my enums should be, what the name of my types should be, what I should parse next, etc.
- int_19h 4y ago
  Writing a simple compiler is a fairly common CS exercise.
fferen 4y ago
> For example, if I have a `Point` object with `x` and `y` fields, and I make a function called `lowest_point`, it knows I want to the point with the minimum `y` value. Because it knows the name `y` is most typically used for vertical direction, and the word `lowest` also implies vertical direction. Unless, as in many systems, y=0 is the top and y increases downward.
- gamegoblin 4y ago
  The magical thing is once shown examples of this in your code, or a comment saying as much, Copilot will adapt. For example, I just gave Copilot: # The screen is inverted, so bigger y values are lower def lowest_point(points): and it completed: # The screen is inverted, so bigger y values are lower def lowest_point(points): lowest = points[0] for point in points: if point.y > lowest.y: lowest = point return lowest Even if you write some code that implies that bigger y is lower, Copilot will pick up on it.
  - thethirdone 4y ago
    Notably, I think writing with copilot encourages you to write comments that provide necessary context. Naturally, this makes it easier for other programmers to understand the code.
joshstrange 4y ago
Yep, for the boilerplate alone it's worth the cost for me. Oh you need to create a DTO from an internal object/class? Define the fields, write your `populateFromX()` function signature, maybe you have to type `$this->id = $input->id` and then CoPilot does the rest. And that's only one feature that I love, it does an excellent job most of the time writing the boilerplate for loops and the like based on my function names. Often I have to make few or no changes to it's code. At a worst it provides a great starting point to jump off from, at best it writes it for me just like I would have (matching style/variable naming/etc).
- gamegoblin 4y ago
  One of the biggest boilerplate time savers for me is I have a service that maps HTTP requests to some specific class. I can paste an example HTTP request into a comment like: /* <http request here> The well-type class representation of this is: */ And then it spits out a totally sane and reasonable class. In this file I already have 10+ classes already defined and manually touched up by me to rename some things, remove some fields, etc. So every time I add a new one, Copilot takes into account the previous examples and each addition requires less or no manual tweaking.
  - joshstrange 4y ago
    100%. Most of my calls to the backend literally only need me to type the function name `fetchItems()` and CoPilot correctly uses my axios instance, set the return type to be my base API response wrapper, sets the generic in my base wrapper to be `IItemDTO[]` (it guesses this correctly probably 90%+ of the time), and even shoves the response into a Vuex store just like I would have done (same deal, guesses right about 90%+). It's not like it's a ton of work to do that myself but normally the way I would accomplish it is to copy an existing function, change the return type, change the url, change the Vuex store it puts the data into but with CoPilot it handles that perfectly for me the vast majority of the time. And if I'm doing CRUD endpoint I don't even have to write the other function names, it will autocomplete the "RUD" part after I write the "C" (and by write I mean I put `createItem(payload: ICreateItemDTO)` or similar and it writes the body of the first function as well). For me it's the perfect blend of "generated code" and "human tending" to get exactly what I need without having to do the repetitive/boring parts. It's been a joy to work with.
jollybean 4y ago
Yes, but no. It feels like magic because it really does get some things right. But I find that in most cases, because it's often wrong, you have to deal with wrong suggestions, and be very careful about what you accept. I feel that for most coding scenarios, it doesn't actually, in most cases, do that much but save a few strokes. I suggest that if there is 'high boilerplate' - then maybe there's upside. But for most things, it really doesn't help, despite how in some cases it's really magical. I turned it off and don't miss it. What I would however prefer is merely 'much better autocomplete'. Somewhere between 'ok autocomplete' and 'code suggestions' is the sweet spot for dev. that's not boilerplate.

onemiketwelve 4y ago

Especially boilerplate in a language or framework you don't usually use. It's such a time saver.

jlarocco 4y ago
I've never used Copilot, but are you worried it will only spit out: up = foo.north() right = foo.east() down = foo.south() And you won't notice?
- antihero 4y ago
  Not really no
  - richbell 4y ago
    Why not? From what I've seen GTP3 works exceptionally well until it doesn't, which can be easily missed.
    - chrisseaton 4y ago
      But I read the code, and I run tests.
      - blibble 4y ago
        does the bot write most of the tests too?
        chrisseaton 4y ago
        Do you use it to write anything you want - even works on READMEs and comments! But like any good test - test it in a different way to how to implement it.
    - simonw 4y ago
      My rule using copilot is to use it as a fancy typing assistant which will frequently make mistakes - so you need to read everything it writes for you, and have solid test coverage too in order to avoid accidentally accepting a dumb mistake.
      - Gigachad 4y ago
        This made it useless for me because writing code is easier and faster than reviewing someone else’s code. And less error prone.
        throwaway1851 4y ago
        And much more enjoyable. I haven’t tried Copilot yet, but the process of letting it generate something and then checking it for accuracy sounds mind-numbing and stressful. Kind of like supervising a car that’s on assisted cruise: I don’t enjoy that either.
    - antihero 4y ago
      I've simply never had an issue with this sort of thing in months of using co-pilot to be my fancy autocomplete. And if there was an issue I'd notice quickly enough, because I am still the primary pilot. It is also great as a starting point for error messages, comments, etc. I think the key is to still give a shit about the code you're writing.
- baby 4y ago
  You have to think about it like auto complete. You don't blindly accept what auto complete tells you, your code would never compile otherwise, you read what it proposes to you and you modify what you don't like.
  - jlarocco 4y ago
    I get that, but I meant how often does it introduce bugs in the situation the OP mentioned, using it for "a language or framework you don't usually use", where I won't know if it looks right or not because I'm unfamiliar. Are they going back to fix the code often? Personally, I don't even rely on normal autocomplete very much when I code because I feel like if I don't know what I'm trying to call, then I don't know the language or library well enough, and it's best to go look at the documentation.
    - baby 4y ago
      I'm not sure, I use it A LOT for cryptographic code in Rust and I don't blindly rely on its output. I believe if there's a bug that was introduced it would be my fault with high probability, not Copilot's.
    - lolinder 4y ago
      I recently used Copilot while learning Rust and it was immensely helpful for the learning process. There were a lot of language features that I learned about by seeing Copilot use them. As for mistakes, I've got a good enough sense for reading code in any language to know if something is obviously wrong with an algorithm. The mistakes Copilot makes aren't usually to do with language-specific syntax, they're typically algorithmic and therefore easy to spot in any language. This makes sense if you think about GPT-3's output for English: it tends to be syntactically correct, but often completely wrong about facts. It helps that Rust's compiler is so picky and so helpful, because on the rare occasions when Copilot was completely wrong about the language the compiler could set me straight pretty quickly. Writing automated tests also helped and was made easy by Copilot. I could write a single test case showing how to use the API, and then just write function names to generate the rest of the tests (obviously I would sanity-check the assertions Copilot produced).
- Gigachad 4y ago
  This is why I stopped using copilot. It was very good at spitting out code that looked correct enough that I didn’t spot the mistakes. And they were mistakes I’d never make myself if I wrote it manually.
  - motoxpro 4y ago
    Maybe don't use it for huge chunks > 10 lines? The point of copilot to me is avg typing speed is 40wpm and read is 250wpm. A 5.25x speedup considering even after I type code I still read it over. There will be errors in copilots code sometimes so even if you said its only 50% accurate thats still a 2.5x speedup on the boilerplate it's used for.
- throwaway675309 4y ago
  No more worried than in the past if I made the same mistake myself by manually typing it or by copying / pasting and leaving something off. As with everything I always do visual sanity checks.

madeofpalk 4y ago

What's funny is that i often see Copilot give you code that Intellisense/a type system will tell you is obviously wrong. I wish Copilot was more integrated with type systems.

phire 4y ago
What's worse, is that if you make a mistake in the code leading up to it's completion (even one picked up by Intellisense), Copilot will now assume that you actually wanted the mistakes, and it's autocompletion will give you even more mistakes.
neoberg 4y ago
It produces very good results with typescript. When there is an interface and I'm creating an object that implements the interface; it produces a result that fits into that type.
- madeofpalk 4y ago
  This has not been my experience with Copilot and Typescript.
  - adamddev1 4y ago
    Same here.
- baby 4y ago
  Same with Rust. Sometimes I just have to write a comment saying "display implementation for X" and it will autocomplete with the trait implementation. This is extra magical because I don't need to remember the exact detail of the trait.
dgunay 4y ago
It is definitely hit or miss, but I don't find it too annoying to just clean up the small mismatches that crop up sometimes.
gamegoblin 4y ago

croes 4y ago

>This requires a kind of linguistic understanding that is beyond basic intellisense Are you sure? If you would look at the statistics what comes after up = foo.north() right = I bet most of the time it's up = foo.north() right = foo.east() down = foo.south() left = foo.west()

elcomet 4y ago
but it contextualizes it with adapted variable names, case and everything. That's very hard to do with only statistical analysis.
- gamegoblin 4y ago
  Exactly, for example if I give it: hn_example_up = foo.north_garbage_jlakd() hn_example_right = it correctly spits out hn_example_up = foo.north_garbage_jlakd() hn_example_right = foo.east_garbage_jlakd() hn_example_down = foo.south_garbage_jlakd() hn_example_left = foo.west_garbage_jlakd() Even though those identifiers including random letters are certainly not in the training data. It can do surprisingly good basic reasoning. Here is another example. I give it: hn_example_up = foo.x_0_y_1() hn_example_left = foo.x_neg_1_y_0() So my function names are like referring to unit distances on the x/y axes. Copilot correctly figures out what I'm doing and completes: hn_example_up = foo.x_0_y_1() hn_example_left = foo.x_neg_1_y_0() hn_example_down = foo.x_0_y_neg_1() hn_example_right = foo.x_1_y_0() A few weeks ago I coded a rubik's cube solver that involved some very nuanced and weird spatial reasoning and was blown away by Copilot's ability demonstrate it "understood" concepts such as turning the right side down is identical to turning the left side up when viewed from the back of the cube, or stuff like that.
- sidlls 4y ago
  No it isn’t. Variable names are literally fungible in this context.
phire 4y ago
Yes, ultimately it's just a really good statistical model. But it does a really good job, and it's not just matching code it's seen before. You can use any variable name. You can do this in a weird programming language which uses weird symbols for assignment and calls. copilot will probably see though it and give you a good prediction. You can probably even swap west for not_east in code elsewhere in the file, and it will manage to predict that you also want not_east here. And I guaranteed nobody has written code like that before.

baby 4y ago

I've been writing a compiler, and I guess enough people have because it always knows exactly what my enums should be, what the name of my types should be, what I should parse next, etc.

int_19h 4y ago
Writing a simple compiler is a fairly common CS exercise.

fferen 4y ago

> For example, if I have a `Point` object with `x` and `y` fields, and I make a function called `lowest_point`, it knows I want to the point with the minimum `y` value. Because it knows the name `y` is most typically used for vertical direction, and the word `lowest` also implies vertical direction. Unless, as in many systems, y=0 is the top and y increases downward.

gamegoblin 4y ago
The magical thing is once shown examples of this in your code, or a comment saying as much, Copilot will adapt. For example, I just gave Copilot: # The screen is inverted, so bigger y values are lower def lowest_point(points): and it completed: # The screen is inverted, so bigger y values are lower def lowest_point(points): lowest = points[0] for point in points: if point.y > lowest.y: lowest = point return lowest Even if you write some code that implies that bigger y is lower, Copilot will pick up on it.
- thethirdone 4y ago
  Notably, I think writing with copilot encourages you to write comments that provide necessary context. Naturally, this makes it easier for other programmers to understand the code.

joshstrange 4y ago

Yep, for the boilerplate alone it's worth the cost for me. Oh you need to create a DTO from an internal object/class? Define the fields, write your `populateFromX()` function signature, maybe you have to type `$this->id = $input->id` and then CoPilot does the rest. And that's only one feature that I love, it does an excellent job most of the time writing the boilerplate for loops and the like based on my function names. Often I have to make few or no changes to it's code. At a worst it provides a great starting point to jump off from, at best it writes it for me just like I would have (matching style/variable naming/etc).

gamegoblin 4y ago
One of the biggest boilerplate time savers for me is I have a service that maps HTTP requests to some specific class. I can paste an example HTTP request into a comment like: /* <http request here> The well-type class representation of this is: */ And then it spits out a totally sane and reasonable class. In this file I already have 10+ classes already defined and manually touched up by me to rename some things, remove some fields, etc. So every time I add a new one, Copilot takes into account the previous examples and each addition requires less or no manual tweaking.
- joshstrange 4y ago
  100%. Most of my calls to the backend literally only need me to type the function name `fetchItems()` and CoPilot correctly uses my axios instance, set the return type to be my base API response wrapper, sets the generic in my base wrapper to be `IItemDTO[]` (it guesses this correctly probably 90%+ of the time), and even shoves the response into a Vuex store just like I would have done (same deal, guesses right about 90%+). It's not like it's a ton of work to do that myself but normally the way I would accomplish it is to copy an existing function, change the return type, change the url, change the Vuex store it puts the data into but with CoPilot it handles that perfectly for me the vast majority of the time. And if I'm doing CRUD endpoint I don't even have to write the other function names, it will autocomplete the "RUD" part after I write the "C" (and by write I mean I put `createItem(payload: ICreateItemDTO)` or similar and it writes the body of the first function as well). For me it's the perfect blend of "generated code" and "human tending" to get exactly what I need without having to do the repetitive/boring parts. It's been a joy to work with.

jollybean 4y ago

Yes, but no. It feels like magic because it really does get some things right. But I find that in most cases, because it's often wrong, you have to deal with wrong suggestions, and be very careful about what you accept. I feel that for most coding scenarios, it doesn't actually, in most cases, do that much but save a few strokes. I suggest that if there is 'high boilerplate' - then maybe there's upside. But for most things, it really doesn't help, despite how in some cases it's really magical. I turned it off and don't miss it. What I would however prefer is merely 'much better autocomplete'. Somewhere between 'ok autocomplete' and 'code suggestions' is the sweet spot for dev. that's not boilerplate.