7 ms·

To me, this is a great illustration of why chat is a terrible interface for a coding tool. I've gone down this path as well, learning that you need to have a de

by cwp 3y ago

To me, this is a great illustration of why chat is a terrible interface for a coding tool. I've gone down this path as well, learning that you need to have a detailed prompt that establishes a lot of context, and iteratively improve it to generate better code. And yup, generating a task list and working from that is definitely a key strategy for getting GPT to do anything bigger than a few paragraphs. But compare that to Copilot: Copilot doesn't help much when you're starting from scratch, and there's nothing for it to work with. But once you have a bit of structure, it starts to make recommendations. Rather than generating large chunks of code, the recommendations are small, chunks of a few lines or maybe even one line at a time. And it's sooooo good at picking up on patterns. As soon as you start something with built-in symmetries, it'll quickly generate all the permutations. It's sort of prompting by pointing. This is so. much. better. than writing prompt for the chat interface. I'm really excited to see where these kinds of tools lead.

barbariangrunge 3y ago
Either way, you’re sending your companys biggest asset to another company, aren’t you? I’ll try these tools when they start being able to run locally
- koonsolo 3y ago
  I surely hope they use my copyrighted code and make millions out of it. Ideal case for me to sue them for lots of money.
  - freedomben 3y ago
    How would you ever know? It will come in chunks of a dozen or less lines at a time and it will be written into your competitor's proprietary codebase (that you don't have access to).
    - survirtual 3y ago
      Right. If you are building something truly valuable locally, and it is innovative or otherwise disruptive and relies on being a first mover, centrally hosted LLMs are a non-starter. Most software corps have countless millions of lines of code. You'd be spending lifetimes tracing where someone ripped your "copyrighted" techniques and methods. The complete lack of security awareness and willingness to compromise privacy for convenience in people deeply saddens me.
      - koonsolo 3y ago
        > willingness to compromise privacy for convenience I have to ask: do you carry a cellphone?
        survirtual 3y ago
        This is not really a valid comparison. The cellphone is not a compromise for convenience. It allows me to make a living, providing internet connectivity and lets me keep in contact with friends and family. Without it, my freedoms would be drastically diminished. With software we develop with, we have choices. We can use OSS. We can try to use open hardware. If we are working on sensitive things, we can use an airgapped system with vim. When you practice these kinds of routines, they are not a burden. Actually, using vim instead of something like vscode increases productivity eventually. It does take a little bit of time. When we couple our productivity with centrally hosted services, we greatly diminish our freedom to be productive on a wide range of problem areas. I don't say this to brag, it is to maximize all of our freedom. In my view, most of us SHOULD be working on "sensitive" things. There is so, so much work to be done for the cause of freedom and liberty in software. We need to reserve that capability in us, we cannot let nameless people have an inside access to our expression.
        koonsolo 3y ago
        A cellphone literally tracks your every move. If that's not a privacy concern then I don't know what is. Maybe a device with a microphone that's constantly on you. Oh no wait, that's also a cellphone. I was born in the 70's, and I can tell you, you can survive just fine without a cellphone. All of what you describe can be done on a desktop. But hey, if you want to compromise your privacy for some convenience, that's your choice.
    - HALtheWise 3y ago
      > GitHub Copilot [for business] transmits snippets of your code from your IDE to GitHub to provide Suggestions to you. Code snippets data is only transmitted in real-time to return Suggestions, and is discarded once a Suggestion is returned. Copilot for Business does not retain any Code Snippets Data. Likely, some employee would whistleblow that they're not complying with their privacy policy, and either government litigation or a class action lawsuit would ensue. That legal process would involve subpoenas and third-party auditors being granted access to GitHub/Microsoft's internal code and communications history, which makes it pretty hard to hide something as big as collecting, storing, and then training from a huge amount of uploaded code snippets they promised not to. It's not inconceivable that they're noncompliant, but my bet would be that if they are collecting data they explicitly promise not to it's an accidental or malicious action by an individual employee, and they will freak out when they discover it and delete everything as soon as they can. If they intended to collect that data, it would be much easier to write that into the policy than deal with all the risk. Notably, this applies to Copilot for Business, which is presumably what you're using if you are at work.
      - zelphirkalt 3y ago
        Couldn't it happen more subtly, without having the code lying around for long? The model could be doing online-learning (ML term) and only then they discard code that they get send. This means your code could appear in other people's completions/suggestions, without it having to lie anywhere. It is basically learned into the model. The code could appear almost or even completely verbatim on someone else's machine, possibly working for a competitor. Even that it is your code would not be obvious, because MS could claim, that Copilot merely accidentally constructed the same code from other learned code. Not sure that this is how the model works, but it is conceivable.
    - koonsolo 3y ago
      I was talking about someone stealing my codebase. Talking about integrating the code into the LLM, others get the same benefit that you are getting, so I don't really see the issue. So you can either develop everything on your own, or you can leverage LLM's, helping both yourself and others.
- SanderNL 3y ago
  I sort of disagree that code is the biggest asset. Take the Yandex leak. What can you do with it? Outcompete them?
  - visarga 3y ago
    > Take the Yandex leak. What can you do with it? Obviously, add it to the big training set of the next code model.
- throwaway202303 3y ago
  No or no company would be able to use it. As you type fragments of code are sent and discarded after use. You need to trust Microsoft to actually do the discarding but contractually they do and you can sue them if they accidentally or deliberately keep your code around or otherwise mismanage it.
  - xiphias2 3y ago
    They are obligated to give data to the government, and government took part of spying in Brazil for Boeing in the past, but I guess they are using this capability only for a few strategic companies, and most companies are not that.
    - matheusmoreira 3y ago
      > government took part of spying in Brazil for Boeing in the past Do you have more details? Please elaborate.
  - zelphirkalt 3y ago
    But that is naive, isn't it? Who has the money and time in their life, to actually sue MS? Even if "you" is a business, few will have the resources for that.
    - justinhj 3y ago
      Individuals do not (although a class action would be feasible), but large companies that use Github and other Microsoft products, of course they have both the means to sue Microsoft and the motivation should their business be impacted.
      - throwaway202303 3y ago
        Exactly
mjr00 3y ago
Absolutely. People will quickly realize that for coding, the natural language part of LLMs is a distraction. Copilot is much better for someone actually writing code, but unfortunately doesn't get as emphasized due to the narrative surrounding LLMs right now.
- yodsanklai 3y ago
  > Copilot is much better for someone actually writing code I haven't used copilot yet, but I'm using occasionally chatgpt with prompts such as "write a bash/python script take takes these parameters and perform this tasks". Then I iterate if needed, and usually, i can get what i want faster than without using chatgpt. It's not a game changer, but it's a performance boost. How natural language is a distraction here? and how copilot would do much better for the same task?
  - mjr00 3y ago
    Try not using natural language and just type what you'd type into Google. You'll get the same results and realize that all of the natural language fluff is totally unnecessary. I just typed in "bash script recursive chmod 777 all files" (as a dumb toy example) and got a resulting script back. It was surrounded by two natural language GPT comments: > It's generally not recommended to give all files and directories the 777 permission as it can pose a security risk. However, if you still want to proceed with this, here's a bash script that recursively changes the permission of all files and directories to 777: [...] Make sure to replace "/path/to/target/directory" with the path of the directory you want to modify. To run the script, save it as a file (e.g., "chmod_all.sh"), make it executable with the command "chmod +x chmod_all.sh", and then run it with "./chmod_all.sh". It's up to the reader to decide if those are necessary, but I'd lean towards no.
    - vorticalbox 3y ago
      you can always include the instruction to only return the code and no other text
maroonblazer 3y ago
As a hobbyist developer with no formal training, I wish Copilot had a 'teaching' or "Senior Dev" mode, where I can play the role of the Junior Dev. I'd like it to pick up on what I'm trying to write, and then prompt me with questions or hints, but not straight up give me the code. Or, if that's too Clippy-like annoying, let me prompt it when I'm stuck, and only then suggest hints or ask suggestive questions that guide me to a solution. I agree, very exciting to see where all this goes.
- cwp 3y ago
  One thing you might try with Copilot is to ask it to explain the code. It can often give insight, even on code that you yourself wrote a few minutes ago.
- ukuina 3y ago
  The Github Copilot Labs extension has "codebrushes" that can transform and explain existing code instead of generating new code, but none of it only gives "hints". Maybe one of the codebrushes can take a custom prompt.
  - SparkyMcUnicorn 3y ago
    You can create custom brushes, or open the "CoPilot Labs" panel and "explain" with a custom prompt.
SamPatt 3y ago
I've noticed that after using copilot on a code base for a while, you can effectively prompt the AI just by creating a descriptive comment. // This function ends the call by sending a disconnection message to all connected peers Bam, copilot will recommend at least the first line, with subsequent lines usually being pretty good, and more and more frequently, it will recommend the whole function. I still use GPT-4 a lot, especially for troubleshooting errors, but I'm always pleasantly surprised at how good copilot can be.
- armchairhacker 3y ago
  Copilot is a game-changer and very underrated IMO. GPT4 is smart but not really used in production yet. Copilot is reportedly generating 50% of new code and I can't imagine going without it.
  - smashface 3y ago
    Where do you get that 50% number? Do you mean 50% of all new code in the industry? That seems beyond extremely unlikely.
    - fnordpiglet 3y ago
      (I read it as 50% of their code)
    - deleted 3y ago
      [deleted]
    - moyix 3y ago
      The number is 40%, and it's 40% of code written by Copilot users. It's also just for Python: > In files where it’s enabled, nearly 40% of code is being written by GitHub Copilot in popular coding languages, like Python—and we expect that to increase. https://github.blog/2022-06-21-github-copilot-is-generally-available-to-all-developers/ https://github.blog/2022-06-21-github-copilot-is-generally-a...
      - 3y ago
throwaway202303 3y ago
People have different preferences and habits. Having tried both models I much prefer having a conversation in one window and constructing my code from that in another. Although copilot is about to add some interesting features that may win me back.
supernikio2 3y ago
Exactly this. I've tried to implement ChatGPT into may daily workflow, but you have to give it an excruciating level of detail to get something that remotely resembles real code I'd use, and even then you have to hold its hand to guide it in the correct direction, and still have to make some manual final touches at the end. This is why I'm looking forward to Copilot X so much. It will hold much more context than the current implementation, and integrate the Chat interface that's so natural to us.

by cwp 3y ago

barbariangrunge 3y ago

Either way, you’re sending your companys biggest asset to another company, aren’t you? I’ll try these tools when they start being able to run locally

koonsolo 3y ago
I surely hope they use my copyrighted code and make millions out of it. Ideal case for me to sue them for lots of money.
- freedomben 3y ago
  How would you ever know? It will come in chunks of a dozen or less lines at a time and it will be written into your competitor's proprietary codebase (that you don't have access to).
  - survirtual 3y ago
    Right. If you are building something truly valuable locally, and it is innovative or otherwise disruptive and relies on being a first mover, centrally hosted LLMs are a non-starter. Most software corps have countless millions of lines of code. You'd be spending lifetimes tracing where someone ripped your "copyrighted" techniques and methods. The complete lack of security awareness and willingness to compromise privacy for convenience in people deeply saddens me.
    - koonsolo 3y ago
      > willingness to compromise privacy for convenience I have to ask: do you carry a cellphone?
      - survirtual 3y ago
        This is not really a valid comparison. The cellphone is not a compromise for convenience. It allows me to make a living, providing internet connectivity and lets me keep in contact with friends and family. Without it, my freedoms would be drastically diminished. With software we develop with, we have choices. We can use OSS. We can try to use open hardware. If we are working on sensitive things, we can use an airgapped system with vim. When you practice these kinds of routines, they are not a burden. Actually, using vim instead of something like vscode increases productivity eventually. It does take a little bit of time. When we couple our productivity with centrally hosted services, we greatly diminish our freedom to be productive on a wide range of problem areas. I don't say this to brag, it is to maximize all of our freedom. In my view, most of us SHOULD be working on "sensitive" things. There is so, so much work to be done for the cause of freedom and liberty in software. We need to reserve that capability in us, we cannot let nameless people have an inside access to our expression.
        koonsolo 3y ago
        A cellphone literally tracks your every move. If that's not a privacy concern then I don't know what is. Maybe a device with a microphone that's constantly on you. Oh no wait, that's also a cellphone. I was born in the 70's, and I can tell you, you can survive just fine without a cellphone. All of what you describe can be done on a desktop. But hey, if you want to compromise your privacy for some convenience, that's your choice.
  - HALtheWise 3y ago
    > GitHub Copilot [for business] transmits snippets of your code from your IDE to GitHub to provide Suggestions to you. Code snippets data is only transmitted in real-time to return Suggestions, and is discarded once a Suggestion is returned. Copilot for Business does not retain any Code Snippets Data. Likely, some employee would whistleblow that they're not complying with their privacy policy, and either government litigation or a class action lawsuit would ensue. That legal process would involve subpoenas and third-party auditors being granted access to GitHub/Microsoft's internal code and communications history, which makes it pretty hard to hide something as big as collecting, storing, and then training from a huge amount of uploaded code snippets they promised not to. It's not inconceivable that they're noncompliant, but my bet would be that if they are collecting data they explicitly promise not to it's an accidental or malicious action by an individual employee, and they will freak out when they discover it and delete everything as soon as they can. If they intended to collect that data, it would be much easier to write that into the policy than deal with all the risk. Notably, this applies to Copilot for Business, which is presumably what you're using if you are at work.
    - zelphirkalt 3y ago
      Couldn't it happen more subtly, without having the code lying around for long? The model could be doing online-learning (ML term) and only then they discard code that they get send. This means your code could appear in other people's completions/suggestions, without it having to lie anywhere. It is basically learned into the model. The code could appear almost or even completely verbatim on someone else's machine, possibly working for a competitor. Even that it is your code would not be obvious, because MS could claim, that Copilot merely accidentally constructed the same code from other learned code. Not sure that this is how the model works, but it is conceivable.
  - koonsolo 3y ago
    I was talking about someone stealing my codebase. Talking about integrating the code into the LLM, others get the same benefit that you are getting, so I don't really see the issue. So you can either develop everything on your own, or you can leverage LLM's, helping both yourself and others.
SanderNL 3y ago
I sort of disagree that code is the biggest asset. Take the Yandex leak. What can you do with it? Outcompete them?
- visarga 3y ago
  > Take the Yandex leak. What can you do with it? Obviously, add it to the big training set of the next code model.
throwaway202303 3y ago
No or no company would be able to use it. As you type fragments of code are sent and discarded after use. You need to trust Microsoft to actually do the discarding but contractually they do and you can sue them if they accidentally or deliberately keep your code around or otherwise mismanage it.
- xiphias2 3y ago
  They are obligated to give data to the government, and government took part of spying in Brazil for Boeing in the past, but I guess they are using this capability only for a few strategic companies, and most companies are not that.
  - matheusmoreira 3y ago
    > government took part of spying in Brazil for Boeing in the past Do you have more details? Please elaborate.
- zelphirkalt 3y ago
  But that is naive, isn't it? Who has the money and time in their life, to actually sue MS? Even if "you" is a business, few will have the resources for that.
  - justinhj 3y ago
    Individuals do not (although a class action would be feasible), but large companies that use Github and other Microsoft products, of course they have both the means to sue Microsoft and the motivation should their business be impacted.
    - throwaway202303 3y ago
      Exactly

mjr00 3y ago

Absolutely. People will quickly realize that for coding, the natural language part of LLMs is a distraction. Copilot is much better for someone actually writing code, but unfortunately doesn't get as emphasized due to the narrative surrounding LLMs right now.

yodsanklai 3y ago
> Copilot is much better for someone actually writing code I haven't used copilot yet, but I'm using occasionally chatgpt with prompts such as "write a bash/python script take takes these parameters and perform this tasks". Then I iterate if needed, and usually, i can get what i want faster than without using chatgpt. It's not a game changer, but it's a performance boost. How natural language is a distraction here? and how copilot would do much better for the same task?
- mjr00 3y ago
  Try not using natural language and just type what you'd type into Google. You'll get the same results and realize that all of the natural language fluff is totally unnecessary. I just typed in "bash script recursive chmod 777 all files" (as a dumb toy example) and got a resulting script back. It was surrounded by two natural language GPT comments: > It's generally not recommended to give all files and directories the 777 permission as it can pose a security risk. However, if you still want to proceed with this, here's a bash script that recursively changes the permission of all files and directories to 777: [...] Make sure to replace "/path/to/target/directory" with the path of the directory you want to modify. To run the script, save it as a file (e.g., "chmod_all.sh"), make it executable with the command "chmod +x chmod_all.sh", and then run it with "./chmod_all.sh". It's up to the reader to decide if those are necessary, but I'd lean towards no.
  - vorticalbox 3y ago
    you can always include the instruction to only return the code and no other text

maroonblazer 3y ago

As a hobbyist developer with no formal training, I wish Copilot had a 'teaching' or "Senior Dev" mode, where I can play the role of the Junior Dev. I'd like it to pick up on what I'm trying to write, and then prompt me with questions or hints, but not straight up give me the code. Or, if that's too Clippy-like annoying, let me prompt it when I'm stuck, and only then suggest hints or ask suggestive questions that guide me to a solution. I agree, very exciting to see where all this goes.

cwp 3y ago
One thing you might try with Copilot is to ask it to explain the code. It can often give insight, even on code that you yourself wrote a few minutes ago.
ukuina 3y ago
The Github Copilot Labs extension has "codebrushes" that can transform and explain existing code instead of generating new code, but none of it only gives "hints". Maybe one of the codebrushes can take a custom prompt.
- SparkyMcUnicorn 3y ago
  You can create custom brushes, or open the "CoPilot Labs" panel and "explain" with a custom prompt.

SamPatt 3y ago

I've noticed that after using copilot on a code base for a while, you can effectively prompt the AI just by creating a descriptive comment. // This function ends the call by sending a disconnection message to all connected peers Bam, copilot will recommend at least the first line, with subsequent lines usually being pretty good, and more and more frequently, it will recommend the whole function. I still use GPT-4 a lot, especially for troubleshooting errors, but I'm always pleasantly surprised at how good copilot can be.

armchairhacker 3y ago
Copilot is a game-changer and very underrated IMO. GPT4 is smart but not really used in production yet. Copilot is reportedly generating 50% of new code and I can't imagine going without it.
- smashface 3y ago
  Where do you get that 50% number? Do you mean 50% of all new code in the industry? That seems beyond extremely unlikely.
  - fnordpiglet 3y ago
    (I read it as 50% of their code)
  - deleted 3y ago
    [deleted]
  - moyix 3y ago
    The number is 40%, and it's 40% of code written by Copilot users. It's also just for Python: > In files where it’s enabled, nearly 40% of code is being written by GitHub Copilot in popular coding languages, like Python—and we expect that to increase. https://github.blog/2022-06-21-github-copilot-is-generally-available-to-all-developers/ https://github.blog/2022-06-21-github-copilot-is-generally-a...
    - 3y ago

throwaway202303 3y ago

People have different preferences and habits. Having tried both models I much prefer having a conversation in one window and constructing my code from that in another. Although copilot is about to add some interesting features that may win me back.

supernikio2 3y ago

Exactly this. I've tried to implement ChatGPT into may daily workflow, but you have to give it an excruciating level of detail to get something that remotely resembles real code I'd use, and even then you have to hold its hand to guide it in the correct direction, and still have to make some manual final touches at the end. This is why I'm looking forward to Copilot X so much. It will hold much more context than the current implementation, and integrate the Chat interface that's so natural to us.