5 ms·

Absolutely. People will quickly realize that for coding, the natural language part of LLMs is a distraction. Copilot is much better for someone actually writing

by mjr00 4y ago

Absolutely. People will quickly realize that for coding, the natural language part of LLMs is a distraction. Copilot is much better for someone actually writing code, but unfortunately doesn't get as emphasized due to the narrative surrounding LLMs right now.

yodsanklai 4y ago
> Copilot is much better for someone actually writing code I haven't used copilot yet, but I'm using occasionally chatgpt with prompts such as "write a bash/python script take takes these parameters and perform this tasks". Then I iterate if needed, and usually, i can get what i want faster than without using chatgpt. It's not a game changer, but it's a performance boost. How natural language is a distraction here? and how copilot would do much better for the same task?
- mjr00 4y ago
  Try not using natural language and just type what you'd type into Google. You'll get the same results and realize that all of the natural language fluff is totally unnecessary. I just typed in "bash script recursive chmod 777 all files" (as a dumb toy example) and got a resulting script back. It was surrounded by two natural language GPT comments: > It's generally not recommended to give all files and directories the 777 permission as it can pose a security risk. However, if you still want to proceed with this, here's a bash script that recursively changes the permission of all files and directories to 777: [...] Make sure to replace "/path/to/target/directory" with the path of the directory you want to modify. To run the script, save it as a file (e.g., "chmod_all.sh"), make it executable with the command "chmod +x chmod_all.sh", and then run it with "./chmod_all.sh". It's up to the reader to decide if those are necessary, but I'd lean towards no.
  - vorticalbox 4y ago
    you can always include the instruction to only return the code and no other text
    - mjr00 4y ago
      Sure, but I want a system built for coding that does that by default... like Copilot.
    - iudqnolq 4y ago
      ... and it'll describe the code anyway, at least to me.
  - gunapologist99 4y ago
    No script needed: chmod ugo+rwX . -R (This is for GNU chmod like in Linux, BSD will be slightly different) Of course, that's not exactly what you asked for (it's better, read the chmod man page: X applies executable only to directories) but you could just replace ugo+rwX with 777 or 0777.
  - deleted 4y ago
    [deleted]
  - kenjackson 4y ago
    I tried this with the following: "Bash script to add a string I specify to the beginning of every file in a directory, unless the file begins with “archive”" I tried looking for this on Google and didn't find anything that did this -- although I could cobble together a solution with a couple of queries. The interesting thing is that I wanted ChatGPT to append the string to the filename -- that's what I meant. But it actually append the string to the actual file. That's actually what I said, so I give it credit for doing what I said, rather than what I meant. And honestly my intent isn't necessarily obvious. I definitely see this as a value add over just searching with Google.
  - scarface74 4y ago
    > Try not using natural language and just type what you'd type into Google. You'll get the same results and realize that all of the natural language fluff is totally unnecessary. I can get similar results with Google sometimes and I can put together what I learned from different places. But I can get scripts that meet my exact requirements with ChatGPT. Most of my ChatGPT related code is scripting AWS related code and CloudFormation templates. I’ve asked it to translate AWS related Python code to Node for a different projects and a bash shell script. It’s well trained on AWS related code. I don’t know PowerShell from a hole in the wall. But I needed to write PS scripts and it did it. I’ve also used it to convert CloudFormation to Terraform
    - mjr00 4y ago
      I think you (and kenjackson above) are misinterpreting what I was saying. I'm not saying use Google instead of ChatGPT; I'm saying pretend ChatGPT is Google and interact with the ChatGPT text prompt the same way. You don't need fully formed coherent sentences like you would when talking to a person; just drop in relevant keywords and ChatGPT will get you what you want.
      - scarface74 4y ago
        Isn’t that the game changer though that you can use natural language and treat it like the “worlds smartest intern” and I can just give it the list of my requirements? It’s the difference between: “Python script to return all of the roles with a given policy AWS” (answer found on StackOverflow with Google) And with ChatGPT “Write a Python script that returns AWS IAM roles that contain one or more policies specified by one or more -p arguments. Use argparse to accept parameters and output the found roles as a comma separated list”
        mjr00 4y ago
        > “Write a Python script that returns AWS IAM roles that contain one or more policies specified by one or more -p arguments. Use argparse to accept parameters and output the found roles as a comma separated list” Again, this is completely unnecessary. This is like in the old days when technically illiterate people would quite literally Ask Jeeves[0] and search for full questions because they didn't know how to interface with a search engine. A prompt that does exactly what you're asking: "python script get AWS IAM roles that contain a policy, policy as -p command line argument, output csv" We'll see more of that terse, efficient, style as people get more comfortable, similar to how people have (mostly) stopped using full questions to search on Google. The "talk to ChatGPT like a human" part is entirely a distraction from taking advantage of the LLM for coding purposes. Perhaps more importantly, the responses being humanized is a distraction, too. [0] https://en.wikipedia.org/wiki/Ask.com https://en.wikipedia.org/wiki/Ask.com
- visarga 4y ago
  > It's not a game changer, but it's a performance boost. The story of all AI in 2023 - maybe 2x performance improvement, maybe a bit less. The big problem is that you can't trust it on its own, so it doesn't improve productivity 100x. Not even a receipt reader is good enough to reach 100%, you got to check the total, maybe it missed the dot and you get the 100x boost after all.
moffkalast 4y ago
Has the Copilot backend been updated to use anything more advanced yet? I tried it out when it was new and free for a while and it really struggled with anything that wasn't incredibly common. GPT 4 in its chat form works a whole lot better for niche stuff than that one did.
- gunapologist99 4y ago
  It's definitely far better than when it was free but not GPT4 yet for most people. It's the opposite of chatGPT: it takes more time to produce useful output but it gets much better in more complex programs while ChatGPT gets worse.
- kgeist 4y ago
  Copilot's original underlying model is currently deprecated, if I remember correctly
avereveard 4y ago
Idk for sure autocomplete is a great interface for someone in the ide coding, but LLM can understand requirements whole and spit out full classes and validate that the output from the server matches the specs, they work great from outside an ide.

by mjr00 4y ago

yodsanklai 4y ago

> Copilot is much better for someone actually writing code I haven't used copilot yet, but I'm using occasionally chatgpt with prompts such as "write a bash/python script take takes these parameters and perform this tasks". Then I iterate if needed, and usually, i can get what i want faster than without using chatgpt. It's not a game changer, but it's a performance boost. How natural language is a distraction here? and how copilot would do much better for the same task?

mjr00 4y ago
Try not using natural language and just type what you'd type into Google. You'll get the same results and realize that all of the natural language fluff is totally unnecessary. I just typed in "bash script recursive chmod 777 all files" (as a dumb toy example) and got a resulting script back. It was surrounded by two natural language GPT comments: > It's generally not recommended to give all files and directories the 777 permission as it can pose a security risk. However, if you still want to proceed with this, here's a bash script that recursively changes the permission of all files and directories to 777: [...] Make sure to replace "/path/to/target/directory" with the path of the directory you want to modify. To run the script, save it as a file (e.g., "chmod_all.sh"), make it executable with the command "chmod +x chmod_all.sh", and then run it with "./chmod_all.sh". It's up to the reader to decide if those are necessary, but I'd lean towards no.
- vorticalbox 4y ago
  you can always include the instruction to only return the code and no other text
  - mjr00 4y ago
    Sure, but I want a system built for coding that does that by default... like Copilot.
  - iudqnolq 4y ago
    ... and it'll describe the code anyway, at least to me.
- gunapologist99 4y ago
  No script needed: chmod ugo+rwX . -R (This is for GNU chmod like in Linux, BSD will be slightly different) Of course, that's not exactly what you asked for (it's better, read the chmod man page: X applies executable only to directories) but you could just replace ugo+rwX with 777 or 0777.
- deleted 4y ago
  [deleted]
- kenjackson 4y ago
  I tried this with the following: "Bash script to add a string I specify to the beginning of every file in a directory, unless the file begins with “archive”" I tried looking for this on Google and didn't find anything that did this -- although I could cobble together a solution with a couple of queries. The interesting thing is that I wanted ChatGPT to append the string to the filename -- that's what I meant. But it actually append the string to the actual file. That's actually what I said, so I give it credit for doing what I said, rather than what I meant. And honestly my intent isn't necessarily obvious. I definitely see this as a value add over just searching with Google.
- scarface74 4y ago
  > Try not using natural language and just type what you'd type into Google. You'll get the same results and realize that all of the natural language fluff is totally unnecessary. I can get similar results with Google sometimes and I can put together what I learned from different places. But I can get scripts that meet my exact requirements with ChatGPT. Most of my ChatGPT related code is scripting AWS related code and CloudFormation templates. I’ve asked it to translate AWS related Python code to Node for a different projects and a bash shell script. It’s well trained on AWS related code. I don’t know PowerShell from a hole in the wall. But I needed to write PS scripts and it did it. I’ve also used it to convert CloudFormation to Terraform
  - mjr00 4y ago
    I think you (and kenjackson above) are misinterpreting what I was saying. I'm not saying use Google instead of ChatGPT; I'm saying pretend ChatGPT is Google and interact with the ChatGPT text prompt the same way. You don't need fully formed coherent sentences like you would when talking to a person; just drop in relevant keywords and ChatGPT will get you what you want.
    - scarface74 4y ago
      Isn’t that the game changer though that you can use natural language and treat it like the “worlds smartest intern” and I can just give it the list of my requirements? It’s the difference between: “Python script to return all of the roles with a given policy AWS” (answer found on StackOverflow with Google) And with ChatGPT “Write a Python script that returns AWS IAM roles that contain one or more policies specified by one or more -p arguments. Use argparse to accept parameters and output the found roles as a comma separated list”
      - mjr00 4y ago
        > “Write a Python script that returns AWS IAM roles that contain one or more policies specified by one or more -p arguments. Use argparse to accept parameters and output the found roles as a comma separated list” Again, this is completely unnecessary. This is like in the old days when technically illiterate people would quite literally Ask Jeeves[0] and search for full questions because they didn't know how to interface with a search engine. A prompt that does exactly what you're asking: "python script get AWS IAM roles that contain a policy, policy as -p command line argument, output csv" We'll see more of that terse, efficient, style as people get more comfortable, similar to how people have (mostly) stopped using full questions to search on Google. The "talk to ChatGPT like a human" part is entirely a distraction from taking advantage of the LLM for coding purposes. Perhaps more importantly, the responses being humanized is a distraction, too. [0] https://en.wikipedia.org/wiki/Ask.com https://en.wikipedia.org/wiki/Ask.com
visarga 4y ago
> It's not a game changer, but it's a performance boost. The story of all AI in 2023 - maybe 2x performance improvement, maybe a bit less. The big problem is that you can't trust it on its own, so it doesn't improve productivity 100x. Not even a receipt reader is good enough to reach 100%, you got to check the total, maybe it missed the dot and you get the 100x boost after all.

moffkalast 4y ago

Has the Copilot backend been updated to use anything more advanced yet? I tried it out when it was new and free for a while and it really struggled with anything that wasn't incredibly common. GPT 4 in its chat form works a whole lot better for niche stuff than that one did.

gunapologist99 4y ago
It's definitely far better than when it was free but not GPT4 yet for most people. It's the opposite of chatGPT: it takes more time to produce useful output but it gets much better in more complex programs while ChatGPT gets worse.
kgeist 4y ago
Copilot's original underlying model is currently deprecated, if I remember correctly

avereveard 4y ago

Idk for sure autocomplete is a great interface for someone in the ide coding, but LLM can understand requirements whole and spit out full classes and validate that the output from the server matches the specs, they work great from outside an ide.