5 ms·

It's even worse. When working with someone else's code e.g. StackOverflow there's a reputation system gating people from the platform and incentivizing them to

It's even worse. When working with someone else's code e.g. StackOverflow there's a reputation system gating people from the platform and incentivizing them to provide correct answers. You can reasonably expect that someone else's code has at least been thought through to some extent to solve the problem at hand, and very likely tested. With LLM-generated code, especially ChatGPT-style decoder models, none of that is true. All of the posts and comments I see about it here seem to be anecdotes "it can do all of my job for me" yet asking it to write the simplest code creates several issues on my end. Personally I think a model geared towards code generation isn't an unsolvable task; the Spider dataset was released some time ago (text to SQL task) and the winning approach there was no fanciness on the model side, but rather to just test all the output queries to ensure it's at least valid SQL. That got a 20%+ boost in accuracy.

Kiro 4y ago
Your experience is no less anecdotal than the millions of people who successfully use Copilot and ChatGPT to write code on a daily basis. I am one of those and can't imagine coding without Copilot or an equivalent ever again.
- soiler 4y ago
  I'm (genuinely) curious what kind of code you write. I haven't tried Copilot and I haven't used ChatGPT very much, but I feel I would be pretty surprised if either of them made significant improvements to my workflow. Copilot I could see, since I already use Intellisense, autocomplete, and snippets to great effect. I'd be annoyed if I had to work without them. But in general, knowing what I want the code to do is >90% of the work of writing new code. I feel there are a few possibilities for why I'm confused: 1. I'm not a very good software engineer, at least in certain respects. Maybe I should have a better understanding of architecture patterns or something I might have learned in a CS degree. Maybe I am hacking everything together and maybe I am already a slow coder. 2. I'm not [being] creative enough as a prompt engineer. I typically can't think of any way that ChatGPT could help me without ingesting my entire repo and figuring out the correct patterns. It could be, however, that there are ways to get the answers I need with better questions. 3. We do completely different kinds of work, and some kinds of coding are better suited for AI assistance than others.
  - zamnos 4y ago
    The opposite of 1 is also possible. You're a really good programmer and know the material better, and just don't need to ask the kinds of questions that other people are asking ChatGPT (or stack overflow, or man pages) for/are happy with your current reference materials.
- harvey9 4y ago
  Define successfully. You might verify what the LLM gives you, but lots of people who blindly copy and paste from stack exchange will do the same with chatgpt
- anileated 4y ago
  Software exists over time. There is no “successful” unless you account for future bugs. I do believe LLM code generators can be used with good results. I just know that for me that way is slower and more painful, because I need to switch between creative mode (when I make stuff) and debugging mode (when I need to figure out how someone else’s stuff works). I find keyboard typing speed is usually not what slows me down the most…
- TeMPOraL 4y ago
  > the millions of people who successfully use Copilot and ChatGPT to write code on a daily basis Where did you get that number from? Are you saying that roughly one in a thousand person on Earth, alive today, is using Copilot and ChatGPT to write code on a daily basis?
  - ivanche 4y ago
    Not the parent but it's not completely impossible. According to [0], there's about 25-30 million software developers in the world. If about 7-8% of them use ChatGPT and Copilot every day, it's already (two) millions. [0] https://www.bairesdev.com/blog/how-many-software-developers-in-the-world/ https://www.bairesdev.com/blog/how-many-software-developers-...
    - abecedarius 4y ago
      I guess it's early for this to matter too much for the count, but people who are not "developers" have also used ChatGPT to write code. I've read anecdotes.

Kiro 4y ago

Your experience is no less anecdotal than the millions of people who successfully use Copilot and ChatGPT to write code on a daily basis. I am one of those and can't imagine coding without Copilot or an equivalent ever again.

soiler 4y ago
I'm (genuinely) curious what kind of code you write. I haven't tried Copilot and I haven't used ChatGPT very much, but I feel I would be pretty surprised if either of them made significant improvements to my workflow. Copilot I could see, since I already use Intellisense, autocomplete, and snippets to great effect. I'd be annoyed if I had to work without them. But in general, knowing what I want the code to do is >90% of the work of writing new code. I feel there are a few possibilities for why I'm confused: 1. I'm not a very good software engineer, at least in certain respects. Maybe I should have a better understanding of architecture patterns or something I might have learned in a CS degree. Maybe I am hacking everything together and maybe I am already a slow coder. 2. I'm not [being] creative enough as a prompt engineer. I typically can't think of any way that ChatGPT could help me without ingesting my entire repo and figuring out the correct patterns. It could be, however, that there are ways to get the answers I need with better questions. 3. We do completely different kinds of work, and some kinds of coding are better suited for AI assistance than others.
- zamnos 4y ago
  The opposite of 1 is also possible. You're a really good programmer and know the material better, and just don't need to ask the kinds of questions that other people are asking ChatGPT (or stack overflow, or man pages) for/are happy with your current reference materials.
harvey9 4y ago
Define successfully. You might verify what the LLM gives you, but lots of people who blindly copy and paste from stack exchange will do the same with chatgpt
anileated 4y ago
Software exists over time. There is no “successful” unless you account for future bugs. I do believe LLM code generators can be used with good results. I just know that for me that way is slower and more painful, because I need to switch between creative mode (when I make stuff) and debugging mode (when I need to figure out how someone else’s stuff works). I find keyboard typing speed is usually not what slows me down the most…
TeMPOraL 4y ago
> the millions of people who successfully use Copilot and ChatGPT to write code on a daily basis Where did you get that number from? Are you saying that roughly one in a thousand person on Earth, alive today, is using Copilot and ChatGPT to write code on a daily basis?
- ivanche 4y ago
  Not the parent but it's not completely impossible. According to [0], there's about 25-30 million software developers in the world. If about 7-8% of them use ChatGPT and Copilot every day, it's already (two) millions. [0] https://www.bairesdev.com/blog/how-many-software-developers-in-the-world/ https://www.bairesdev.com/blog/how-many-software-developers-...
  - abecedarius 4y ago
    I guess it's early for this to matter too much for the count, but people who are not "developers" have also used ChatGPT to write code. I've read anecdotes.