11 ms·

A 26% productivity increase sounds inline with in my experience. I think one dimension they should explore is whether you're working with a new technology or on

A 26% productivity increase sounds inline with in my experience. I think one dimension they should explore is whether you're working with a new technology or one that you're already familiar with. AI helps me much more with languages/frameworks that I'm trying to learn.

itchyjunk 2y ago
Maybe that's what the new vs experienced difference is indirectly capturing.
jstummbillig 2y ago
That might be. It might also be a consequence of how flexible people at different stages in their career are. I have seen mostly senior programmers argue why ai tools don't work. Juniors just use them without prejudice.
- tuyiown 2y ago
  Flexible, or just that only juniors have real benefits
- infecto 2y ago
  I wish I could upvote this comment more than once. There does appear to be a prejudice with more senior programmers arguing why it cannot work, how they just cause more trouble, and other various complaints. The tools today are not perfect but they still amaze me at what is being accomplished, even a 10% gain is incredible for something that costs $10/month. I believe progress will be made in the space and the tooling in 5 years will be even better.
  - randomdata 2y ago
    > The tools today are not perfect The trouble is that they seem to be getting worse. Some time ago I was able to write an entire small application by simply providing some guidance around function names and data structures, with an LLM filling in all of the rest of the code. It worked fantastically and really showed how these tools can be a boon. I want to taste that same thrill again, but these days I'm lucky if I can get something out of it that will even compile, never mind the logical correctness. Maybe I'm just getting worse at using the tools.
  - mattgreenrocks 2y ago
    The prejudice comes down to whether they want to talk the LLM into the right solution vs applying what they know already. If you know your way around then there’s no need to go through the LLM. I think sr devs often tend to be more task focused so the idea of outsourcing the thinking to an LLM feels like another step to take on. I find Claude good at helping me find how to do things that I know are possible but I don’t have the right nomenclature for. This is an area where Google fails you, as you’re hoping someone else on the internet used similar terms as you when describing the problem. Once it spits out some sort of jargon I can latch onto, then I can Google and find docs to help. I prefer to use multiple sources vs just LLMs, partially because of hallucination, but also to keep amassing my own personal context. LLMs are excellent as librarians.
- jncfhnb 2y ago
  My experience is juniors using them without prejudice and then not understanding why their code is wrong
  - SketchySeaBeast 2y ago
    I am seeing that a lot - juniors who can put out a lot of code but when they get stuck they can't unstick themselves, and it's hard for me to unstick them because they have a hard time walking me through what they are doing.
    - carlmr 2y ago
      I've gotten responses now on PRs of the form. "I don't know either, this is what Copilot told me." If you don't even understand your own PR, I'm not sure why you expect other people can. I have used LLMs myself, but mostly for boilerplate and one-off stuff. I think it can be quite helpful. But as soon as you stop understanding the code it generates you will create subtle bugs everywhere that will cost you dearly in the long run. I have the strong feeling that if LLMs really outsmart us to the degree that some AI gung-ho types believe, the old Kernighan quote will get a new meaning: "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." We'll be left with code nobody can debug because it was handed to us by our super smart AI that only hallucinates sometimes. We'll take the words of another AI that the code works. And then we'll hope for the best.
      - SketchySeaBeast 2y ago
        Copilot is best for me when it's letting me hit tab to auto-complete something I was going to write anyways.
        carlmr 2y ago
        Copilot does hit that sweet to spot sometimes.
    - 2y ago
- GoToRO 2y ago
  As a senior, you know the problem is actually finishing a project. That's the moment all those bad decisions made by junior need to be fixed. This also means that a 80% done project is more like 20% done, because in the state it is, it can not be finished: you fix one thing and break 2 more.
- SketchySeaBeast 2y ago
  As a senior, I find that trying to use copilot really only gives me gains maybe half the time, the other half the time it leads me in the wrong direction. Googling tends to give me a better result because I can actually move through the data quicker. My belief is this is because when I need help I'm doing something uncommon or hard, as opposed to juniors who need help doing regular stuff which will have plenty of examples in the training data. I don't need help with that. It certainly has its uses - it's awesome at mocking and filling in the boilerplate unit tests.
- marginalia_nu 2y ago
  I find their value depends a lot on what I'm doing. Anything easy I'll get insane leverage, no exaggeration I'll slap together that shit 25x faster. It's seen likely billions of lines of simple CRUD endpoints, so yeah it'll write those flawlessly for you. Anything the difficult or complex, and it's really a coinflip if it's even an advantage, most of the time it's just distracting and giving irrelevant suggestions or bad textbook-style implementations intended to demonstrate a principle but with god-awful performance. Likely because there's simply not enough training data for these types of tasks. With this in mind, I don't think it's strange that junior devs would be gushing over this and senior devs would be raising a skeptical eyebrow. Both may be correct, depending on what you work on.
  - jeremy151 2y ago
    I think for me, I'm still learning how to make these tools operate effectively. But even only a few months in, it has removed most all the annoying work and lets me concentrate on the stuff that I like. At this point, I'll often give it some context, tell it what to make and it spits out something relatively close. I look it over, call out like 10 things, each time it says "you're right to question..." and we do an iteration. After we're thru that, I tell it to write a comprehensive set of unit tests, it does that, most of them fail, it fixes them, and them we usually have something pretty solid. Once we have that base pattern, I can have it pattern and extend variants after the first solid bit of code. "Using this pattern for style and approach, make one that does XYZ instead." But what I really appreciate is, I don't have to do the plug and chug stuff. Those patterns are well defined, I'm more than happy to let the LLM do that and concentrate on steering whether it's making a wise conceptual or architectural choice. It really seems to act like a higher abstraction layer. But I think how the engineer uses the tool matters too.
- SoftTalker 2y ago
  Juniors don't know enough to know what problems the AI code might be introducing. It might work, and the tests might pass, but it might be very fragile, full of duplicated code, unnecessary side-effects, etc. that will make future maintenance and debugging difficult. But I guess we'll be using AI for that too, so the hopefully the AI can clean up the messes that it made.
- nuancebydefault 2y ago
  Now some junior dev can quickly make something new and fully functional in days, without knowing in detail what they are doing. As opposed to weeks by a senior originally. Personally I think that senior devs might fear a conflict within their identity. Hence they draw the 'You and the AI have no cue' card.
pqdbr 2y ago
Same impression here. I've been using Cursor for around 10 days on a massive Ruby on Rails project (a stack I've been coding in for +13 years). I didn't enjoy any productivity boost on top of what GitHub Copilot already gave me (which I'd estimate around the 25% mark). However, for crafting a new project from scratch (empty folder) in, say, Node.js, it's uncanny; I can get an API serving requests from a OpenAPI schema (serving the OpenAPI schema via swagger) in ~5 minutes just by prompting. Starting a project from scratch, for me at least, is rare, which probably means going back to Copilot and vanilla VSCode.
- deleted 2y ago
  [deleted]
- gamerDude 2y ago
  I start new projects a lot and just have a template that has everything I would need already set up for me. Do you think there is a unique value prop that AI gives you when setting up that a template would not have?
  - lherron 2y ago
    I do both. I have a basic template that also includes my specific AI instructions and conventions that are inputs for Aider/Cursor. Best of both worlds.
  - carlmr 2y ago
    AI has templates for the languages you seldom use (as long as the language is not too niche for the LLM to have trained on it).
  - pjot 2y ago
    GH Copilot chat specifically has directives like @workspace /new “scaffold a ${language} project” that automagically creates a full project structure and boilerplate. It’s been great for one off things for me at least
    - sethops1 2y ago
moolcool 2y ago
I've found Copilot pretty good at removing some tedium, like it'll write docstrings pretty well most of the time, but it does almost nothing to alleviate the actual mental labour of software engineering
- jacobsenscott 2y ago
  Removing the tedium frees your brain up to focus on the interesting parts of the problem. The "actual mental labor" is the fun part. So I like that.
  - danielvaughn 2y ago
    Yeah this has been my experience. I bucket my work largely into two categories - "creative work" and "toil". I don't want or need AI to replace my creative work, but the more it can handle toil for me, the better. As it happens, I think co-pilot is a pretty poor user experience, because it's essentially just an autocomplete, which doesn't really help me all that much, and often gets in my way. I like using Cursor with the autocomplete turned off. It gives you the option to highlight a bit of text and either refactor it with a prompt, or ask a question about it in a side chat window. That puts me in the driver seat, so to speak, so I (the user) can reach out to AI when I want to.
hobofan 2y ago
I'd also expand it to "languages/frameworks that I'll never properly learn". I'm not great at remembering specific quirks/pitfalls about secondary languages like e.g. what the specific quoting incantations are to write conditionals in Bash, so I rarely wrote bash scripts for automation in the past. Basically only if that was a common enough task to be worth the effort. Same for processing JSON with jq, or parsing with AWK. Now with LLMs, I'm creating a lot more bash scripts, and it has gotten so easy that I'll do it for process-documentation more often. E.g. what previously was a more static step-by-step README with instructions is now accompanied with an interactive bash script that takes user input.
- sgarland 2y ago
  While I’ll grant you that LLMs are in fact shockingly good with shell scripts, I also highly recommend shellcheck [0]. You can get it as a treesitter plugin for nvim, and I assume others, so it lints as you write. [0]: https://www.shellcheck.net/ https://www.shellcheck.net/
  - jcgrillo 2y ago
    I was just about to recommend the same, as well as shfmt and shunit, or even all 3 + a build system: https://www.pantsbuild.org/stable/docs/shell https://www.pantsbuild.org/stable/docs/shell
- danielvaughn 2y ago
  Oh god yes, LLM for bash scripting is fantastic. Bash is one of those languages that I simply don't want to learn. Reading it makes my eyes hurt.
  - a_wild_dandan 2y ago
    You could try zx[1]! I prefer it to Bash scripting. [1] https://github.com/google/zx https://github.com/google/zx
    - 2y ago
empath75 2y ago
Sounds about right to me, which is why the hysteria about AI wiping out developer jobs was always absurd. _Every_ time there has been a technology that improved developer productivity, developer jobs and pay have _increased_. There is not a limited amount of automation that can be done in the world and the cheaper and easier it gets to automate stuff, the more stuff will be economically viable to automate that wasn't before. Did IDE's eliminate developer jobs? Compilers? It's just a tool.
- randomdata 2y ago
  The automated elevator is just a tool, but it "wiped out" the elevator operator. Which is really to say not that the elevator operator was wiped out, but that everyone became the elevator operator. Thus, by the transitive properties of supply and demand, the value of operating an elevator declined to nothing. Said hysteria was built on the same idea. After all, LLMs themselves are just compilers for a programming language that is incredibly similar to spoken language. But as the programming language is incredibly similar to the spoken language that nearly everyone already knows, the idea was that everyone would become the metaphorical elevator operator, "wiping out" programming as a job just as elevator operators were "wiped out" of a job when operating an elevator became accessible to all. The key difference, and where the hysteria is likely to fall flat, is that when riding in an elevator there isn't much else to do but be the elevator operator. You may as well do it. Your situation would not be meaningfully improved if another person was there to press the button for you. When it comes to programming, though, there is more effort involved. Even when a new programming language makes programming accessible, there remains a significant time commitment to carry out the work. The business people are still best to leave that work to the peons so they can continue to focus on the important things.
insane_dreamer 2y ago
I haven't found it that useful in my main product development (which while python based, uses our own framework and therefore there's not much code for CoPilot to go on and it usually suggests methods and arguments that don't exist, which just makes extra work). Where I do find it useful are 1) questions about frameworks/languages that I don't work in much and for which there is a lot of example content (i.e., Qt, CSS); 2) very specific questions I would have done a Google Search (usually StackOverflow) for ("what's the most efficient way to CPU and RAM usage on Windows using python") - the result is pointing me to a library or some example rather than directly generating code that I can copy/paste 3) boilerplate code that I already know how to write but saves me a little time and avoids typing errors. I have the CoPilot plugin for PyCharm so I'll write it as a comment in the file and then it'll complete the next few lines. Again best results is something that is very short and specific. With anything longer I almost always have to iterate so much with CoPilot that it's not worth it anymore. 4) a quick way to search documentation Some people have said it's good at writing unit tests but I have not found that to be the case (at least not the right kind of unit tests). If I had to quantify it, I'd probably give it a 5-10% increase in productivity. Much less than I get from using a full featured IDE like PyCharm over coding in Notepad, or a really good git client over typing the git commands in the CLI. In other words, it's a productivity tool like many other tools, but I would not say it's "revolutionary".
- nuancebydefault 2y ago
  It is revolutionary no doubt. How many pre-AI tools would accomplish an amount of 4 big features you mentioned at once?
  - insane_dreamer 2y ago
    Well, like I said, a well-designed IDE is a much bigger productivity booster than CoPilot, and I've never heard anyone describe JetBrains, NetBeans or VSCode as "revolutionary".
    - nuancebydefault 2y ago
rm_-rf_slash 2y ago
AI speeds my coding up 2x-4x, depending on the breadth of requirements and complexity of new tech to implement. But coding is just a fraction of my weekly workload, and AI has been less impactful for other aspects of project management. So overall it’s 25%-50% increase in productivity.
jakub_g 2y ago
I only use (free) ChatGPT sporadically, and it works best for me in areas where I'm familiar enough to call bullshit, but not familiar enough to write things myself quickly / confidently / without checking a lot of docs: - writing robust bash and using unix/macos tools - how to do X in github actions - which API endpoint do I use to do Y - synthesizing knowledge on some topic that would require dozens of browser tabs - enumerating things to consider when investigating things. Like "I'm seeing X, what could be the cause, and how I do check if it's that". For example I told it last week "git rebase is very slow, what can it be?" and it told me to use GIT_TRACE=1 which made me find a slow post-commit hook, and suggested how to skip this hook while rebasing.
- thiht 2y ago
  Same for me. I also use it for some SQL queries involving syntax I’m unfamiliar with, like JSONB operators in Postgres. ChatGPT gives me better results, faster than Google.

itchyjunk 2y ago

Maybe that's what the new vs experienced difference is indirectly capturing.

jstummbillig 2y ago

That might be. It might also be a consequence of how flexible people at different stages in their career are. I have seen mostly senior programmers argue why ai tools don't work. Juniors just use them without prejudice.

tuyiown 2y ago
Flexible, or just that only juniors have real benefits
infecto 2y ago
I wish I could upvote this comment more than once. There does appear to be a prejudice with more senior programmers arguing why it cannot work, how they just cause more trouble, and other various complaints. The tools today are not perfect but they still amaze me at what is being accomplished, even a 10% gain is incredible for something that costs $10/month. I believe progress will be made in the space and the tooling in 5 years will be even better.
- randomdata 2y ago
  > The tools today are not perfect The trouble is that they seem to be getting worse. Some time ago I was able to write an entire small application by simply providing some guidance around function names and data structures, with an LLM filling in all of the rest of the code. It worked fantastically and really showed how these tools can be a boon. I want to taste that same thrill again, but these days I'm lucky if I can get something out of it that will even compile, never mind the logical correctness. Maybe I'm just getting worse at using the tools.
- mattgreenrocks 2y ago
  The prejudice comes down to whether they want to talk the LLM into the right solution vs applying what they know already. If you know your way around then there’s no need to go through the LLM. I think sr devs often tend to be more task focused so the idea of outsourcing the thinking to an LLM feels like another step to take on. I find Claude good at helping me find how to do things that I know are possible but I don’t have the right nomenclature for. This is an area where Google fails you, as you’re hoping someone else on the internet used similar terms as you when describing the problem. Once it spits out some sort of jargon I can latch onto, then I can Google and find docs to help. I prefer to use multiple sources vs just LLMs, partially because of hallucination, but also to keep amassing my own personal context. LLMs are excellent as librarians.
jncfhnb 2y ago
My experience is juniors using them without prejudice and then not understanding why their code is wrong
- SketchySeaBeast 2y ago
  I am seeing that a lot - juniors who can put out a lot of code but when they get stuck they can't unstick themselves, and it's hard for me to unstick them because they have a hard time walking me through what they are doing.
  - carlmr 2y ago
    I've gotten responses now on PRs of the form. "I don't know either, this is what Copilot told me." If you don't even understand your own PR, I'm not sure why you expect other people can. I have used LLMs myself, but mostly for boilerplate and one-off stuff. I think it can be quite helpful. But as soon as you stop understanding the code it generates you will create subtle bugs everywhere that will cost you dearly in the long run. I have the strong feeling that if LLMs really outsmart us to the degree that some AI gung-ho types believe, the old Kernighan quote will get a new meaning: "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." We'll be left with code nobody can debug because it was handed to us by our super smart AI that only hallucinates sometimes. We'll take the words of another AI that the code works. And then we'll hope for the best.
    - SketchySeaBeast 2y ago
      Copilot is best for me when it's letting me hit tab to auto-complete something I was going to write anyways.
      - carlmr 2y ago
        Copilot does hit that sweet to spot sometimes.
  - 2y ago
GoToRO 2y ago
As a senior, you know the problem is actually finishing a project. That's the moment all those bad decisions made by junior need to be fixed. This also means that a 80% done project is more like 20% done, because in the state it is, it can not be finished: you fix one thing and break 2 more.
SketchySeaBeast 2y ago
As a senior, I find that trying to use copilot really only gives me gains maybe half the time, the other half the time it leads me in the wrong direction. Googling tends to give me a better result because I can actually move through the data quicker. My belief is this is because when I need help I'm doing something uncommon or hard, as opposed to juniors who need help doing regular stuff which will have plenty of examples in the training data. I don't need help with that. It certainly has its uses - it's awesome at mocking and filling in the boilerplate unit tests.
marginalia_nu 2y ago
I find their value depends a lot on what I'm doing. Anything easy I'll get insane leverage, no exaggeration I'll slap together that shit 25x faster. It's seen likely billions of lines of simple CRUD endpoints, so yeah it'll write those flawlessly for you. Anything the difficult or complex, and it's really a coinflip if it's even an advantage, most of the time it's just distracting and giving irrelevant suggestions or bad textbook-style implementations intended to demonstrate a principle but with god-awful performance. Likely because there's simply not enough training data for these types of tasks. With this in mind, I don't think it's strange that junior devs would be gushing over this and senior devs would be raising a skeptical eyebrow. Both may be correct, depending on what you work on.
- jeremy151 2y ago
  I think for me, I'm still learning how to make these tools operate effectively. But even only a few months in, it has removed most all the annoying work and lets me concentrate on the stuff that I like. At this point, I'll often give it some context, tell it what to make and it spits out something relatively close. I look it over, call out like 10 things, each time it says "you're right to question..." and we do an iteration. After we're thru that, I tell it to write a comprehensive set of unit tests, it does that, most of them fail, it fixes them, and them we usually have something pretty solid. Once we have that base pattern, I can have it pattern and extend variants after the first solid bit of code. "Using this pattern for style and approach, make one that does XYZ instead." But what I really appreciate is, I don't have to do the plug and chug stuff. Those patterns are well defined, I'm more than happy to let the LLM do that and concentrate on steering whether it's making a wise conceptual or architectural choice. It really seems to act like a higher abstraction layer. But I think how the engineer uses the tool matters too.
SoftTalker 2y ago
Juniors don't know enough to know what problems the AI code might be introducing. It might work, and the tests might pass, but it might be very fragile, full of duplicated code, unnecessary side-effects, etc. that will make future maintenance and debugging difficult. But I guess we'll be using AI for that too, so the hopefully the AI can clean up the messes that it made.
nuancebydefault 2y ago
Now some junior dev can quickly make something new and fully functional in days, without knowing in detail what they are doing. As opposed to weeks by a senior originally. Personally I think that senior devs might fear a conflict within their identity. Hence they draw the 'You and the AI have no cue' card.

pqdbr 2y ago

Same impression here. I've been using Cursor for around 10 days on a massive Ruby on Rails project (a stack I've been coding in for +13 years). I didn't enjoy any productivity boost on top of what GitHub Copilot already gave me (which I'd estimate around the 25% mark). However, for crafting a new project from scratch (empty folder) in, say, Node.js, it's uncanny; I can get an API serving requests from a OpenAPI schema (serving the OpenAPI schema via swagger) in ~5 minutes just by prompting. Starting a project from scratch, for me at least, is rare, which probably means going back to Copilot and vanilla VSCode.

deleted 2y ago
[deleted]
gamerDude 2y ago
I start new projects a lot and just have a template that has everything I would need already set up for me. Do you think there is a unique value prop that AI gives you when setting up that a template would not have?
- lherron 2y ago
  I do both. I have a basic template that also includes my specific AI instructions and conventions that are inputs for Aider/Cursor. Best of both worlds.
- carlmr 2y ago
  AI has templates for the languages you seldom use (as long as the language is not too niche for the LLM to have trained on it).
- pjot 2y ago
  GH Copilot chat specifically has directives like @workspace /new “scaffold a ${language} project” that automagically creates a full project structure and boilerplate. It’s been great for one off things for me at least
  - sethops1 2y ago

moolcool 2y ago

I've found Copilot pretty good at removing some tedium, like it'll write docstrings pretty well most of the time, but it does almost nothing to alleviate the actual mental labour of software engineering

jacobsenscott 2y ago
Removing the tedium frees your brain up to focus on the interesting parts of the problem. The "actual mental labor" is the fun part. So I like that.
- danielvaughn 2y ago
  Yeah this has been my experience. I bucket my work largely into two categories - "creative work" and "toil". I don't want or need AI to replace my creative work, but the more it can handle toil for me, the better. As it happens, I think co-pilot is a pretty poor user experience, because it's essentially just an autocomplete, which doesn't really help me all that much, and often gets in my way. I like using Cursor with the autocomplete turned off. It gives you the option to highlight a bit of text and either refactor it with a prompt, or ask a question about it in a side chat window. That puts me in the driver seat, so to speak, so I (the user) can reach out to AI when I want to.

hobofan 2y ago

I'd also expand it to "languages/frameworks that I'll never properly learn". I'm not great at remembering specific quirks/pitfalls about secondary languages like e.g. what the specific quoting incantations are to write conditionals in Bash, so I rarely wrote bash scripts for automation in the past. Basically only if that was a common enough task to be worth the effort. Same for processing JSON with jq, or parsing with AWK. Now with LLMs, I'm creating a lot more bash scripts, and it has gotten so easy that I'll do it for process-documentation more often. E.g. what previously was a more static step-by-step README with instructions is now accompanied with an interactive bash script that takes user input.

sgarland 2y ago
While I’ll grant you that LLMs are in fact shockingly good with shell scripts, I also highly recommend shellcheck [0]. You can get it as a treesitter plugin for nvim, and I assume others, so it lints as you write. [0]: https://www.shellcheck.net/ https://www.shellcheck.net/
- jcgrillo 2y ago
  I was just about to recommend the same, as well as shfmt and shunit, or even all 3 + a build system: https://www.pantsbuild.org/stable/docs/shell https://www.pantsbuild.org/stable/docs/shell
danielvaughn 2y ago
Oh god yes, LLM for bash scripting is fantastic. Bash is one of those languages that I simply don't want to learn. Reading it makes my eyes hurt.
- a_wild_dandan 2y ago
  You could try zx[1]! I prefer it to Bash scripting. [1] https://github.com/google/zx https://github.com/google/zx
  - 2y ago

empath75 2y ago

Sounds about right to me, which is why the hysteria about AI wiping out developer jobs was always absurd. _Every_ time there has been a technology that improved developer productivity, developer jobs and pay have _increased_. There is not a limited amount of automation that can be done in the world and the cheaper and easier it gets to automate stuff, the more stuff will be economically viable to automate that wasn't before. Did IDE's eliminate developer jobs? Compilers? It's just a tool.

randomdata 2y ago
The automated elevator is just a tool, but it "wiped out" the elevator operator. Which is really to say not that the elevator operator was wiped out, but that everyone became the elevator operator. Thus, by the transitive properties of supply and demand, the value of operating an elevator declined to nothing. Said hysteria was built on the same idea. After all, LLMs themselves are just compilers for a programming language that is incredibly similar to spoken language. But as the programming language is incredibly similar to the spoken language that nearly everyone already knows, the idea was that everyone would become the metaphorical elevator operator, "wiping out" programming as a job just as elevator operators were "wiped out" of a job when operating an elevator became accessible to all. The key difference, and where the hysteria is likely to fall flat, is that when riding in an elevator there isn't much else to do but be the elevator operator. You may as well do it. Your situation would not be meaningfully improved if another person was there to press the button for you. When it comes to programming, though, there is more effort involved. Even when a new programming language makes programming accessible, there remains a significant time commitment to carry out the work. The business people are still best to leave that work to the peons so they can continue to focus on the important things.

insane_dreamer 2y ago

I haven't found it that useful in my main product development (which while python based, uses our own framework and therefore there's not much code for CoPilot to go on and it usually suggests methods and arguments that don't exist, which just makes extra work). Where I do find it useful are 1) questions about frameworks/languages that I don't work in much and for which there is a lot of example content (i.e., Qt, CSS); 2) very specific questions I would have done a Google Search (usually StackOverflow) for ("what's the most efficient way to CPU and RAM usage on Windows using python") - the result is pointing me to a library or some example rather than directly generating code that I can copy/paste 3) boilerplate code that I already know how to write but saves me a little time and avoids typing errors. I have the CoPilot plugin for PyCharm so I'll write it as a comment in the file and then it'll complete the next few lines. Again best results is something that is very short and specific. With anything longer I almost always have to iterate so much with CoPilot that it's not worth it anymore. 4) a quick way to search documentation Some people have said it's good at writing unit tests but I have not found that to be the case (at least not the right kind of unit tests). If I had to quantify it, I'd probably give it a 5-10% increase in productivity. Much less than I get from using a full featured IDE like PyCharm over coding in Notepad, or a really good git client over typing the git commands in the CLI. In other words, it's a productivity tool like many other tools, but I would not say it's "revolutionary".

nuancebydefault 2y ago
It is revolutionary no doubt. How many pre-AI tools would accomplish an amount of 4 big features you mentioned at once?
- insane_dreamer 2y ago
  Well, like I said, a well-designed IDE is a much bigger productivity booster than CoPilot, and I've never heard anyone describe JetBrains, NetBeans or VSCode as "revolutionary".
  - nuancebydefault 2y ago

rm_-rf_slash 2y ago

AI speeds my coding up 2x-4x, depending on the breadth of requirements and complexity of new tech to implement. But coding is just a fraction of my weekly workload, and AI has been less impactful for other aspects of project management. So overall it’s 25%-50% increase in productivity.

jakub_g 2y ago

I only use (free) ChatGPT sporadically, and it works best for me in areas where I'm familiar enough to call bullshit, but not familiar enough to write things myself quickly / confidently / without checking a lot of docs: - writing robust bash and using unix/macos tools - how to do X in github actions - which API endpoint do I use to do Y - synthesizing knowledge on some topic that would require dozens of browser tabs - enumerating things to consider when investigating things. Like "I'm seeing X, what could be the cause, and how I do check if it's that". For example I told it last week "git rebase is very slow, what can it be?" and it told me to use GIT_TRACE=1 which made me find a slow post-commit hook, and suggested how to skip this hook while rebasing.

thiht 2y ago
Same for me. I also use it for some SQL queries involving syntax I’m unfamiliar with, like JSONB operators in Postgres. ChatGPT gives me better results, faster than Google.