4 ms·

> ChatGPT doesn't do much for experienced SWEs, but it demolishes the difficulty curve for newbies. Interesting, my impression was that ChatGPT helps experienc

> ChatGPT doesn't do much for experienced SWEs, but it demolishes the difficulty curve for newbies. Interesting, my impression was that ChatGPT helps experienced SWEs by filling out large amounts of boilerplate-ish code. It sounds like that's not your experience, though?

RussianCow 3y ago
Which might make you, what, maybe 10% more productive overall? The more experienced you are, the less time you spend writing boilerplate code, so it's not that huge of a win in the grand scheme of things compared to what it provides when you're learning something new.
- koonsolo 3y ago
  Not writing any unit tests are you?!?!? ;)
- anonzzzies 3y ago
  It makes us far more productive; all of us, so we just do more as a team rather than fire someone. For me, copilot writing out if / then / else stuff like it's reading my mind, so tab -> 15 lines, all correct. Or mappings (.map .filter etc) which it completely does within milliseconds. As an experienced programmer, I cannot avoid these things and they do save me a lot of time and debugging. Oh yes, and tests. Going from a jira task to a bunch of jest tests for code that doesn't exist yet is a pretty big win. Very much more than 10% of the average day. You have to be sharp, be a good code reader and writer and it cannot do anything 'alone', but it makes us vastly more efficient.
- raincole 3y ago
  > Which might make you, what, maybe 10% more productive overall? The more experienced you are... But the more exeperienced you are, more valuable your output is (in most cases). So the 10% of your productivity could be more valuable than a novice's 50%.
  - tuyiown 3y ago
    The more experience you are, the less the value is in the output objective productivity. The ability to write self descriptive, maintainable and bitrot resistant code in one pass goes way above feature delivery.
pinkcan 3y ago
that would be copilot not chatgpt
repox 3y ago
I think it works well in both ways. I've been developing software professionally for the past ~20 years and there's no doubt that the combination of ChatGPT and CoPilot greatly reduces the amount of time I would usually spend on boilerplate code. Now I can more or less just review the suggestions, review tests and progress with my solutions a lot faster. At the same time, I see my 9 year old girl using ChatGPT to create Minecraft extensions; she has no prerequisites for actually developing software, but she manages to make some small extensions and have them work either way. Acknowledging that she doesn't actually learn much from this, I think one can assume that everything in between (e.g. using ChatGPT for pair learning) is definitely also possible. But whatever AI is outputting, there's no way these qualified guesses can beat experience from a professional developer. I know that the output will never be better than the prompt, but sometimes the output - no matter how much effort you put into the prompt - is visibly just a qualified guess.
hickelpickle 3y ago
While I am not the most experienced, I did recently finish a CS a degree and find chatgpt a great resource for getting a general understanding to go off of when approaching something outside my domain of experience. Having a core understanding of things in general makes it easier to identify when it's wrong about things, and the majority of time I am not using it to generate code, outside of asking for an example. Being able to ask a few direct question on a topic "demolishes" the time spent to become familiar with something, as most of the time I am just looking for answer about some specifics or to clear up some ambiguity in my understanding of a new concept. I would say with someone new to coding it can be bad and good, as a lot of times it glosses over things, or can be slightly incorrect as it makes assumptions (or more so just answers in a more general context, and when asked to elaborate, or challenged on specifics it will reformat/improve it's answer, but without knowing you need to do so, I could see it easily see it providing half-baked foundational knowledge. You can ask it "x" and it will give a answer, but then if you ask it I am trying to do "y" with "x" and isn't "z" an issue or area of concern with its answer it will reformulate the information provided as its original response was flawed, but if you don't know exactly what the "y" you want to do is, or the "z" being foundational knowledge to challenge it on, you can easily get a whole wall of text that is out of context with what you are actually trying to learn.
- worrycue 3y ago
  > find chatgpt a great resource for getting a general understanding to go off of when approaching something outside my domain of experience. This is my hang up with LLMs … I don’t trust them. Frankly it seems just as easy if not easier to just google keywords and read sites. Google vs LLM is like asking random people on the internet (some are brilliant, some don’t know anything, some are nuts, … etc.) vs asking random people on the internet but all of them have a history of suffering from hallucinations and are routine liars with a compulsive need to answer confidently even when they know nothing.
  - abenga 3y ago
    Especially with an Idea of how they work, I find it very hard to trust them too. Doesn't help that the couple of times I asked them about something subtle, GPT gave me code with the same bugs I was trying to fix. I can see how a beginner would find it useful though.
411111111111111 3y ago
I tried that a few times, but by the time I've explained enough for the code to be close to what I want i could've just implemented it myself twice over... And that's ignoring the fact that it's always generating subtly wrong or just poorly performing code. It might change eventually as ide integrations improve, but for now it's a novelty for me.
mgaunard 3y ago
If you have boilerplate anywhere in your code, then you have a design problem.
- tangjurine 3y ago
  Yeah, my design problem is the language I'm using, the language version I'm using, the outdated framework I'm using, and the 3! frontend frameworks I'm using on my job.
  - mgaunard 3y ago
    Stop using them? A good craftsman crafts his own tools.
    - Toutouxc 3y ago
      Both this post and the one two levels back honestly sound like someone who's never worked in a SW company. Craft my own tools? When? Who's paying for that? Too much boilerplate? Tell that to the 20+ people who built this thing and are long gone. You really have to work for a specific kind of company to be able to change any of that.
      - mgaunard 3y ago
        I have about 15 years experience professionally as a software engineer, in all kinds of companies, at all kinds of different seniority levels. I call bullshit. Your task is to deliver a project that does what it's supposed to, all in a timely fashion. You do whatever it takes, including getting rid of old code and rewriting stuff if it makes you more productive overall. Writing code that does what you want is much faster than taking someone else's code and twist it to do something it was not written to do. You should always be very careful about what dependencies you use, as most often you will have to deal with their limitations and bugs, and they could kill your project. If you had 20 separate people who contributed to the same thing and left then you most likely have huge code debt. Managing that sort of thing is a daily task for any software engineer. Apparently your strategy is to just add debt onto the pile and put your head in the sand pretending it's not your problem. Instead you should plan tactical refactoring moves to make the codebase leaner and better able to adapt to the business needs, all without breaking any existing functionality and workflows. You're asking "when". That's the wrong question. It's up to you to manage your time and invest it into tasks that help you save time later on. A software engineer is usually not paid by the hour either, so you just do the work until you're satisfied with the effort you've put in.
layoric 3y ago
In my experience, the key is that it can also provide good explanations for the basics as well. You can fairly easily overwhelm it, but with some practise of prompting, learning new language/domain basics it is pretty good, and much better than googling to learn the same. The best of all is that you can query your understanding and get decent feedback.
shortcake27 3y ago
As an SWE, my problem isn’t boilerplate. It’s architecture and orchestration. I cancelled my Copilot subscription because it just doesn’t help with this. Let alone ChatGPT which has zero context about the application. I would like an AI to tell me if I’m reinventing the wheel or if something similar can be abstracted. Whether what I’m doing aligns with existing patterns in the codebase. How the application as a whole could be refactored to improve maintainability, performance, or both. This can all already be done by a human with experience and context, so it must be possible with AI. This will be the biggest game changer for me.
- slashdev 3y ago
  I can’t believe you canceled copilot, at $10 a month it pays for itself every single day for me. I’m just baffled how it can be so useful for me and not useful for you.
  - __loam 3y ago
    Once you get over the novelty it gets pretty annoying.
    - slashdev 3y ago
      I’ve been using it for a full year now, that’s just not been my experience.
  - TillE 3y ago
    I've found it extremely similar to IntelliCode for C#, which is not very impressive. Surprisingly I've found it does a better job of writing accurate comments than generating useful code. Maybe it's good for some languages, or for code that's doing the exact stuff a hundred people have already written. Copilot X certainly looks cool, but who knows when that'll be ready.
    - slashdev 3y ago
      I use it the other way around. I write the comments first and then I find it can often fill in the code. I find it especially helpful with SQL queries, and with boilerplate type code. It’s also very helpful when I’d need to lookup the usage of various methods. I just start with the comments and most of the time it saves me the context switch to the browser to lookup the usage.

RussianCow 3y ago

Which might make you, what, maybe 10% more productive overall? The more experienced you are, the less time you spend writing boilerplate code, so it's not that huge of a win in the grand scheme of things compared to what it provides when you're learning something new.

koonsolo 3y ago
Not writing any unit tests are you?!?!? ;)
anonzzzies 3y ago
It makes us far more productive; all of us, so we just do more as a team rather than fire someone. For me, copilot writing out if / then / else stuff like it's reading my mind, so tab -> 15 lines, all correct. Or mappings (.map .filter etc) which it completely does within milliseconds. As an experienced programmer, I cannot avoid these things and they do save me a lot of time and debugging. Oh yes, and tests. Going from a jira task to a bunch of jest tests for code that doesn't exist yet is a pretty big win. Very much more than 10% of the average day. You have to be sharp, be a good code reader and writer and it cannot do anything 'alone', but it makes us vastly more efficient.
raincole 3y ago
> Which might make you, what, maybe 10% more productive overall? The more experienced you are... But the more exeperienced you are, more valuable your output is (in most cases). So the 10% of your productivity could be more valuable than a novice's 50%.
- tuyiown 3y ago
  The more experience you are, the less the value is in the output objective productivity. The ability to write self descriptive, maintainable and bitrot resistant code in one pass goes way above feature delivery.

pinkcan 3y ago

that would be copilot not chatgpt

repox 3y ago

I think it works well in both ways. I've been developing software professionally for the past ~20 years and there's no doubt that the combination of ChatGPT and CoPilot greatly reduces the amount of time I would usually spend on boilerplate code. Now I can more or less just review the suggestions, review tests and progress with my solutions a lot faster. At the same time, I see my 9 year old girl using ChatGPT to create Minecraft extensions; she has no prerequisites for actually developing software, but she manages to make some small extensions and have them work either way. Acknowledging that she doesn't actually learn much from this, I think one can assume that everything in between (e.g. using ChatGPT for pair learning) is definitely also possible. But whatever AI is outputting, there's no way these qualified guesses can beat experience from a professional developer. I know that the output will never be better than the prompt, but sometimes the output - no matter how much effort you put into the prompt - is visibly just a qualified guess.

hickelpickle 3y ago

While I am not the most experienced, I did recently finish a CS a degree and find chatgpt a great resource for getting a general understanding to go off of when approaching something outside my domain of experience. Having a core understanding of things in general makes it easier to identify when it's wrong about things, and the majority of time I am not using it to generate code, outside of asking for an example. Being able to ask a few direct question on a topic "demolishes" the time spent to become familiar with something, as most of the time I am just looking for answer about some specifics or to clear up some ambiguity in my understanding of a new concept. I would say with someone new to coding it can be bad and good, as a lot of times it glosses over things, or can be slightly incorrect as it makes assumptions (or more so just answers in a more general context, and when asked to elaborate, or challenged on specifics it will reformat/improve it's answer, but without knowing you need to do so, I could see it easily see it providing half-baked foundational knowledge. You can ask it "x" and it will give a answer, but then if you ask it I am trying to do "y" with "x" and isn't "z" an issue or area of concern with its answer it will reformulate the information provided as its original response was flawed, but if you don't know exactly what the "y" you want to do is, or the "z" being foundational knowledge to challenge it on, you can easily get a whole wall of text that is out of context with what you are actually trying to learn.

worrycue 3y ago
> find chatgpt a great resource for getting a general understanding to go off of when approaching something outside my domain of experience. This is my hang up with LLMs … I don’t trust them. Frankly it seems just as easy if not easier to just google keywords and read sites. Google vs LLM is like asking random people on the internet (some are brilliant, some don’t know anything, some are nuts, … etc.) vs asking random people on the internet but all of them have a history of suffering from hallucinations and are routine liars with a compulsive need to answer confidently even when they know nothing.
- abenga 3y ago
  Especially with an Idea of how they work, I find it very hard to trust them too. Doesn't help that the couple of times I asked them about something subtle, GPT gave me code with the same bugs I was trying to fix. I can see how a beginner would find it useful though.

411111111111111 3y ago

I tried that a few times, but by the time I've explained enough for the code to be close to what I want i could've just implemented it myself twice over... And that's ignoring the fact that it's always generating subtly wrong or just poorly performing code. It might change eventually as ide integrations improve, but for now it's a novelty for me.

mgaunard 3y ago

If you have boilerplate anywhere in your code, then you have a design problem.

tangjurine 3y ago
Yeah, my design problem is the language I'm using, the language version I'm using, the outdated framework I'm using, and the 3! frontend frameworks I'm using on my job.
- mgaunard 3y ago
  Stop using them? A good craftsman crafts his own tools.
  - Toutouxc 3y ago
    Both this post and the one two levels back honestly sound like someone who's never worked in a SW company. Craft my own tools? When? Who's paying for that? Too much boilerplate? Tell that to the 20+ people who built this thing and are long gone. You really have to work for a specific kind of company to be able to change any of that.
    - mgaunard 3y ago
      I have about 15 years experience professionally as a software engineer, in all kinds of companies, at all kinds of different seniority levels. I call bullshit. Your task is to deliver a project that does what it's supposed to, all in a timely fashion. You do whatever it takes, including getting rid of old code and rewriting stuff if it makes you more productive overall. Writing code that does what you want is much faster than taking someone else's code and twist it to do something it was not written to do. You should always be very careful about what dependencies you use, as most often you will have to deal with their limitations and bugs, and they could kill your project. If you had 20 separate people who contributed to the same thing and left then you most likely have huge code debt. Managing that sort of thing is a daily task for any software engineer. Apparently your strategy is to just add debt onto the pile and put your head in the sand pretending it's not your problem. Instead you should plan tactical refactoring moves to make the codebase leaner and better able to adapt to the business needs, all without breaking any existing functionality and workflows. You're asking "when". That's the wrong question. It's up to you to manage your time and invest it into tasks that help you save time later on. A software engineer is usually not paid by the hour either, so you just do the work until you're satisfied with the effort you've put in.

layoric 3y ago

In my experience, the key is that it can also provide good explanations for the basics as well. You can fairly easily overwhelm it, but with some practise of prompting, learning new language/domain basics it is pretty good, and much better than googling to learn the same. The best of all is that you can query your understanding and get decent feedback.

shortcake27 3y ago

As an SWE, my problem isn’t boilerplate. It’s architecture and orchestration. I cancelled my Copilot subscription because it just doesn’t help with this. Let alone ChatGPT which has zero context about the application. I would like an AI to tell me if I’m reinventing the wheel or if something similar can be abstracted. Whether what I’m doing aligns with existing patterns in the codebase. How the application as a whole could be refactored to improve maintainability, performance, or both. This can all already be done by a human with experience and context, so it must be possible with AI. This will be the biggest game changer for me.

slashdev 3y ago
I can’t believe you canceled copilot, at $10 a month it pays for itself every single day for me. I’m just baffled how it can be so useful for me and not useful for you.
- __loam 3y ago
  Once you get over the novelty it gets pretty annoying.
  - slashdev 3y ago
    I’ve been using it for a full year now, that’s just not been my experience.
- TillE 3y ago
  I've found it extremely similar to IntelliCode for C#, which is not very impressive. Surprisingly I've found it does a better job of writing accurate comments than generating useful code. Maybe it's good for some languages, or for code that's doing the exact stuff a hundred people have already written. Copilot X certainly looks cool, but who knows when that'll be ready.
  - slashdev 3y ago
    I use it the other way around. I write the comments first and then I find it can often fill in the code. I find it especially helpful with SQL queries, and with boilerplate type code. It’s also very helpful when I’d need to lookup the usage of various methods. I just start with the comments and most of the time it saves me the context switch to the browser to lookup the usage.