3 ms·

I'd argue that the productivity boost is mich higher for the lower end developers than for the best ones. They are best at the simple tasks in the most verbose

I'd argue that the productivity boost is mich higher for the lower end developers than for the best ones. They are best at the simple tasks in the most verbose programming languages: Easier to think of them as higher quality auto-completion. The terser the language, and more innovative the problem is, the smalle the LLM help. So if anyone gains from this, is the companies paying under-market, and therefore typically getting somewhat substandard devs to do boring things. There's major savings in productivity there, precisely because the training for those tasks is so good. If you had your highest performance developers spending a significant amount of time on that kind of busywork, you weren't getting that great of a performance in the first place.

JohnFen 3y ago
Maybe this is the answer to a disconnect I've noticed. A number of devs where I work have started using ChatGPT to help with their work, but it hasn't resulted in any noticeable productivity or quality gains. But the devs here are all very experienced senior-level engineers. Perhaps that's why we aren't seeing any gains?
- giantg2 3y ago
  Of it's anything like my job, it's likely that a majority of the slowness is in the non-coding steps - meetings, upgrades/config, business ideation and prioritization, etc.
  - ryandrake 3y ago
    I mean, that's like most non-junior software development jobs--at least it has been the case for all the ones I've had: Actual, physical, typing-in-of-code is usually about 5-10% of any software developer's job. The rest is tools management, configuration, compiling, merging/source control, code reviews, helping other developers, brainstorming, planning, design reviews, bug management, status reporting, project management rituals like scrum, and so on. I think many software developers have this romantic, idealized notion that their entire job should be furiously typing code into a computer, and that anything else they have to grudgingly do is unproductive overhead.
- hackermatic 3y ago
  I was asked to help evaluate a couple code-generating LLMs at work, so I set out to build a couple toy apps in languages and frameworks I wasn't very familiar with. My experience was that it was about as useful as searching Google and Stack Overflow without knowing what a good answer looks like. You copy and paste a lot of code, a lot of it works, but inevitably, something breaks and you have no idea why. So you still have to go back to "the hard way" of reading the documentation and building an understanding. I'm sure things will improve over time, especially for products built specifically for code generation, but my first and second impression is that so far, LLMs don't actually lower the bar for junior programmers.
- smabie 3y ago
  Maybe the devs now can work fewer hours since they more productive per hour. Otherwise you're suggesting that these senior devs are using a tool which provides no value whatsoever which seems unlikely.
marcosdumay 3y ago
This is a first order short term phenomenon. But also, the LLMs by themselves guide the developers into creating repetitive low quality code, and any bias into using a LLM will bias the developers into adopting verbose boilerplate-happy languages and frameworks.
two_in_one 3y ago
> I'd argue that the productivity boost is mich higher for the lower end developers than for the best ones. I'd say it depends on the task more than on dev level. If it's maintaining, improving, or refactoring of a large project LLM can't help much. When it comes to small utilities GPT-4 is a time saver. You can even write a small playable game with it in a matter of hours. Using any popular language. Simple web pages, JS are much easier. With ChatGPT-4 I've learned how to do certain things using ffmpeg. Without it would take hours of googling. PS: talking about ChatGPT degradation with the time. I didn't see it on programming tasks. But it became noticeably faster.

JohnFen 3y ago

Maybe this is the answer to a disconnect I've noticed. A number of devs where I work have started using ChatGPT to help with their work, but it hasn't resulted in any noticeable productivity or quality gains. But the devs here are all very experienced senior-level engineers. Perhaps that's why we aren't seeing any gains?

giantg2 3y ago
Of it's anything like my job, it's likely that a majority of the slowness is in the non-coding steps - meetings, upgrades/config, business ideation and prioritization, etc.
- ryandrake 3y ago
  I mean, that's like most non-junior software development jobs--at least it has been the case for all the ones I've had: Actual, physical, typing-in-of-code is usually about 5-10% of any software developer's job. The rest is tools management, configuration, compiling, merging/source control, code reviews, helping other developers, brainstorming, planning, design reviews, bug management, status reporting, project management rituals like scrum, and so on. I think many software developers have this romantic, idealized notion that their entire job should be furiously typing code into a computer, and that anything else they have to grudgingly do is unproductive overhead.
hackermatic 3y ago
I was asked to help evaluate a couple code-generating LLMs at work, so I set out to build a couple toy apps in languages and frameworks I wasn't very familiar with. My experience was that it was about as useful as searching Google and Stack Overflow without knowing what a good answer looks like. You copy and paste a lot of code, a lot of it works, but inevitably, something breaks and you have no idea why. So you still have to go back to "the hard way" of reading the documentation and building an understanding. I'm sure things will improve over time, especially for products built specifically for code generation, but my first and second impression is that so far, LLMs don't actually lower the bar for junior programmers.
smabie 3y ago
Maybe the devs now can work fewer hours since they more productive per hour. Otherwise you're suggesting that these senior devs are using a tool which provides no value whatsoever which seems unlikely.

marcosdumay 3y ago

This is a first order short term phenomenon. But also, the LLMs by themselves guide the developers into creating repetitive low quality code, and any bias into using a LLM will bias the developers into adopting verbose boilerplate-happy languages and frameworks.

two_in_one 3y ago

> I'd argue that the productivity boost is mich higher for the lower end developers than for the best ones. I'd say it depends on the task more than on dev level. If it's maintaining, improving, or refactoring of a large project LLM can't help much. When it comes to small utilities GPT-4 is a time saver. You can even write a small playable game with it in a matter of hours. Using any popular language. Simple web pages, JS are much easier. With ChatGPT-4 I've learned how to do certain things using ffmpeg. Without it would take hours of googling. PS: talking about ChatGPT degradation with the time. I didn't see it on programming tasks. But it became noticeably faster.