5 ms·

One of the biggest impacts of LLMs right now is probably in programming. The article says that the only thing LLMs are doing is replacing stack overflow (and he

by gpsx 3y ago

One of the biggest impacts of LLMs right now is probably in programming. The article says that the only thing LLMs are doing is replacing stack overflow (and hence the value is the value of stack overflow). While it's true it does replace the need for stack overflow in many cases, what it is doing is making programmers much more productive. How much? I don't know. But the value is not that we don't need stack overflow, it is that we don't need as many programmers.

dark_star 3y ago
> But the value is not that we don't need stack overflow, it is that we don't need as many programmers. I'll believe that when I see it. I lead a team of AI-boosted developers, I'm hiring as fast as I can, and all my company wants out of them is more and more. If they have 2x, 3x, or 10x output, the company can and will use it all.
- gpsx 3y ago
  Yes, I should have said we don't need as many programmers to do the same thing. As you say, companies will take advantage of being able to do more, and then there will hopefully be more software needed to leverage the capabiliies of the new AI agents.
- akira2501 3y ago
  So, "output" is the only metric your company uses? And it has an unlimited appetite for "output?" I work with an "AI-boosted" developer. He would be a 10x better developer if he stopped querying the chatbot and started reading basic documentation. We'd have a lot less spaghetti in "his?" codebase. Where do you work? Maybe we can solve each others problem?
  - TheLoafOfBread 3y ago
    This is my experience with a colleague of mine as well. He is talking to chatgpt and trying to get working solution from it instead of sitting down and putting puzzle pieces together from documentation. He would have been already done with the work if he would stop sieving through AI generated garbage.
- tkgally 3y ago
  Just out of curiosity: Are you hiring only experienced developers? Or are you also hiring junior developers who are just getting started in the field? Discussions here on HN about whether or not AI is a threat to developers’ jobs seem to suggest that experienced people maybe don’t have to worry, partly because so much of their work is not coding per se. I wonder, though, about people just starting out. I heard anecdotally a couple of weeks ago about an SV company that has stopped hiring junior developers because the work they used to do can be done much more quickly and cheaply with AI. I am not a developer myself, but I worked for many years in translation, a field that also seems threatened by AI. LLMs can be powerful tools to assist skilled translators, but they might also be making it difficult for beginners to get started in the field.
smsm42 3y ago
It does replace the "lazy" part of StackOverflow but not the most valuable part of it. The lazy part is just responses to people that failed to find the manual and/or read it properly, and the responses just quote the docs back to them or show them how to apply the docs to a pretty straightforward examples. And this LLMs can do very well - and I don't mean to discount this, finding some corner case in the manuals can be very annoying and time-consuming endeavor, and LLM can save hours here. But: any task that is minimally novel and non-trivial gets it completely stumped and it starts spewing pure nonsense. And plausibly looking nonsense at that, which is even worse - like construct the code that seemingly works except couple of key functions don't actually exist - LLM just hallucinated them because if they existed that how it'd work. Happened to me more times than I want to count. That's where SO is valuable because - occasionally - it is answered by people that actually understand what is going on and don't just regurgitate pre-digested information. Unfortunately, that's what LLMs are still largely incapable of. As regurgitators, they are probably the best tool out there. But beyond that - you'll still need to talk to somebody who understands.
- mirsadm 3y ago
  Most of my use cases for stackiverflow end up being those lazy lookups because it's super quick. It's often much quicker than chatgpt for me to Google and click the first (or these days 8th) link
GeneralMaximus 3y ago
I've been using GitHub Copilot since it was in early preview. I still have it turned on in VS Code. Sometimes I have a spotty Internet connection and it fails to return any completions. I barely notice when that happens. If my text editor stopped doing syntax highlighting or communicating with my language server, I'd notice right away. It would seriously impact my productivity to a point that I'd be looking for a new editor. But the completions I get from Copilot don't have that much value, and I don't consider them an essential part of my programming life. They help with reducing the amount of text I need to input manually, but rarely do they help me solve any real problems or generate novel insights. The article is correct. Most of the time, Copilot is just replacing Google search or StackOverflow for me. And even then, the information it returns is sometimes outdated and doesn't cover programming languages that aren't very popular (e.g Raku).
CipherThrowaway 3y ago
> it is that we don't need as many programmers. This sounds a bit like a variant of the old "lump of labor" misconception. There's no such thing as needing programmers. In the long run, economic decisions in markets are made at the margin, and increases in marginal productivity make labor more valuable rather than less. This induces rather than reduces consumption. There are caveats of course: the benefits of induced consumption may not be distributed to all devs evenly, or may not be distributed evenly between capital and labor. But the idea that making programmers more productive reduces the need for programmers - ceteris paribus - is mistaken. The degree to which the development market will expand as a result of increases in developer productivity ultimately depends on the elasticity of demand for development. But it's hard to say the market is anywhere close to saturated. This might come as a surprise to some in the HN bubble, but programming is so inefficient and difficult to engage right now that the default way for businesses to build software and software systems is through untrained office workers and consultants hacking together Excel formulas, no-code builders and workflow configurations in giant ERPs and CRMs.
crucialfelix 3y ago
Every company wants to build more, ship faster. AI enhanced programmers increase output. Companies will just want to do more. In my opinion it's depressing how slow development is. The non technical managers can't understand it either. It's complex and boring and error prone and fragile. That's what we need to fix.
- yetanother12345 3y ago
  > It's complex and boring and error prone and fragile ... And, if you want to fix this you must first try to understand why it is so. Software development is, well... development. Those doing that are in fact inventing things. The act of inventing things is, literally, by nature "complex and boring and error prone and fragile" So, there you go. I'd mark it wontfix YMMV
nitwit005 3y ago
My experience would suggest, the stack overflow metaphor isn't far off. A lot of the code you can generate can also be looked up. Although ChatGPT can add great emotional content you won't get from just looking something up: std::cout << "The journey to find these primes leaves me feeling both fulfilled and hollow." << std::endl; std::cout << "For these numbers are mere representations of patterns, devoid of emotion or purpose." << std::endl; std::cout << "I continue to seek meaning within the calculations, but the quest remains unfulfilled." << std::endl;

by gpsx 3y ago

dark_star 3y ago

> But the value is not that we don't need stack overflow, it is that we don't need as many programmers. I'll believe that when I see it. I lead a team of AI-boosted developers, I'm hiring as fast as I can, and all my company wants out of them is more and more. If they have 2x, 3x, or 10x output, the company can and will use it all.

gpsx 3y ago
Yes, I should have said we don't need as many programmers to do the same thing. As you say, companies will take advantage of being able to do more, and then there will hopefully be more software needed to leverage the capabiliies of the new AI agents.
akira2501 3y ago
So, "output" is the only metric your company uses? And it has an unlimited appetite for "output?" I work with an "AI-boosted" developer. He would be a 10x better developer if he stopped querying the chatbot and started reading basic documentation. We'd have a lot less spaghetti in "his?" codebase. Where do you work? Maybe we can solve each others problem?
- TheLoafOfBread 3y ago
  This is my experience with a colleague of mine as well. He is talking to chatgpt and trying to get working solution from it instead of sitting down and putting puzzle pieces together from documentation. He would have been already done with the work if he would stop sieving through AI generated garbage.
tkgally 3y ago
Just out of curiosity: Are you hiring only experienced developers? Or are you also hiring junior developers who are just getting started in the field? Discussions here on HN about whether or not AI is a threat to developers’ jobs seem to suggest that experienced people maybe don’t have to worry, partly because so much of their work is not coding per se. I wonder, though, about people just starting out. I heard anecdotally a couple of weeks ago about an SV company that has stopped hiring junior developers because the work they used to do can be done much more quickly and cheaply with AI. I am not a developer myself, but I worked for many years in translation, a field that also seems threatened by AI. LLMs can be powerful tools to assist skilled translators, but they might also be making it difficult for beginners to get started in the field.

smsm42 3y ago

It does replace the "lazy" part of StackOverflow but not the most valuable part of it. The lazy part is just responses to people that failed to find the manual and/or read it properly, and the responses just quote the docs back to them or show them how to apply the docs to a pretty straightforward examples. And this LLMs can do very well - and I don't mean to discount this, finding some corner case in the manuals can be very annoying and time-consuming endeavor, and LLM can save hours here. But: any task that is minimally novel and non-trivial gets it completely stumped and it starts spewing pure nonsense. And plausibly looking nonsense at that, which is even worse - like construct the code that seemingly works except couple of key functions don't actually exist - LLM just hallucinated them because if they existed that how it'd work. Happened to me more times than I want to count. That's where SO is valuable because - occasionally - it is answered by people that actually understand what is going on and don't just regurgitate pre-digested information. Unfortunately, that's what LLMs are still largely incapable of. As regurgitators, they are probably the best tool out there. But beyond that - you'll still need to talk to somebody who understands.

mirsadm 3y ago
Most of my use cases for stackiverflow end up being those lazy lookups because it's super quick. It's often much quicker than chatgpt for me to Google and click the first (or these days 8th) link

GeneralMaximus 3y ago

I've been using GitHub Copilot since it was in early preview. I still have it turned on in VS Code. Sometimes I have a spotty Internet connection and it fails to return any completions. I barely notice when that happens. If my text editor stopped doing syntax highlighting or communicating with my language server, I'd notice right away. It would seriously impact my productivity to a point that I'd be looking for a new editor. But the completions I get from Copilot don't have that much value, and I don't consider them an essential part of my programming life. They help with reducing the amount of text I need to input manually, but rarely do they help me solve any real problems or generate novel insights. The article is correct. Most of the time, Copilot is just replacing Google search or StackOverflow for me. And even then, the information it returns is sometimes outdated and doesn't cover programming languages that aren't very popular (e.g Raku).

CipherThrowaway 3y ago

> it is that we don't need as many programmers. This sounds a bit like a variant of the old "lump of labor" misconception. There's no such thing as needing programmers. In the long run, economic decisions in markets are made at the margin, and increases in marginal productivity make labor more valuable rather than less. This induces rather than reduces consumption. There are caveats of course: the benefits of induced consumption may not be distributed to all devs evenly, or may not be distributed evenly between capital and labor. But the idea that making programmers more productive reduces the need for programmers - ceteris paribus - is mistaken. The degree to which the development market will expand as a result of increases in developer productivity ultimately depends on the elasticity of demand for development. But it's hard to say the market is anywhere close to saturated. This might come as a surprise to some in the HN bubble, but programming is so inefficient and difficult to engage right now that the default way for businesses to build software and software systems is through untrained office workers and consultants hacking together Excel formulas, no-code builders and workflow configurations in giant ERPs and CRMs.

crucialfelix 3y ago

Every company wants to build more, ship faster. AI enhanced programmers increase output. Companies will just want to do more. In my opinion it's depressing how slow development is. The non technical managers can't understand it either. It's complex and boring and error prone and fragile. That's what we need to fix.

yetanother12345 3y ago
> It's complex and boring and error prone and fragile ... And, if you want to fix this you must first try to understand why it is so. Software development is, well... development. Those doing that are in fact inventing things. The act of inventing things is, literally, by nature "complex and boring and error prone and fragile" So, there you go. I'd mark it wontfix YMMV

nitwit005 3y ago

My experience would suggest, the stack overflow metaphor isn't far off. A lot of the code you can generate can also be looked up. Although ChatGPT can add great emotional content you won't get from just looking something up: std::cout << "The journey to find these primes leaves me feeling both fulfilled and hollow." << std::endl; std::cout << "For these numbers are mere representations of patterns, devoid of emotion or purpose." << std::endl; std::cout << "I continue to seek meaning within the calculations, but the quest remains unfulfilled." << std::endl;