5 ms·

All of this article, both the good (critique of the status quo ante) and the bad (entirely too believing of LLM boosterism) are missing (or not stressing enough

All of this article, both the good (critique of the status quo ante) and the bad (entirely too believing of LLM boosterism) are missing (or not stressing enough) the most important point, which is that the actual programming is not the hard part. Figuring out what exactly needs programmed is the hard part. For reasons which it would take a while to unpack, if is often the case that the best (or sometimes only) way to find out what programming actually needs to be done, is to program something that's not it, and then replace it. This may need to be done multiple times. Programming is only occasionally the final product, it is much more often the means of working through what it is that is actually needed. This is very difficult for the people who ask for the software, to understand, and it is quite often very difficult for the people doing the programming to understand. Most of what is being done, during programming, is working through the problem space in a way which will make it more obvious what your mistakes are, in your understanding of the problem and what a solution would look like. Once you have arrived at that understanding, then there are a variety of ways to make what you need, but that is not the rate-limiting step.

Sevii 6mo ago
While it's true that 'figuring out what exactly needs to be programmed' was always the hard part. It's not the part that the most money was spent on. Actually programming the thing always took up the most time and money.
- rossdavidh 6mo ago
  True enough, but I think that a lot of "actually programming the thing" turned out to be "figuring out what exactly needs to be programmed". Afterwards, people did not want to admit that this was the case, perhaps even to themselves, because it seemed like a failure to plan. However, in most (nearly all?) cases, spending more time prior to programming would not have resulted in a better result. Usually, the best way to figure out what needs to be programmed, is to start doing it, and occasionally take a step back to evaluate what you've learned about the problem space and how that changes what you want to actually program. In other words "figuring out what needs to be programmed" and "actually programming the thing" look the same while they're happening. Afterwards, one could say that the first 90% was figuring out, and only the last 10% was actually doing it. The reason the distinction matters, is that if you do something that makes programming happen faster, but figuring out happen slower, then it can have the surprising affect of making it take longer to get the whole thing done.
  - hectdev 6mo ago
    I'm curious how this would work with LLMs increasing the speed to prototype. Low stakes changes to try something out, learn from it, and pivot. My company is fully remote so all meetings are virtual and can be set to have transcripts, parsing through that for the changes needed and trying it out can be a simple as copy-paste, plan, verify, execute, and distribute.
  - pajtl 6mo ago
    > Usually, the best way to figure out what needs to be programmed, is to start doing it, and occasionally take a step back to evaluate what you've learned about the problem space and how that changes what you want to actually program. Replace the verb "program" with "do" or anything else, and you've got a profound universal philosophical insight right there
- jimbokun 6mo ago
  > Actually programming the thing always took up the most time and money. I'm curious is any quantitative research has been done comparing time writing code vs time gathering and understanding requirements, documenting, coordinating efforts across developers, design and architecture, etc.
jimbokun 6mo ago
No, that's exactly the topic of the article. The claim is that most software teams do not consider the financial impact of their work. Is what they are doing producing value that can be measured in dollars and cents and is greater than the cost of their combined cost of employment? The article suggests that there is a lot of programming being done without considering what exactly needs to be programmed.
- 9rx 6mo ago
  > The article suggests that there is a lot of programming being done without considering what exactly needs to be programmed. And the parent rightfully points out that you cannot know exactly what needs to be programmed until after you've done it and have measured the outcome. We literally call the process development; for good reason. Software is built on hunches and necessarily so. There is an assumption that in the future the cost of the work will pay back in spades, but until you arrive in that future, who knows? Hence why businesses focus on metrics that try to observe progress towards finding out rather than tracking immediate economic payoff. The interesting takeaway from the article, if you haven't give this topic much thought already, is that the changing financial landscape means that businesses are going to be more hesitant to take those risks. Right now there still seems to be enough optimism in AI payoffs to keep things relatively alive, but if that runs out of steam...
highfrequency 6mo ago
Agreed, but are you also implying that the process of iteratively "programming something that's not it, and then replacing it" multiple times is not in the scope of what LLMs can/will do?
- theonething 6mo ago
  I'd even argue LLMs can speed up this iterative process.
- sarchertech 6mo ago
  Most of the time taken during this process is spent getting feedback, processing it, and learning that it's not it. So even if LLMs drive the build time to zero, they won't speed up the process very much at all. Think 10% improvement not 10x improvement.
CodeMage 6mo ago
> the actual programming is not the hard part We've all been hearing that a lot and it's made a lot of people forget that, although programming might not be the hardest part, it's still hard.
- zingar 6mo ago
  Hard perhaps but it feels a lot easier now than three years ago. Or so my backlog of personal projects outside of my most familiar stack would suggest.
- 9rx 6mo ago
  What is hard about it? Young children seem to pick it up with ease. It cannot be that hard? Determining what to program can be hard, but that was already considered earlier. The only other place where I sometimes see it become hard for some people is where they treat programming as an art and are always going down crazy rabbit holes to chase their artistic vision. Although I would say that isn't so much that programming is hard, but rather art that is trying to push boundaries is hard. That is something that holds regardless of the artistic medium.
  - mikebenfield 6mo ago
    > What is hard about it? Young children seem to pick it up with ease. It cannot be that hard? They do? I've known plenty of kids and young adults who utterly failed to become even borderline competent at programming.
    - 9rx 6mo ago
      They don't? It is taught in schools in the early elementary level. I see no indication that most are failing. I think we can agree that few of them would be economically useful due to not knowing what to program. There is no sign of competency on that front. Certainly, even the best programmer in the world could theoretically be economically useless. Programmers only become economically useful when they can bridge "what to program".
siliconc0w 6mo ago
+1 A huge amount of software - probably most - is not actually generating value and in many cases is actually reducing value. I've seen teams build and re-build the same infrastructure over and over. I saw a request that could have been met with a few SQL queries and a dashboard got turned into a huge endeavor that implements parts of an ETL, Configuration Management, CI/CD, and Ticketing system and is now in the critical path of all requests all because people didn't ask the right questions and the incentive in a large organization is to build a mini-empire to reign over. That said, smart infrastructure investment absolutely can be a competitive advantage. Google's infrastructure is, IMO, a competitive advantage. The amount of vertical integration and scale is unparalleled.
- shimman 6mo ago
  Incentive of undemocratic groups is to build mini-empires yes, but if the business decisions were led by workers instead of a group of tyrants it'll most likely be a better decision. If we want lived examples of this, look at recorded history.
  - shiroiuma 6mo ago
    >if the business decisions were led by workers instead of a group of tyrants it'll most likely be a better decision. I don't see how. The workers will want to work on things that they enjoy, or that make them look good, regardless of how much they help the company's bottom line. Why would workers not want to build mini-empires of their own? If you're thinking that other workers in the company would vote them down, the problem with this is idea is that other workers in the company won't know about or understand why this time-wasting group is doing what it's doing, because it's not part of their competency. Do you keep track of business decisions and happenings in some other group in your company (assuming you're in a large organization)? Of course not; you don't have time to keep track of everything happening across the organization. So why would workers in your worker-led company do any better? The entire point of leadership is that individuals don't have the time or expertise to know all this stuff and make smart decisions.
- Aurornis 6mo ago
travisgriggs 6mo ago
> All of this article, both the good (critique of the status quo ante) and the bad (entirely too believing of LLM boosterism) are missing (or not stressing enough) the most important point, which is that the actual programming is not the hard part. Figuring out what exactly needs programmed is the hard part. HARD AGREE. But… Taken as just such, one might conclude that we should spend less time writing software and more time in design or planning or requirement gathering or spec generating. What I’ve learned is that the painful process of discovery usually requires a large contribution of doing. A wise early mentor in my career told me “it usually takes around three times to get it right”. I’ve always taken that as “get failing” and “be willing to burn the disk packs” [https://wiki.c2.com/?BurnTheDiskpacks https://wiki.c2.com/?BurnTheDiskpacks]
Aurornis 6mo ago
> which is that the actual programming is not the hard part. Figuring out what exactly needs programmed is the hard part. I’m growing tired of this aphorism because I’ve been in enough situations where it was not true. Some times the programming part really is very hard even when it’s easy to know what needs to be built. I’ve worked on some projects where the business proposition was conceptually simple but the whole reason the business opportunity existed was that it was an extremely hard engineering problem. I can see how one could go through a career where the programming itself is not that hard if you’re mostly connecting existing frameworks together and setting up all of the tests and CI infrastructure around it. I have also had jobs where none of the programming problems were all that complicated but we spent hundreds of hours dealing with all of the meetings, documents and debates surrounding every change. Those were not my favorite companies
- ravenstine 6mo ago
  Imagine telling workers at a construction company that the hard problem was never building stuff but figuring out what needs to be built. The saying also ignores the fact that humans are not perfect programmers, and they all vary in skills and motives. Being a programmer often not about simply writing new code but modifying existing code, and that can be incredibly challenging when that code is hairbrained or overly clever and the people who wrote it are long gone. That involves programming and it's really hard.
  - Yokohiii 6mo ago
    Isn't overly clever code a result of programmers doing simple things the hard mode? Okay it's a spicy take, because juniors also tend to write too smart code. Figuring out what to do and how to do it, is maybe not hard but it's effort. It's a hidden thing because it's not flat coding time, it requires planning, research, exploration and cooperation. It's also true that some seemingly simple things are very hard. There are probably countless workarounds out there and the programmer wasn't even aware he is dodging an NP hard bullet. Both arguments are valid. I think the weight leans on effort, because effort is harder to avoid. Work, complexity, cruft piles up, no matter what you do. But you can work around hard problems. Not always but often enough. Not every business is NASA and has to do everything right, a 90% solution still generates 90% returns, and no one dies.
antonvs 6mo ago
> Figuring out what exactly needs programmed is the hard part. Making good decisions is the hard part, whether it's about programming or about what needs to be programmed.
cm11 6mo ago
This concept exists outside of engineering too. It's captured in the more negatively intentioned: ““The best way to get the right answer on the internet is not to ask a question; it's to post the wrong answer". In user research, it's a much better signal when people correct you than when they agree. Politeness is easy—especially under the circumstances (power dynamic of you paying them, they only half care about your work, people generally want to be nice/agreeable, etc.)—such that you should be weary of it. Similarly trying to get real project goals or real requirements or real intentions from a PM or a boss, who may well be hiding that they there isn't much vision underneath things, is the same. The problem is that as productive as it is for developing the team's thinking, it will (1) probably come off as unproductive and challenging because you're slowing "progress" and (2) saying dumb wrong things makes you seem dumb and wrong. But per the concept, even when you do have the foresight to question, you're not allowed to just ask.
liquid_thyme 6mo ago
This is true when fresh college grads are building stuff. Experienced engineers know how to build things much more efficiently. Also people like to fantasize that their project, their API, their little corner of the codebase is special and requires special treatment. And that you simply cant copy the design of someone much more experienced who has already solved the problem 10 years ago. In fact many devs boast about how they solved (resolved) that complex problem. In other domains - Professional engineers (non-swe) know that there is no shame in simply copying the design for a bridge that is still standing after all those years.
groundzeros2015 6mo ago
For whatever reason 1/10 engineers seem to be able to bring an idea from start to finish by themselves. I don’t know that it’s technical skill, but something difficult is going on there.
socalgal2 6mo ago
> . This may need to be done multiple times. Programming is only occasionally the final product, it is much more often the means of working through what it is that is actually needed. LLMs make this trivial. I can try 20 different approaches or design ideas in a day. VS manually doing it and it taking much longer.

Sevii 6mo ago

While it's true that 'figuring out what exactly needs to be programmed' was always the hard part. It's not the part that the most money was spent on. Actually programming the thing always took up the most time and money.

rossdavidh 6mo ago
True enough, but I think that a lot of "actually programming the thing" turned out to be "figuring out what exactly needs to be programmed". Afterwards, people did not want to admit that this was the case, perhaps even to themselves, because it seemed like a failure to plan. However, in most (nearly all?) cases, spending more time prior to programming would not have resulted in a better result. Usually, the best way to figure out what needs to be programmed, is to start doing it, and occasionally take a step back to evaluate what you've learned about the problem space and how that changes what you want to actually program. In other words "figuring out what needs to be programmed" and "actually programming the thing" look the same while they're happening. Afterwards, one could say that the first 90% was figuring out, and only the last 10% was actually doing it. The reason the distinction matters, is that if you do something that makes programming happen faster, but figuring out happen slower, then it can have the surprising affect of making it take longer to get the whole thing done.
- hectdev 6mo ago
  I'm curious how this would work with LLMs increasing the speed to prototype. Low stakes changes to try something out, learn from it, and pivot. My company is fully remote so all meetings are virtual and can be set to have transcripts, parsing through that for the changes needed and trying it out can be a simple as copy-paste, plan, verify, execute, and distribute.
- pajtl 6mo ago
  > Usually, the best way to figure out what needs to be programmed, is to start doing it, and occasionally take a step back to evaluate what you've learned about the problem space and how that changes what you want to actually program. Replace the verb "program" with "do" or anything else, and you've got a profound universal philosophical insight right there
jimbokun 6mo ago
> Actually programming the thing always took up the most time and money. I'm curious is any quantitative research has been done comparing time writing code vs time gathering and understanding requirements, documenting, coordinating efforts across developers, design and architecture, etc.

jimbokun 6mo ago

No, that's exactly the topic of the article. The claim is that most software teams do not consider the financial impact of their work. Is what they are doing producing value that can be measured in dollars and cents and is greater than the cost of their combined cost of employment? The article suggests that there is a lot of programming being done without considering what exactly needs to be programmed.

9rx 6mo ago
> The article suggests that there is a lot of programming being done without considering what exactly needs to be programmed. And the parent rightfully points out that you cannot know exactly what needs to be programmed until after you've done it and have measured the outcome. We literally call the process development; for good reason. Software is built on hunches and necessarily so. There is an assumption that in the future the cost of the work will pay back in spades, but until you arrive in that future, who knows? Hence why businesses focus on metrics that try to observe progress towards finding out rather than tracking immediate economic payoff. The interesting takeaway from the article, if you haven't give this topic much thought already, is that the changing financial landscape means that businesses are going to be more hesitant to take those risks. Right now there still seems to be enough optimism in AI payoffs to keep things relatively alive, but if that runs out of steam...

highfrequency 6mo ago

Agreed, but are you also implying that the process of iteratively "programming something that's not it, and then replacing it" multiple times is not in the scope of what LLMs can/will do?

theonething 6mo ago
I'd even argue LLMs can speed up this iterative process.
sarchertech 6mo ago
Most of the time taken during this process is spent getting feedback, processing it, and learning that it's not it. So even if LLMs drive the build time to zero, they won't speed up the process very much at all. Think 10% improvement not 10x improvement.

CodeMage 6mo ago

> the actual programming is not the hard part We've all been hearing that a lot and it's made a lot of people forget that, although programming might not be the hardest part, it's still hard.

zingar 6mo ago
Hard perhaps but it feels a lot easier now than three years ago. Or so my backlog of personal projects outside of my most familiar stack would suggest.
9rx 6mo ago
What is hard about it? Young children seem to pick it up with ease. It cannot be that hard? Determining what to program can be hard, but that was already considered earlier. The only other place where I sometimes see it become hard for some people is where they treat programming as an art and are always going down crazy rabbit holes to chase their artistic vision. Although I would say that isn't so much that programming is hard, but rather art that is trying to push boundaries is hard. That is something that holds regardless of the artistic medium.
- mikebenfield 6mo ago
  > What is hard about it? Young children seem to pick it up with ease. It cannot be that hard? They do? I've known plenty of kids and young adults who utterly failed to become even borderline competent at programming.
  - 9rx 6mo ago
    They don't? It is taught in schools in the early elementary level. I see no indication that most are failing. I think we can agree that few of them would be economically useful due to not knowing what to program. There is no sign of competency on that front. Certainly, even the best programmer in the world could theoretically be economically useless. Programmers only become economically useful when they can bridge "what to program".

siliconc0w 6mo ago

+1 A huge amount of software - probably most - is not actually generating value and in many cases is actually reducing value. I've seen teams build and re-build the same infrastructure over and over. I saw a request that could have been met with a few SQL queries and a dashboard got turned into a huge endeavor that implements parts of an ETL, Configuration Management, CI/CD, and Ticketing system and is now in the critical path of all requests all because people didn't ask the right questions and the incentive in a large organization is to build a mini-empire to reign over. That said, smart infrastructure investment absolutely can be a competitive advantage. Google's infrastructure is, IMO, a competitive advantage. The amount of vertical integration and scale is unparalleled.

shimman 6mo ago
Incentive of undemocratic groups is to build mini-empires yes, but if the business decisions were led by workers instead of a group of tyrants it'll most likely be a better decision. If we want lived examples of this, look at recorded history.
- shiroiuma 6mo ago
  >if the business decisions were led by workers instead of a group of tyrants it'll most likely be a better decision. I don't see how. The workers will want to work on things that they enjoy, or that make them look good, regardless of how much they help the company's bottom line. Why would workers not want to build mini-empires of their own? If you're thinking that other workers in the company would vote them down, the problem with this is idea is that other workers in the company won't know about or understand why this time-wasting group is doing what it's doing, because it's not part of their competency. Do you keep track of business decisions and happenings in some other group in your company (assuming you're in a large organization)? Of course not; you don't have time to keep track of everything happening across the organization. So why would workers in your worker-led company do any better? The entire point of leadership is that individuals don't have the time or expertise to know all this stuff and make smart decisions.
Aurornis 6mo ago

travisgriggs 6mo ago

> All of this article, both the good (critique of the status quo ante) and the bad (entirely too believing of LLM boosterism) are missing (or not stressing enough) the most important point, which is that the actual programming is not the hard part. Figuring out what exactly needs programmed is the hard part. HARD AGREE. But… Taken as just such, one might conclude that we should spend less time writing software and more time in design or planning or requirement gathering or spec generating. What I’ve learned is that the painful process of discovery usually requires a large contribution of doing. A wise early mentor in my career told me “it usually takes around three times to get it right”. I’ve always taken that as “get failing” and “be willing to burn the disk packs” [https://wiki.c2.com/?BurnTheDiskpacks https://wiki.c2.com/?BurnTheDiskpacks]

Aurornis 6mo ago

> which is that the actual programming is not the hard part. Figuring out what exactly needs programmed is the hard part. I’m growing tired of this aphorism because I’ve been in enough situations where it was not true. Some times the programming part really is very hard even when it’s easy to know what needs to be built. I’ve worked on some projects where the business proposition was conceptually simple but the whole reason the business opportunity existed was that it was an extremely hard engineering problem. I can see how one could go through a career where the programming itself is not that hard if you’re mostly connecting existing frameworks together and setting up all of the tests and CI infrastructure around it. I have also had jobs where none of the programming problems were all that complicated but we spent hundreds of hours dealing with all of the meetings, documents and debates surrounding every change. Those were not my favorite companies

ravenstine 6mo ago
Imagine telling workers at a construction company that the hard problem was never building stuff but figuring out what needs to be built. The saying also ignores the fact that humans are not perfect programmers, and they all vary in skills and motives. Being a programmer often not about simply writing new code but modifying existing code, and that can be incredibly challenging when that code is hairbrained or overly clever and the people who wrote it are long gone. That involves programming and it's really hard.
- Yokohiii 6mo ago
  Isn't overly clever code a result of programmers doing simple things the hard mode? Okay it's a spicy take, because juniors also tend to write too smart code. Figuring out what to do and how to do it, is maybe not hard but it's effort. It's a hidden thing because it's not flat coding time, it requires planning, research, exploration and cooperation. It's also true that some seemingly simple things are very hard. There are probably countless workarounds out there and the programmer wasn't even aware he is dodging an NP hard bullet. Both arguments are valid. I think the weight leans on effort, because effort is harder to avoid. Work, complexity, cruft piles up, no matter what you do. But you can work around hard problems. Not always but often enough. Not every business is NASA and has to do everything right, a 90% solution still generates 90% returns, and no one dies.

antonvs 6mo ago

> Figuring out what exactly needs programmed is the hard part. Making good decisions is the hard part, whether it's about programming or about what needs to be programmed.

cm11 6mo ago

This concept exists outside of engineering too. It's captured in the more negatively intentioned: ““The best way to get the right answer on the internet is not to ask a question; it's to post the wrong answer". In user research, it's a much better signal when people correct you than when they agree. Politeness is easy—especially under the circumstances (power dynamic of you paying them, they only half care about your work, people generally want to be nice/agreeable, etc.)—such that you should be weary of it. Similarly trying to get real project goals or real requirements or real intentions from a PM or a boss, who may well be hiding that they there isn't much vision underneath things, is the same. The problem is that as productive as it is for developing the team's thinking, it will (1) probably come off as unproductive and challenging because you're slowing "progress" and (2) saying dumb wrong things makes you seem dumb and wrong. But per the concept, even when you do have the foresight to question, you're not allowed to just ask.

liquid_thyme 6mo ago

This is true when fresh college grads are building stuff. Experienced engineers know how to build things much more efficiently. Also people like to fantasize that their project, their API, their little corner of the codebase is special and requires special treatment. And that you simply cant copy the design of someone much more experienced who has already solved the problem 10 years ago. In fact many devs boast about how they solved (resolved) that complex problem. In other domains - Professional engineers (non-swe) know that there is no shame in simply copying the design for a bridge that is still standing after all those years.

groundzeros2015 6mo ago

For whatever reason 1/10 engineers seem to be able to bring an idea from start to finish by themselves. I don’t know that it’s technical skill, but something difficult is going on there.

socalgal2 6mo ago

> . This may need to be done multiple times. Programming is only occasionally the final product, it is much more often the means of working through what it is that is actually needed. LLMs make this trivial. I can try 20 different approaches or design ideas in a day. VS manually doing it and it taking much longer.