4 ms·

LLMs can't be strategic because they do not understand the big picture -- that the real work of good software is balancing a hundred different constraints in a

by gtowey 8mo ago

LLMs can't be strategic because they do not understand the big picture -- that the real work of good software is balancing a hundred different constraints in a way that produces the optimal result for the humans who use it. It's not all that different from the state of big corp software today! Large organizations with layers of management tend to lose all abiliy to keep a consistent strategy. They tend to go all in on a single dimension such as ROI for the next quarter, but it misses the bigger picture. Good software is about creating longer term value and takes consistent skill & vision to execute. Those software engineers who focus on this big picture thinking are going to be more valuable than ever.

pixl97 8mo ago
>LLMs can't be strategic because they do not understand the big picture While I do tend to believe you, what evidence based data do you have to prove this is true?
- gtowey 8mo ago
  > While I do tend to believe you, what evidence based data do you have to prove this is true? IMO the onus is to prove that they can be strategic. Otherwise you're asking me to prove a negative.
  - pixl97 8mo ago
    Saying the tiger has to prove it can eat you is not a great strategy to survive a tiger attack.
    - bigstrat2003 8mo ago
      Well so far the tiger faceplants in an embarrassing fashion every time it tries to eat someone. So I'm not really worried about that.
      - pixl97 8mo ago
        Gary Marcus: "LLMs will never be able to a..... wait, what do you mean they can already do that?"
  - vonneumannstan 8mo ago
    Task Time horizons are improving exponentially with doubling times around 4 months per METR. At what timescale would you accept that they "can be strategic"? Theres little reason to think they wont be at multi week or month time horizons very soon. Do you need to be strategic to complete multi month tasks?
    - gtowey 8mo ago
      Can an LLM give you an upfront estimate that a task will take multiple months? Can it decide intelligently what it would have to change if you said "do what you can to have it ready in half the time?"
      - vonneumannstan 8mo ago
        >Can an LLM give you an upfront estimate that a task will take multiple months? >Can it decide intelligently what it would have to change if you said "do what you can to have it ready in half the time?" Do you think ChatGPT 5.2 Pro can't estimate how long a task might take? Do you think that estimate would necessarily be worse than the estimates, which are notoriously poor, coming from human engineers? But you can still answer my question. When an LLM can complete a task that takes a person N months or years, is it capable of being strategic?
        manxiemanx 7mo ago
        Multiple people have already answered your question in this thread. An LLM can’t be strategic because that’s not a capability of the technology itself
- Terr_ 8mo ago
  Prompt-injection in all its forms. If the hyper-mad-libs machine doesn't reliably "understand" and model the difference between internal and external words, how can we trust them to model fancier stuff?
  - bigstrat2003 8mo ago
    We can't even trust LLMs to get basic logic right, or even the language syntax sometimes. They reliably generate code worse than a human would write, and have zero reasoning ability. Anyone who thinks they can model something complicated is either uncritically absorbing hype or has a financial stake in convincing people of the hype.
  - manxiemanx 7mo ago
    That’s no problem. Just tell it to make sure the code it generates has no security vulnerabilities! That’ll take care of the issue
- manxiemanx 7mo ago
  This is fundamental to the way LLMs work. If you don’t know this, you should be reading up on that instead of asking for evidence based data on things that are fundamental to the technology we’re discussing
bee_rider 8mo ago
Why can’t LLMs understand the big picture? I mean, a lot of companies have most of their information available in a digital form at this point, so it could be consumed by the LLM. I think if anything, we have a better chance in the little picture: you can go to lunch with your engineering coworkers or talk to somebody on the factory floor and get insights that will never touch the computers. Giant systems of constraints, optimizing many-dimensional user metrics: eventually we will hit the wall where it is easier to add RAM to machines than humans.
- butILoveLife 8mo ago
  I basically just posted the same response. I generally agree with everything you said. Only thing to add, maybe we have the most senior of seniors verifying the decisions of AI.
  - bee_rider 8mo ago
    Most senior could make sense (although I’d like to see a collection of independent guilds coordinated by an LLM “CEO” just to see how it could work—might not be good enough yet, but it’d be an interesting experiment). Ultimately, I suspect “AI” (although, maybe much more advanced than current LLMs) will be able to do just about any information based task. But in the end only humans can actually be responsible/accountable.
- troupo 8mo ago
  > Why can’t LLMs understand the big picture? Because LLMs don't understand things to begin with. Because LLMs only have access to aource code and whatever .md files you've given them. Because they have biases in their training data that overfit them on certain solutions. Because LLMs have a tiny context window. Because LLMs largely suck at UI/UX/design especially when they don't have referense images. Because...
  - gtowey 8mo ago
    Yeah, it's strange to me that the default assumption is that current LLMs are already human-level AGI.
antonvs 8mo ago
> LLMs can't be strategic because they do not understand the big picture -- that the real work of good software is balancing a hundred different constraints in a way that produces the optimal result for the humans who use it. There’s good reason to think that they could understand the big picture just fine, even today, except that they’re currently severely constrained by what we choose, or have time, to tell them. They can already easily give a much more comprehensive survey of suitable options for solving a given problem than most humans can. If they had more direct access to the information we have access to, that we currently grudgingly dole out to them in dribs and drabs, they would be much more capable.
butILoveLife 8mo ago
>Good software is about creating longer term value and takes consistent skill & vision to execute. >Those software engineers who focus on this big picture thinking are going to be more valuable than ever. Not to rain on our hopes, but AI can give us some options and we can pick the best. I think this eliminates all middle level positions. Newbies are low cost and make decisions that are low stakes. The most senior or seniors can make 30 major decisions per day when AI lays them out. I own a software shop and my hires have been: Interns and people with the specific skill of my industry(Mechanical engineers). 2 years ago, I hired experienced programmers. Now I turn my mechanical engineers into programmers.
- gtowey 8mo ago
  > Not to rain on our hopes, but AI can give us some options and we can pick the best. But that's kind of my point. A bunch of decisions like that tend to end up with a "random walk" effect. It's a bunch of tactical choices which don't add up to something strategic. It could be, but it takes the human in the loop to hold onto that overall strategy.
- AlotOfReading 8mo ago
  Not to rain on our hopes, but AI can give us some options and we can pick the best. a.k.a. greedy algorithms, a subject those of us on HN should be well-acquainted with. You can watch the horizon effect frequently play out in corporate decisionmaking.
- zaphar 8mo ago
  So what you are a saying is that you removed the people who can make the decisions that keep your software maintainable and kept the people who will slowly over time cause your software to become less maintainable? I'm not sure that tradeoff is a a good one.
  - butILoveLife 8mo ago
    This might have been true pre-agent AI programming, but honestly the code seems better than ever. It finds edge cases better than me. I know... I know buddy. The world changed and I don't know if I'm going to have a job.
sp1nningaway 8mo ago
Yes! "Does an AI know how to do that? Does a coding assistant know that an app is really a giant collection of details?" There are just so many small decisions that add up to a consistent vision for a piece of software. It doesn't seem like LLMs are going to be able to meaningfully contribute to that in the near future. I tried vibecoding my own workout tracker, but there were so many small details to think through that it was frustrating. I gave up and found an app that is clearly made by a team of experienced, thoughtful people and AI can't replicate the sheer thoughtfulness of every decision that was made to create this app. The inputs for reps/sets, algorithms for adjusting effort on the fly, an exercise library with clear videos and explanations; there's just no way to replicate that without people who have been trainers and sport scientists for decades. LLMs can help increase the speed that these details turn in to something tangible, but you definitely can't "skip all that crap and just jump to the end and get on with it."
- Throaway1985123 8mo ago
  Right, it's people who can't code (and cant afford to hire a coder) that can use LLM's to replace coders, but oyu still need to sit through it and iterate. More so than with a human being, because of the dumb mistakes LLMs still make, so you need a patient and consistent vision.
h3lp 8mo ago
Greg mentions discipline and vision as determinants of successful software, which is correct but I think he misses another aspect of vision: the ability to attract and crystallize a community around their project. Arguably, most successful softwares thrive in the long term because they have a team of people that inspire each other, fill in with complementary talents, and provide continuity.

by gtowey 8mo ago

pixl97 8mo ago

>LLMs can't be strategic because they do not understand the big picture While I do tend to believe you, what evidence based data do you have to prove this is true?

gtowey 8mo ago
> While I do tend to believe you, what evidence based data do you have to prove this is true? IMO the onus is to prove that they can be strategic. Otherwise you're asking me to prove a negative.
- pixl97 8mo ago
  Saying the tiger has to prove it can eat you is not a great strategy to survive a tiger attack.
  - bigstrat2003 8mo ago
    Well so far the tiger faceplants in an embarrassing fashion every time it tries to eat someone. So I'm not really worried about that.
    - pixl97 8mo ago
      Gary Marcus: "LLMs will never be able to a..... wait, what do you mean they can already do that?"
- vonneumannstan 8mo ago
  Task Time horizons are improving exponentially with doubling times around 4 months per METR. At what timescale would you accept that they "can be strategic"? Theres little reason to think they wont be at multi week or month time horizons very soon. Do you need to be strategic to complete multi month tasks?
  - gtowey 8mo ago
    Can an LLM give you an upfront estimate that a task will take multiple months? Can it decide intelligently what it would have to change if you said "do what you can to have it ready in half the time?"
    - vonneumannstan 8mo ago
      >Can an LLM give you an upfront estimate that a task will take multiple months? >Can it decide intelligently what it would have to change if you said "do what you can to have it ready in half the time?" Do you think ChatGPT 5.2 Pro can't estimate how long a task might take? Do you think that estimate would necessarily be worse than the estimates, which are notoriously poor, coming from human engineers? But you can still answer my question. When an LLM can complete a task that takes a person N months or years, is it capable of being strategic?
      - manxiemanx 7mo ago
        Multiple people have already answered your question in this thread. An LLM can’t be strategic because that’s not a capability of the technology itself
Terr_ 8mo ago
Prompt-injection in all its forms. If the hyper-mad-libs machine doesn't reliably "understand" and model the difference between internal and external words, how can we trust them to model fancier stuff?
- bigstrat2003 8mo ago
  We can't even trust LLMs to get basic logic right, or even the language syntax sometimes. They reliably generate code worse than a human would write, and have zero reasoning ability. Anyone who thinks they can model something complicated is either uncritically absorbing hype or has a financial stake in convincing people of the hype.
- manxiemanx 7mo ago
  That’s no problem. Just tell it to make sure the code it generates has no security vulnerabilities! That’ll take care of the issue
manxiemanx 7mo ago
This is fundamental to the way LLMs work. If you don’t know this, you should be reading up on that instead of asking for evidence based data on things that are fundamental to the technology we’re discussing

bee_rider 8mo ago

Why can’t LLMs understand the big picture? I mean, a lot of companies have most of their information available in a digital form at this point, so it could be consumed by the LLM. I think if anything, we have a better chance in the little picture: you can go to lunch with your engineering coworkers or talk to somebody on the factory floor and get insights that will never touch the computers. Giant systems of constraints, optimizing many-dimensional user metrics: eventually we will hit the wall where it is easier to add RAM to machines than humans.

butILoveLife 8mo ago
I basically just posted the same response. I generally agree with everything you said. Only thing to add, maybe we have the most senior of seniors verifying the decisions of AI.
- bee_rider 8mo ago
  Most senior could make sense (although I’d like to see a collection of independent guilds coordinated by an LLM “CEO” just to see how it could work—might not be good enough yet, but it’d be an interesting experiment). Ultimately, I suspect “AI” (although, maybe much more advanced than current LLMs) will be able to do just about any information based task. But in the end only humans can actually be responsible/accountable.
troupo 8mo ago
> Why can’t LLMs understand the big picture? Because LLMs don't understand things to begin with. Because LLMs only have access to aource code and whatever .md files you've given them. Because they have biases in their training data that overfit them on certain solutions. Because LLMs have a tiny context window. Because LLMs largely suck at UI/UX/design especially when they don't have referense images. Because...
- gtowey 8mo ago
  Yeah, it's strange to me that the default assumption is that current LLMs are already human-level AGI.

antonvs 8mo ago

> LLMs can't be strategic because they do not understand the big picture -- that the real work of good software is balancing a hundred different constraints in a way that produces the optimal result for the humans who use it. There’s good reason to think that they could understand the big picture just fine, even today, except that they’re currently severely constrained by what we choose, or have time, to tell them. They can already easily give a much more comprehensive survey of suitable options for solving a given problem than most humans can. If they had more direct access to the information we have access to, that we currently grudgingly dole out to them in dribs and drabs, they would be much more capable.

butILoveLife 8mo ago

>Good software is about creating longer term value and takes consistent skill & vision to execute. >Those software engineers who focus on this big picture thinking are going to be more valuable than ever. Not to rain on our hopes, but AI can give us some options and we can pick the best. I think this eliminates all middle level positions. Newbies are low cost and make decisions that are low stakes. The most senior or seniors can make 30 major decisions per day when AI lays them out. I own a software shop and my hires have been: Interns and people with the specific skill of my industry(Mechanical engineers). 2 years ago, I hired experienced programmers. Now I turn my mechanical engineers into programmers.

gtowey 8mo ago
> Not to rain on our hopes, but AI can give us some options and we can pick the best. But that's kind of my point. A bunch of decisions like that tend to end up with a "random walk" effect. It's a bunch of tactical choices which don't add up to something strategic. It could be, but it takes the human in the loop to hold onto that overall strategy.
AlotOfReading 8mo ago
Not to rain on our hopes, but AI can give us some options and we can pick the best. a.k.a. greedy algorithms, a subject those of us on HN should be well-acquainted with. You can watch the horizon effect frequently play out in corporate decisionmaking.
zaphar 8mo ago
So what you are a saying is that you removed the people who can make the decisions that keep your software maintainable and kept the people who will slowly over time cause your software to become less maintainable? I'm not sure that tradeoff is a a good one.
- butILoveLife 8mo ago
  This might have been true pre-agent AI programming, but honestly the code seems better than ever. It finds edge cases better than me. I know... I know buddy. The world changed and I don't know if I'm going to have a job.

sp1nningaway 8mo ago

Yes! "Does an AI know how to do that? Does a coding assistant know that an app is really a giant collection of details?" There are just so many small decisions that add up to a consistent vision for a piece of software. It doesn't seem like LLMs are going to be able to meaningfully contribute to that in the near future. I tried vibecoding my own workout tracker, but there were so many small details to think through that it was frustrating. I gave up and found an app that is clearly made by a team of experienced, thoughtful people and AI can't replicate the sheer thoughtfulness of every decision that was made to create this app. The inputs for reps/sets, algorithms for adjusting effort on the fly, an exercise library with clear videos and explanations; there's just no way to replicate that without people who have been trainers and sport scientists for decades. LLMs can help increase the speed that these details turn in to something tangible, but you definitely can't "skip all that crap and just jump to the end and get on with it."

Throaway1985123 8mo ago
Right, it's people who can't code (and cant afford to hire a coder) that can use LLM's to replace coders, but oyu still need to sit through it and iterate. More so than with a human being, because of the dumb mistakes LLMs still make, so you need a patient and consistent vision.

h3lp 8mo ago

Greg mentions discipline and vision as determinants of successful software, which is correct but I think he misses another aspect of vision: the ability to attract and crystallize a community around their project. Arguably, most successful softwares thrive in the long term because they have a team of people that inspire each other, fill in with complementary talents, and provide continuity.