5 ms·

This blog post illustrates the importance of NEVER outsourcing your critical thinking or outsourcing decision-making to an LLM. And to never take shortcuts with

This blog post illustrates the importance of NEVER outsourcing your critical thinking or outsourcing decision-making to an LLM. And to never take shortcuts with learning. Learning is hard, but learning things properly allows you to understand what is happening and allows you to ask the right questions about whether the changes an agent wants to make are the changes that will best serve the goals of the project (without creating an unwieldy level of tech debt in the future). Personally speaking (and I'd love to hear others' takes on this): when using an LLM for work-related and development tasks, I never use the "full-auto" mode, and I never manually approve of something that I don't understand. When I don't understand something an agent wants to do, I go on a side-quest to learn more about said thing and to educate myself first. This takes extra time, but I feel that it's the right thing to do, so I can at least approve/deny/redirect from a more informed position, rather than flying blind and hoping for the best. In addition to what the author discusses, I think skill-atrophy, stagnation due to complacency (i.e.: "why grow and learn if an agent can do it" mindset), and cognitive laziness are additional risks that come with overrelying on LLMs. Humans were meant to think. LLMs are a tool.

bluefirebrand 2mo ago
> In addition to what the author discusses, I think skill-atrophy, stagnation due to complacency (i.e.: "why grow and learn if an agent can do it" mindset), and cognitive laziness are additional risks that come with overrelying on LLMs. Humans were meant to think. LLMs are a tool. I know it's a cliche but the old IBM adage holds very true today Machines should work, people should think Maybe that's part of the problem I have with LLMs writing code. It would be nice if they just did the work, but there's a lot of underlying thinking and decision making with software work that is being offloaded any time you have the LLM do it It seems like those decisions are being handwaved off as "not important" nowadays. Just let the LLM make those choices! But that is not sitting right with me for whatever reason. Something I should think about I guess
- win311fwg 2mo ago
  The decisions are important, but the understanding is that you define them as constraints. Which is what a segment of the software community, the upper class if you will, has always believed you should do, even before LLMs were a thing. The LLM simply becomes a constraint solver: machines do the work, people do the thinking. This is the challenge of the middle class developer. They aren't accustomed to working with constraint thinking and they're past the junior stage where one is expected to still be shaping their thinking so they are struggling to find a fit.
  - bluefirebrand 2mo ago
    > They aren't accustomed to working with constraint thinking That's one way to frame it I guess I would frame it as "they have been abused by AGILE and impatient PMs into never actually taking time to do constraint based thinking or design" It is impossible to think about constraints, or really do any proper engineering, if you are constantly "sprinting" Edit: I also don't personally like engineering based on constraints. It feels like constructing a building based on where walls aren't allowed to go. I get that defining constraints and letting the software go is where a lot of useful emergent behavior lives, but I struggle a lot to think that way. Maybe I'm not wired right for software in the end
    - win311fwg 2mo ago
      > "they have been abused by AGILE [...]" That's ironic given that some of the Agile Manifesto signatories are some of the biggest proponents of constraint-based development. While the Principles behind the Agile Manifesto is not prescriptive, I am not certain you could, in practice, even satisfy many of the principles without constraint-based development. > I also don't personally like engineering based on constraints. It feels like constructing a building based on where walls aren't allowed to go. Engineering is all about constraints. That you then shift to talking about constructing a building is curious as it would be atypical for an engineer to work on constructing a building. The engineer's role in building constriction is typically only in defining the constraints. Construction crews, made up of an entirely different group of people, then construct the building within the constraints set. "Constructing a building based on where walls aren't allowed to go" is essentially how buildings are usually constructed. It is an interesting parallel as the LLM is somewhat like the construction crew, which is something software hasn't really had before in any kind of big way. Engineer gets thrown around pretty loosely in the world of software, but elsewhere, especially in places where failure can cause serious harm, someone who constructs a building by applying judgment, experience, and craftsmanship as they go would be considered an artisan, craftsman, or something to that effect rather than an engineer. That seems closer to the world you like to live in. There is nothing wrong with being a craftsman, of course, but the economic fit is becoming less clear as the industry matures. Which is something that seems par for the course. The early days of building construction was also dominated by craftsmen but as it matured engineers became dominant.
gnarlouse 2mo ago
> NEVER with critical thought* fixed that for you, only sith's (and unreasonable people) deal in absolutes; you almost did!
Valakas_ 2mo ago
"This blog post illustrates the importance of NEVER outsourcing your critical thinking or outsourcing decision-making to an LLM." This is a sentence that if written just 5 years ago would result in you being seen as crazy. "LLM capable of any sort of critical thinking? Haha. Not in 50 years." It never ceases to amaze me how people are stuck in the here in the now and just accept the new reality as if it had always been this way, and will continue being this way in the foreseeable future. This is the worst LLMs will ever be. Not outsourcing critical thinking to LLM will be seen as a liability in the not so distant future. Just a year or two ago almost no one even took LLM seriously for coding. Now the status quo has moved for also critical thinking, and we hear comments like this. LLM's chances of making a mistake will be orders of magnitude lowers than humans. Slowly learn to let the wheel go. AI will be much better at holding and controlling it. "We should NEVER outsource number crunching to calculators. They are not 100% reliable and we can make mistakes when inputting the data." Someone in 1920's probably. I know it's a metaphor, and like all metaphors can't be compared 1-1. But it's to make a point how it illustrates my argument above.
- palmotea 2mo ago
  > This is the worst LLMs will ever be. Not outsourcing critical thinking to LLM will be seen as a liability in the not so distant future. Just a year or two ago almost no one even took LLM seriously for coding. Now the status quo has moved for also critical thinking, and we hear comments like this. LLM's chances of making a mistake will be orders of magnitude lowers than humans. Slowly learn to let the wheel go. AI will be much better at holding and controlling it. If that's true, you're fucked. Because then what good are you for? How long do you think they'll pay you to be useless meat-in-the-middle? Not outsourcing your critical thinking means trying to preserve a scrap of value-add for having you around. > "We should NEVER outsource number crunching to calculators. They are not 100% reliable and we can make mistakes when inputting the data." Someone in 1920's probably. I know it's a metaphor, and like all metaphors can't be compared 1-1. But it's to make a point how it illustrates my argument above. That's a dumb analogy. Automating one activity cannot be extrapolated justify automating all the activities, including some of the most human ones.
  - 2mo ago

bluefirebrand 2mo ago

> In addition to what the author discusses, I think skill-atrophy, stagnation due to complacency (i.e.: "why grow and learn if an agent can do it" mindset), and cognitive laziness are additional risks that come with overrelying on LLMs. Humans were meant to think. LLMs are a tool. I know it's a cliche but the old IBM adage holds very true today Machines should work, people should think Maybe that's part of the problem I have with LLMs writing code. It would be nice if they just did the work, but there's a lot of underlying thinking and decision making with software work that is being offloaded any time you have the LLM do it It seems like those decisions are being handwaved off as "not important" nowadays. Just let the LLM make those choices! But that is not sitting right with me for whatever reason. Something I should think about I guess

win311fwg 2mo ago
The decisions are important, but the understanding is that you define them as constraints. Which is what a segment of the software community, the upper class if you will, has always believed you should do, even before LLMs were a thing. The LLM simply becomes a constraint solver: machines do the work, people do the thinking. This is the challenge of the middle class developer. They aren't accustomed to working with constraint thinking and they're past the junior stage where one is expected to still be shaping their thinking so they are struggling to find a fit.
- bluefirebrand 2mo ago
  > They aren't accustomed to working with constraint thinking That's one way to frame it I guess I would frame it as "they have been abused by AGILE and impatient PMs into never actually taking time to do constraint based thinking or design" It is impossible to think about constraints, or really do any proper engineering, if you are constantly "sprinting" Edit: I also don't personally like engineering based on constraints. It feels like constructing a building based on where walls aren't allowed to go. I get that defining constraints and letting the software go is where a lot of useful emergent behavior lives, but I struggle a lot to think that way. Maybe I'm not wired right for software in the end
  - win311fwg 2mo ago
    > "they have been abused by AGILE [...]" That's ironic given that some of the Agile Manifesto signatories are some of the biggest proponents of constraint-based development. While the Principles behind the Agile Manifesto is not prescriptive, I am not certain you could, in practice, even satisfy many of the principles without constraint-based development. > I also don't personally like engineering based on constraints. It feels like constructing a building based on where walls aren't allowed to go. Engineering is all about constraints. That you then shift to talking about constructing a building is curious as it would be atypical for an engineer to work on constructing a building. The engineer's role in building constriction is typically only in defining the constraints. Construction crews, made up of an entirely different group of people, then construct the building within the constraints set. "Constructing a building based on where walls aren't allowed to go" is essentially how buildings are usually constructed. It is an interesting parallel as the LLM is somewhat like the construction crew, which is something software hasn't really had before in any kind of big way. Engineer gets thrown around pretty loosely in the world of software, but elsewhere, especially in places where failure can cause serious harm, someone who constructs a building by applying judgment, experience, and craftsmanship as they go would be considered an artisan, craftsman, or something to that effect rather than an engineer. That seems closer to the world you like to live in. There is nothing wrong with being a craftsman, of course, but the economic fit is becoming less clear as the industry matures. Which is something that seems par for the course. The early days of building construction was also dominated by craftsmen but as it matured engineers became dominant.

gnarlouse 2mo ago

> NEVER with critical thought* fixed that for you, only sith's (and unreasonable people) deal in absolutes; you almost did!

Valakas_ 2mo ago

"This blog post illustrates the importance of NEVER outsourcing your critical thinking or outsourcing decision-making to an LLM." This is a sentence that if written just 5 years ago would result in you being seen as crazy. "LLM capable of any sort of critical thinking? Haha. Not in 50 years." It never ceases to amaze me how people are stuck in the here in the now and just accept the new reality as if it had always been this way, and will continue being this way in the foreseeable future. This is the worst LLMs will ever be. Not outsourcing critical thinking to LLM will be seen as a liability in the not so distant future. Just a year or two ago almost no one even took LLM seriously for coding. Now the status quo has moved for also critical thinking, and we hear comments like this. LLM's chances of making a mistake will be orders of magnitude lowers than humans. Slowly learn to let the wheel go. AI will be much better at holding and controlling it. "We should NEVER outsource number crunching to calculators. They are not 100% reliable and we can make mistakes when inputting the data." Someone in 1920's probably. I know it's a metaphor, and like all metaphors can't be compared 1-1. But it's to make a point how it illustrates my argument above.

palmotea 2mo ago
> This is the worst LLMs will ever be. Not outsourcing critical thinking to LLM will be seen as a liability in the not so distant future. Just a year or two ago almost no one even took LLM seriously for coding. Now the status quo has moved for also critical thinking, and we hear comments like this. LLM's chances of making a mistake will be orders of magnitude lowers than humans. Slowly learn to let the wheel go. AI will be much better at holding and controlling it. If that's true, you're fucked. Because then what good are you for? How long do you think they'll pay you to be useless meat-in-the-middle? Not outsourcing your critical thinking means trying to preserve a scrap of value-add for having you around. > "We should NEVER outsource number crunching to calculators. They are not 100% reliable and we can make mistakes when inputting the data." Someone in 1920's probably. I know it's a metaphor, and like all metaphors can't be compared 1-1. But it's to make a point how it illustrates my argument above. That's a dumb analogy. Automating one activity cannot be extrapolated justify automating all the activities, including some of the most human ones.
- 2mo ago