5 ms·

> I would love to see an anti-AI take that doesn't hinge on the idea that technology forces people to be lazy/careless/thoughtless. The article sort of goes si

> I would love to see an anti-AI take that doesn't hinge on the idea that technology forces people to be lazy/careless/thoughtless. The article sort of goes sideways with this idea but pointing out that AI coding robs you a deep understanding of the code it produces is a valid and important criticism of AI coding. A software engineer's primary job isn't producing code, but producing a functional software system. Most important to that is the extremely hard to convey "mental model" of how the code works and an expertise of the domain it works in. Code is a derived asset of this mental model. And you will never know code as well as a reader and you would have as the author for anything larger than a very small project. There are other consequences of not building this mental model of a piece of software. Reasoning at the level of syntax is proving to have limits that LLM-based coding agents are having trouble scaling beyond.

KoolKat23 1y ago
Not really, its more a case of "potentially can" rather than "will". This dynamic has always been there with the whole junior, senior dev. split, its not a new problem. You 100% can use it without losing this, in an ideal world you can even go so far as to not worry about the understanding for parts that are inconsequential.
danpat 1y ago
> And you will never know code as well as a reader and you would have as the author for anything larger than a very small project. This feels very true - but also consider how much code exists for which many of the current maintainers were not involved in the original writing. There are many anecdotal rules out there about how much time is spent reading code vs writing. If you consider the industry as a whole, it seems to me that the introduction of generative code-writing tools is actually not moving the needle as far as people are claiming. We _already_ live in a world where most of us spend much of our time reading and trying to comprehend code written by others from the past. What's the difference between a messy codebase created by a genAI, and a messy codebase where all the original authors of the code have moved on and aren't available to ask questions?
- PessimalDecimal 1y ago
  > What's the difference between a messy codebase created by a genAI, and a messy codebase where all the original authors of the code have moved on and aren't available to ask questions? The difference is the hope of getting out of that situation. If you've inherited a messy and incoherent code base, you recognize that as a problem and work on fixing it. You can build an understanding of the code through first reading and then probably rewriting some of it. This over time improves your ability to reason about that code. If you're constantly putting yourself back into that situation through relegating the reasoning about code to coding agent, then you won't develop a mental model. You're constantly back at Day 1 of having to "own" someone else's code.
  - baq 1y ago
    > You're constantly back at Day 1 of having to "own" someone else's code. If only there were some people in software engineering in this situation before AI… oh wait. In the current times you’re either an agent manager or you’re in for a surprise.
    - AdieuToLogic 1y ago
      > In the current times you’re either an agent manager or you’re in for a surprise. This opinion seems to be popular, if only in this forum and not in general. What I do not understand is this; In order to use LLM's to generate code, the engineer has to understand the problem sufficient enough to formulate prompt(s) to use in order to get usable output (code). Assuming the engineer has this level of understanding along with knowledge of the target programming language and libraries used, how is using LLM code generation anything more than a typing saver?
      - baq 1y ago
        The point is an engineering manager is using software engineers as typing savers, too. LLMs are, for now, still on an exponential curve of capability on some measures (e.g. task duration with 50% completion chance is doubling every ~7 months) and you absolutely must understand the paradigm shift that will be forced upon you in a few years or you'll have a bad time. Understanding non-critical code paths at all times will simply be pointless; you'll want to make sure test coverage is good and actually test the requirements, etc.
enraged_camel 1y ago
>> The article sort of goes sideways with this idea but pointing out that AI coding robs you a deep understanding of the code it produces is a valid and important criticism of AI coding. All code is temporary and should be treated as ephemeral. Even if it lives for a long time, at the end of the day what really matters is data. Data is what helps you develop the type of deep understanding and expertise of the domain that is needed to produce high quality software. In most problem domains, if you understand the data and how it is modeled, the need to be on top of how every single line of code works and the nitty-gritty of how things are wired together largely disappears. This is the thought behind the idiom “Don’t tell me what the code says—show me the data, and I’ll tell you what the code does.” It is therefore crucial to start every AI-driven development effort with data modeling, and have lots of long conversations with AI to make sure you learn the domain well and have all your questions answered. In most cases, the rest is mostly just busywork, and handing it off to AI is how people achieve the type of productivity gains you read about. Of course, that's not to say you should blindly accept everything the AI generates. Reading the code and asking the AI questions is still important. But the idea that the only way to develop an understanding of the problem is to write the code yourself is no longer true. In fact, it was never true to begin with.
- jay_kyburz 1y ago
  In many cases, the code is "data". The code is a set of rules or instructions by which the data must be transformed for business cases. You need to understand both the data, and the transformations that are being made to the data.
jstummbillig 1y ago
> The article sort of goes sideways with this idea but pointing out that AI coding robs you a deep understanding of the code it produces is a valid and important criticism of AI coding. Why? Code has always been the artifact. Thinking about and understanding the domain clearly and solving problems is where the intrinsic value is at (but I'd suspect that in the future this, too, will go away).
- guy_5676 1y ago
  Code isn't an "artifact", it's the actual product that you are building and delivering. You can use flowery language and pontificate about the importance of the problem domain if you like, but at the end of the day we are producing a low level sequences of instructions that will be executed by a real world device. There has always been, and likely will always be, value in understanding exactly what you are asking the computer to do
  - HPsquared 1y ago
    Code is just one way of representing a specification.
  - alexandre_m 1y ago
    Product != Artifact Artifacts are snapshots of system knowledge (code, builds, docs, configs, etc.). The product is the living whole that emerges from these artifacts working together and delivering value.
    - guy_5676 1y ago
      I'm familiar with "artifact" being used to describe the inconsequential and easy to reproduce output of some deterministic process (e.g. build artifact). Even given the terminology you provide here it doesn't change the content of my point above. When I see someone dismissing the code as a small irrelevant part of the task of writing software, it's like hearing that the low-level design and physical construction of a bridge is an irrelevant side-effect of my desire to cross a body of water. Like, maybe that's true in a philosophical sense, but at the end of the day we are building a real-world bridge that needs to conform to real-world constraints, and every little detail is going to be important. I wouldn't want to cross a bridge built by someone who thinks otherwise.
mattlutze 1y ago
> The article sort of goes sideways with this idea but pointing out that AI coding robs you a deep understanding of the code it produces is a valid and important criticism of AI coding. In any of my teams with moderate to significant code bases, we've always had to lean very hard into code comments and documentation, because a developer will forget in a few months the fine details of what they've previously built. And further, any org with turnover needs to have someone new come in and be able to understand what's there. I don't think I've met a developer that keeps all of the architecture and design deeply in their mind at all times. We all often enough need to go walk back through and rediscover what we have. Which is to say... if the LLM generator was instead a colleague or neighboring team, you'd still need to keep up with them. If you can adapt those habits to the generative code then it doesn't seem to be a bit leap.
halfadot 1y ago
> The article sort of goes sideways with this idea but pointing out that AI coding robs you a deep understanding of the code it produces is a valid and important criticism of AI coding. No it isn't. There's literally nothing about the process that forces you to skip understanding. Any such skips are purely due to the lack of will on the developer's side. This lack of will to learn will not change the outcomes for you regardless of whether you're using an LLM. You can spend as much time as you want asking the LLM for in-depth explanations and examples to test your understanding. So many of the criticisms of coding with LLMs I've seen really do sound like they're coming from people who already started with a pre-existing bias, fiddled with with for a short bit (or worse, never actually tried it at all) and assumed their limited experience is the be-all end-all of the subject. Either that, or they're typical skill issues.
- ZephyrBlu 1y ago
  > No it isn't. There's literally nothing about the process that forces you to skip understanding. Any such skips are purely due to the lack of will on the developer's side This is the whole point. The marginal dev will go to the path of least resistance, which is to skip the understanding and churn out a bunch of code. That is why it's a problem. You are effectively saying "just be a good dev, there's literally nothing about AI which is stopping you from being a good dev" which is completely correct and also missing the point. The marginal developer is not going to put in the effort to wield AI in a skillful way. They're going to slop their way through. It is a concern for widespread AI coding, even if it's not a concern for you or your skill peers in particular.
  - v3xro 1y ago
    To add to the above - I see a parallel to the "if you are a good and diligent developer there is nothing to stop you from writing secure C code" argument. Which is to say - sure, if you also put in extra effort to avoid all the unsafe bits that lead to use-after-free or race conditions it's also possible to write perfect assembly, but in practice we have found that using memory safe languages leads to a huge reduction of safety bugs in production. I think we will find similarly that not using AI will lead to a huge reduction of bugs in production later on when we have enough data to compare to human-generated systems. If that's a pre-existing bias, then so be it.
weego 1y ago
Who are this endless cohort of develops who need to maintain a 'deep understanding' of their code. I'd argue a high % of all code written globally on any given day that is not some flavour of boilerplate, while written with good intention, is ultimately just short-lived engineering detritus of it even gets a code review to pass.
- majormajor 1y ago
  If you're on HN there's a good chance you've self-selected into "caring about the craft and looking for roles that require more attention." You need to care if (a) your business logic requirements are super annoyingly complex, (b) you have hard performance requirements, or (c) both. (c) is the most rare, (a) is the most common of those three conditions; much of the programmer pay disparity between the top and the middle or bottom is due to this, but even the jobs where the complexity is "only" business requirements tend to be quite a bit better compensated than the "simple requirements, simple needs" ones. I think there's a case to be made that LLM tools will likely make it harder for people to make that jump, if they want to. (Alternately they could advance to the point where the distinction changes a bit, and is more purely architectural; or they could advance to the point where anyone can use an LLM to do anything - but there are so many conditional nuances to what the "right decision" is in any given scenario there that I'm skeptical.) A lot of times floor-raising things don't remove the levels, they just push everything higher. Like a cheap crap movie today will visually look "better" from a technology POV (sharpness, special effects, noise, etc) than Jurassic Park from the 90s, but the craft parts won't (shot framing, deliberate shifts of focus, selection of the best takes). So everyone will just get more efficient and more will be expected, but still stratified. And so some people will still want to figure out how to go from a lower-paying job to a higher-paying one. And hopefully there are still opportunities, and we don't just turn into other fields, picking by university reputations and connections.
  - Jensson 1y ago
    > You need to care if (a) your business logic requirements are super annoyingly complex, (b) you have hard performance requirements, or (c) both. (c) is the most rare But one of the most fun things you can do is C: creative game development coding. Like coding world simulations etc, you want to be both very fast but the rules and interactions etc is very coupled and complex compared to most regular enterprise logic that is more decoupled. So while most work programmers do fits A, the work people dream about doing is C, and that means LLM doesn't help you make fun things, it just removes the boring jobs.
posix86 1y ago
What is "understanding code", mental model of the problem? These are terms for which we all have developed a strong & clear picture of what they mean. But may I remind us all that used to not be the case before we entered this industry - we developed it over time. And we developed it based on a variety of highly interconnected factors, some of which are e.g.: what is a program, what is a programming language, what languages are there, what is a computer, what software is there, what editors are there, what problems are there. And as we mapped put this landscape, hadn't there been countless situations where things felt dumb and annoying, and then situation in sometimes they became useful, and sometimes they remained dumb? Something you thought is making you actively loosing brain cells as you're doing them, because you're doing them wrong? Or are you to claim that every hurdle you cross, every roadblock you encounter, every annoyance you overcome has pedagogical value to your career? There are so many dumb things out there. And what's more, there's so many things that appear dumb at first and then, when used right, become very powerful. AI is that: Something that you can use to shoot yourself in the foot, if used wrong, but if used right, it can be incredibly powerful. Just like C++, Linux, CORS, npm, tcp, whatever, everything basically.
noosphr 1y ago
>The article sort of goes sideways with this idea but pointing out that AI coding robs you a deep understanding of the code it produces is a valid and important criticism of AI coding. You can describe what the code should do with natural language. I've found that using literate programming with agent calls to write the tests first, then code, then the human refining the description of the code, and going back to 1 is surprisingly good at this. One of these days I'll get around to writing an emacs mode to automate it because right now it's yanking and killing between nearly a dozen windows. Of course this is much slower than regular development but you end up with world class documentation and understanding of the code base.
jay_kyburz 1y ago
I can imagine an industry where we describe business rules to apply to data in natural language, and the AI simply provides an executable without source at all. The role of the programmer would then be to test if the rules are being applied correctly. If not, there are no bugs to fix, you simply clarify the business rules and ask for a new program. I like to imagine what it must be like for a non technical business owner who employees programmers today. There is a meeting where a process or outcome is described, and a few weeks / months / years a program is delivered. The only way to know if it does what was requested is to poke it a bit and see if it works. The business owner has no metal modal of the code and can't go in and fix bugs. update: I'm not suggesting I believe AI is anywhere near being this capable.
- aryehof 1y ago
  I cant imagine that yet. Programmers to-date cannot reliably achieve such an outcome, so how would a LLM achieve it? We can’t even agree a definition or determine a system for “business rules”? The programming building blocks popular today (lines of code in sub-routines and modules) do not support such a jump?

KoolKat23 1y ago

Not really, its more a case of "potentially can" rather than "will". This dynamic has always been there with the whole junior, senior dev. split, its not a new problem. You 100% can use it without losing this, in an ideal world you can even go so far as to not worry about the understanding for parts that are inconsequential.

danpat 1y ago

> And you will never know code as well as a reader and you would have as the author for anything larger than a very small project. This feels very true - but also consider how much code exists for which many of the current maintainers were not involved in the original writing. There are many anecdotal rules out there about how much time is spent reading code vs writing. If you consider the industry as a whole, it seems to me that the introduction of generative code-writing tools is actually not moving the needle as far as people are claiming. We _already_ live in a world where most of us spend much of our time reading and trying to comprehend code written by others from the past. What's the difference between a messy codebase created by a genAI, and a messy codebase where all the original authors of the code have moved on and aren't available to ask questions?

PessimalDecimal 1y ago
> What's the difference between a messy codebase created by a genAI, and a messy codebase where all the original authors of the code have moved on and aren't available to ask questions? The difference is the hope of getting out of that situation. If you've inherited a messy and incoherent code base, you recognize that as a problem and work on fixing it. You can build an understanding of the code through first reading and then probably rewriting some of it. This over time improves your ability to reason about that code. If you're constantly putting yourself back into that situation through relegating the reasoning about code to coding agent, then you won't develop a mental model. You're constantly back at Day 1 of having to "own" someone else's code.
- baq 1y ago
  > You're constantly back at Day 1 of having to "own" someone else's code. If only there were some people in software engineering in this situation before AI… oh wait. In the current times you’re either an agent manager or you’re in for a surprise.
  - AdieuToLogic 1y ago
    > In the current times you’re either an agent manager or you’re in for a surprise. This opinion seems to be popular, if only in this forum and not in general. What I do not understand is this; In order to use LLM's to generate code, the engineer has to understand the problem sufficient enough to formulate prompt(s) to use in order to get usable output (code). Assuming the engineer has this level of understanding along with knowledge of the target programming language and libraries used, how is using LLM code generation anything more than a typing saver?
    - baq 1y ago
      The point is an engineering manager is using software engineers as typing savers, too. LLMs are, for now, still on an exponential curve of capability on some measures (e.g. task duration with 50% completion chance is doubling every ~7 months) and you absolutely must understand the paradigm shift that will be forced upon you in a few years or you'll have a bad time. Understanding non-critical code paths at all times will simply be pointless; you'll want to make sure test coverage is good and actually test the requirements, etc.

enraged_camel 1y ago

>> The article sort of goes sideways with this idea but pointing out that AI coding robs you a deep understanding of the code it produces is a valid and important criticism of AI coding. All code is temporary and should be treated as ephemeral. Even if it lives for a long time, at the end of the day what really matters is data. Data is what helps you develop the type of deep understanding and expertise of the domain that is needed to produce high quality software. In most problem domains, if you understand the data and how it is modeled, the need to be on top of how every single line of code works and the nitty-gritty of how things are wired together largely disappears. This is the thought behind the idiom “Don’t tell me what the code says—show me the data, and I’ll tell you what the code does.” It is therefore crucial to start every AI-driven development effort with data modeling, and have lots of long conversations with AI to make sure you learn the domain well and have all your questions answered. In most cases, the rest is mostly just busywork, and handing it off to AI is how people achieve the type of productivity gains you read about. Of course, that's not to say you should blindly accept everything the AI generates. Reading the code and asking the AI questions is still important. But the idea that the only way to develop an understanding of the problem is to write the code yourself is no longer true. In fact, it was never true to begin with.

jay_kyburz 1y ago
In many cases, the code is "data". The code is a set of rules or instructions by which the data must be transformed for business cases. You need to understand both the data, and the transformations that are being made to the data.

jstummbillig 1y ago

> The article sort of goes sideways with this idea but pointing out that AI coding robs you a deep understanding of the code it produces is a valid and important criticism of AI coding. Why? Code has always been the artifact. Thinking about and understanding the domain clearly and solving problems is where the intrinsic value is at (but I'd suspect that in the future this, too, will go away).

guy_5676 1y ago
Code isn't an "artifact", it's the actual product that you are building and delivering. You can use flowery language and pontificate about the importance of the problem domain if you like, but at the end of the day we are producing a low level sequences of instructions that will be executed by a real world device. There has always been, and likely will always be, value in understanding exactly what you are asking the computer to do
- HPsquared 1y ago
  Code is just one way of representing a specification.
- alexandre_m 1y ago
  Product != Artifact Artifacts are snapshots of system knowledge (code, builds, docs, configs, etc.). The product is the living whole that emerges from these artifacts working together and delivering value.
  - guy_5676 1y ago
    I'm familiar with "artifact" being used to describe the inconsequential and easy to reproduce output of some deterministic process (e.g. build artifact). Even given the terminology you provide here it doesn't change the content of my point above. When I see someone dismissing the code as a small irrelevant part of the task of writing software, it's like hearing that the low-level design and physical construction of a bridge is an irrelevant side-effect of my desire to cross a body of water. Like, maybe that's true in a philosophical sense, but at the end of the day we are building a real-world bridge that needs to conform to real-world constraints, and every little detail is going to be important. I wouldn't want to cross a bridge built by someone who thinks otherwise.

mattlutze 1y ago

> The article sort of goes sideways with this idea but pointing out that AI coding robs you a deep understanding of the code it produces is a valid and important criticism of AI coding. In any of my teams with moderate to significant code bases, we've always had to lean very hard into code comments and documentation, because a developer will forget in a few months the fine details of what they've previously built. And further, any org with turnover needs to have someone new come in and be able to understand what's there. I don't think I've met a developer that keeps all of the architecture and design deeply in their mind at all times. We all often enough need to go walk back through and rediscover what we have. Which is to say... if the LLM generator was instead a colleague or neighboring team, you'd still need to keep up with them. If you can adapt those habits to the generative code then it doesn't seem to be a bit leap.

halfadot 1y ago

> The article sort of goes sideways with this idea but pointing out that AI coding robs you a deep understanding of the code it produces is a valid and important criticism of AI coding. No it isn't. There's literally nothing about the process that forces you to skip understanding. Any such skips are purely due to the lack of will on the developer's side. This lack of will to learn will not change the outcomes for you regardless of whether you're using an LLM. You can spend as much time as you want asking the LLM for in-depth explanations and examples to test your understanding. So many of the criticisms of coding with LLMs I've seen really do sound like they're coming from people who already started with a pre-existing bias, fiddled with with for a short bit (or worse, never actually tried it at all) and assumed their limited experience is the be-all end-all of the subject. Either that, or they're typical skill issues.

ZephyrBlu 1y ago
> No it isn't. There's literally nothing about the process that forces you to skip understanding. Any such skips are purely due to the lack of will on the developer's side This is the whole point. The marginal dev will go to the path of least resistance, which is to skip the understanding and churn out a bunch of code. That is why it's a problem. You are effectively saying "just be a good dev, there's literally nothing about AI which is stopping you from being a good dev" which is completely correct and also missing the point. The marginal developer is not going to put in the effort to wield AI in a skillful way. They're going to slop their way through. It is a concern for widespread AI coding, even if it's not a concern for you or your skill peers in particular.
- v3xro 1y ago
  To add to the above - I see a parallel to the "if you are a good and diligent developer there is nothing to stop you from writing secure C code" argument. Which is to say - sure, if you also put in extra effort to avoid all the unsafe bits that lead to use-after-free or race conditions it's also possible to write perfect assembly, but in practice we have found that using memory safe languages leads to a huge reduction of safety bugs in production. I think we will find similarly that not using AI will lead to a huge reduction of bugs in production later on when we have enough data to compare to human-generated systems. If that's a pre-existing bias, then so be it.

weego 1y ago

Who are this endless cohort of develops who need to maintain a 'deep understanding' of their code. I'd argue a high % of all code written globally on any given day that is not some flavour of boilerplate, while written with good intention, is ultimately just short-lived engineering detritus of it even gets a code review to pass.

majormajor 1y ago
If you're on HN there's a good chance you've self-selected into "caring about the craft and looking for roles that require more attention." You need to care if (a) your business logic requirements are super annoyingly complex, (b) you have hard performance requirements, or (c) both. (c) is the most rare, (a) is the most common of those three conditions; much of the programmer pay disparity between the top and the middle or bottom is due to this, but even the jobs where the complexity is "only" business requirements tend to be quite a bit better compensated than the "simple requirements, simple needs" ones. I think there's a case to be made that LLM tools will likely make it harder for people to make that jump, if they want to. (Alternately they could advance to the point where the distinction changes a bit, and is more purely architectural; or they could advance to the point where anyone can use an LLM to do anything - but there are so many conditional nuances to what the "right decision" is in any given scenario there that I'm skeptical.) A lot of times floor-raising things don't remove the levels, they just push everything higher. Like a cheap crap movie today will visually look "better" from a technology POV (sharpness, special effects, noise, etc) than Jurassic Park from the 90s, but the craft parts won't (shot framing, deliberate shifts of focus, selection of the best takes). So everyone will just get more efficient and more will be expected, but still stratified. And so some people will still want to figure out how to go from a lower-paying job to a higher-paying one. And hopefully there are still opportunities, and we don't just turn into other fields, picking by university reputations and connections.
- Jensson 1y ago
  > You need to care if (a) your business logic requirements are super annoyingly complex, (b) you have hard performance requirements, or (c) both. (c) is the most rare But one of the most fun things you can do is C: creative game development coding. Like coding world simulations etc, you want to be both very fast but the rules and interactions etc is very coupled and complex compared to most regular enterprise logic that is more decoupled. So while most work programmers do fits A, the work people dream about doing is C, and that means LLM doesn't help you make fun things, it just removes the boring jobs.

posix86 1y ago

What is "understanding code", mental model of the problem? These are terms for which we all have developed a strong & clear picture of what they mean. But may I remind us all that used to not be the case before we entered this industry - we developed it over time. And we developed it based on a variety of highly interconnected factors, some of which are e.g.: what is a program, what is a programming language, what languages are there, what is a computer, what software is there, what editors are there, what problems are there. And as we mapped put this landscape, hadn't there been countless situations where things felt dumb and annoying, and then situation in sometimes they became useful, and sometimes they remained dumb? Something you thought is making you actively loosing brain cells as you're doing them, because you're doing them wrong? Or are you to claim that every hurdle you cross, every roadblock you encounter, every annoyance you overcome has pedagogical value to your career? There are so many dumb things out there. And what's more, there's so many things that appear dumb at first and then, when used right, become very powerful. AI is that: Something that you can use to shoot yourself in the foot, if used wrong, but if used right, it can be incredibly powerful. Just like C++, Linux, CORS, npm, tcp, whatever, everything basically.

noosphr 1y ago

>The article sort of goes sideways with this idea but pointing out that AI coding robs you a deep understanding of the code it produces is a valid and important criticism of AI coding. You can describe what the code should do with natural language. I've found that using literate programming with agent calls to write the tests first, then code, then the human refining the description of the code, and going back to 1 is surprisingly good at this. One of these days I'll get around to writing an emacs mode to automate it because right now it's yanking and killing between nearly a dozen windows. Of course this is much slower than regular development but you end up with world class documentation and understanding of the code base.

jay_kyburz 1y ago

I can imagine an industry where we describe business rules to apply to data in natural language, and the AI simply provides an executable without source at all. The role of the programmer would then be to test if the rules are being applied correctly. If not, there are no bugs to fix, you simply clarify the business rules and ask for a new program. I like to imagine what it must be like for a non technical business owner who employees programmers today. There is a meeting where a process or outcome is described, and a few weeks / months / years a program is delivered. The only way to know if it does what was requested is to poke it a bit and see if it works. The business owner has no metal modal of the code and can't go in and fix bugs. update: I'm not suggesting I believe AI is anywhere near being this capable.

aryehof 1y ago
I cant imagine that yet. Programmers to-date cannot reliably achieve such an outcome, so how would a LLM achieve it? We can’t even agree a definition or determine a system for “business rules”? The programming building blocks popular today (lines of code in sub-routines and modules) do not support such a jump?