4 ms·

For anyone that does LLM supported work it's crystal clear that skill decline is real. Surprisingly, the leadership at my work doesn't give rats ass about it. T

by xendo 2mo ago

For anyone that does LLM supported work it's crystal clear that skill decline is real. Surprisingly, the leadership at my work doesn't give rats ass about it. They published AI engineering manifesto, pushing for loops, workflows, advocating for producing many times more code. All of this ignoring the elephant in the room.

chrisjj 2mo ago
Why surprised? The elephant is in someone else's room.
noir_lord 2mo ago
Cynically they may be banking (however misguided it turns out to be) on model skill in aggregate growing faster than the skills of their staff decline at which point, bin the staff, use the model. The hype from non-technical people about AI would make you think they already discovered AGI.
- burningChrome 2mo ago
  >> model skill in aggregate growing faster than the skills of their staff decline at which point, bin the staff, use the model. How people in tech are aware they're training their replacements? Or are we quickly approaching an apex where the people running these companies realize AI cannot completely replace human developers?
  - ThePhysicist 2mo ago
    I don't know, I find AI still misses the mark so much for stuff that's not happening inside a well constrained framework with very specific guardrails and boundary conditions. Every time I have agents build something half complex that I would be easily able to pull off myself (albeit not at their pace) I always ask if everything is correct and working and I get a resounding "yes!", then I test it and it's essentially a steaming pile of garbage which I then tell the agent and after which it goes "you're absolutely right to push back, this isn't good at all!" as if it was clear from the beginning. I don't know if we'll achieve real intelligence soon with these things but what we have now isn't there yet. It's incredibly useful but I don't think you can replace a good programmer with it, you can make a good programmer into an excellent one though! Same goes for writing, anyone who has written long complex texts with LLMs knows that a ton of editing is required to make it half decent.
    - deleted 2mo ago
      [deleted]
    - brokenmachine 2mo ago
      Even for my tiny toy projects, I notice a bug and I tell the LLM and it corrects it, most of the time, but I shudder at the amount of code it generates or changes. If it has some subtle data corruption bug then you're hosed. It's just crazy to me to think about running AI changes in production on a real product.
  - taurath 2mo ago
    > Or are we quickly approaching an apex where the people running these companies realize AI cannot completely replace human developers? The boosters are loud and the failures are made silent, until the failures become impossible to ignore. Tens to hundreds of millions of dollars spent on AI in enterprise software companies need to show RoI at some point, and the promises of executives leaning on AI's productivity improvements to make their accelerated release schedules need to actually make the releases. Its currently a very dangerous career move to publically apply critical thinking to LLM capabilities. I think the math sort of does itself.
FrustratedMonky 2mo ago
" skill decline is real" Is this -> More people that are not devs, are using AI to create things, and are skilling up to a mediocre level? or Is this -> Existing experts, are actually loosing there skills? Using AI is reducing someone already known skills.
- jselysianeagle 2mo ago
  > More people that are not devs, are using AI to create things, and are skilling up to a mediocre level? What skills are being developed by non-devs prompting AI to churn out code they don't understand? Note that I'm not debating if the code is good or works or whatever, I'm asking if any real skills are being developed merely by prompting AI towards some goal.
  - FrustratedMonky 2mo ago
    I don't think it is zero. If they are producing code, there is some knowledge being absorbed. Nobody is using AI to create an app is learning zero. Its just what they are learning, is not as much as they think, and really random, not-structured. Like reading just a few random chapter of a book across multiple classes. They get a smattering of ad-hoc tidbits of knowledge. I am really worried if using AI is reducing someone's already gained knowledge. Moving someone backwards. That has me a bit more scared.
    - bluefirebrand 2mo ago
      > If they are producing code, there is some knowledge being absorbed. Nobody is using AI to create an app is learning zero Maybe not actually absolutely zero, but it's much closer to zero than what they would learn creating the app themselves
    - jplusequalt 2mo ago
      >I am really worried if using AI is reducing someone's already gained knowledge. Moving someone backwards. That has me a bit more scared. It absolutely is. It removes a lot of friction from pathways that kept previous knowledge firm. This isn't unique to AI. When calculators came around, people had a new tool that decreased the need to do arithmetic in your head. This has lead to a decline in the percentage of people who knew arithmetic, but can no longer do it adequately without the external tool. Another example would be assembly skills declining amongst software engineers after compiled languages become dominant. What I would argue is unique to AI is two-fold. The first being that the scale of things being automated is enormous. Keeping with just software, AI isn't only a tool that decreases the need to write code, it's also a tool that decreases the need to do your own research, debugging, system design, version control, etc. The list of tasks a regular engineer is responsible for that have not been consumed by AI is small. To make matters more frightening, this is just a single industry. The same applies to multiple other knowledge heavy fields (e.g. mathematics). The second thing I'd argue is unique about AI is the addictive nature of it. These LLMs are deliberately coerced into being sycophants that blow smoke up your ass unless you deliberately tell the model not too (and it may still do it anyways!). A calculator didn't compliment your insight for asking it to calculate the square root of -1. Furthermore, many people have talked about the resemblance of LLMs to slot machines--there's a feeling that anything could be possible if you only prompt the model correctly. Your first prompt is wildly off the mark, but your next prompt is better though still not quite right, so you continue forward. You keep iterating your instructions, your word choices, your tone, all in search of the desired response. In this way, an LLM is a slot machine, your prompt the lever you pull, and the output is the jackpot. All this taken together, we have a machine that decreases the need to engage with the critical faculties of your own brain, which has been trained to keep your attention through flattery, and whose probabilistic nature preys on our love for gambling.
sushusshs 2mo ago
I don’t think it’s as clear cut as you make it out to be. Maintenance <> growth Once you got a skill it’s quite hard to actually lose it. It’s like riding a bike as they say. I find all the reading of those walls of code exercises my mind better than the writing did. I’m sure I lost memories of how to read a file by hand or balance a tree but who cares? I am thinking about architecture and design a lot more these days and every decision I make has to be actually argued for even to myself. I can no longer lean back and say “that’s how we always do it” or “too expensive to change now” as I find many people in practice actually do. They were just coasting on premade architectural choices and their “skills” consisted of knowing arcane incantations and syntactic details completely unrelated to the (business) problem at hand. I am not convinced many developers actually have the skills they think they have. They could wrestle syntax and mess around with tooling, but could they abstract properly? Define clear semantic boundaries? Have proper civil discussions about responsibilities and where they should lie on the right level of abstraction? Nothing has changed in that regard. If anything that part has been amplified. (“taste”)
ndriscoll 2mo ago
It doesn't seem clear to me at all. e.g. I've been having a fun time learning Godot this last week with codex laying down groundwork for logic, generating assets, etc. This gives me something concrete that I can work with and ground myself within instead of starting from a blank canvas. I don't much care for the actual scripting logic because it's all basically trivial to me with a few decades of programming experience, and I'm not going to be able to develop my artistic abilities to a useful capacity anytime within the next few years, so I can instead focus on learning architecture/organization patterns and game systems that I'm interested in while I read through docs. So basically, if you use it to wave away things you've mastered or explicitly don't want to learn at the moment, you can stay focused on things you are figuring out. I don't see it as any different from writing "by induction" in a math proof without writing all the details because you and the reader know you could easily work them out. In this way I can learn things that I simply would not have the time to dig into before, increasing my skillset. As is was before AI, metacognition is the most widely useful skill to have.
doug_durham 2mo ago
I don't see it being "crystal clear" that skill decline is real. People are doing work they wouldn't have been able to do before. They are getting a more broad exposure to technologies than they have before.
- nomel 2mo ago
  Seems like pretty clear breadth vs depth tradeoff, with a reduced total area. People rarely cared about the details to begin with. Now those that did care can no longer justify the time that was previously required.

by xendo 2mo ago

chrisjj 2mo ago

Why surprised? The elephant is in someone else's room.

noir_lord 2mo ago

Cynically they may be banking (however misguided it turns out to be) on model skill in aggregate growing faster than the skills of their staff decline at which point, bin the staff, use the model. The hype from non-technical people about AI would make you think they already discovered AGI.

burningChrome 2mo ago
>> model skill in aggregate growing faster than the skills of their staff decline at which point, bin the staff, use the model. How people in tech are aware they're training their replacements? Or are we quickly approaching an apex where the people running these companies realize AI cannot completely replace human developers?
- ThePhysicist 2mo ago
  I don't know, I find AI still misses the mark so much for stuff that's not happening inside a well constrained framework with very specific guardrails and boundary conditions. Every time I have agents build something half complex that I would be easily able to pull off myself (albeit not at their pace) I always ask if everything is correct and working and I get a resounding "yes!", then I test it and it's essentially a steaming pile of garbage which I then tell the agent and after which it goes "you're absolutely right to push back, this isn't good at all!" as if it was clear from the beginning. I don't know if we'll achieve real intelligence soon with these things but what we have now isn't there yet. It's incredibly useful but I don't think you can replace a good programmer with it, you can make a good programmer into an excellent one though! Same goes for writing, anyone who has written long complex texts with LLMs knows that a ton of editing is required to make it half decent.
  - deleted 2mo ago
    [deleted]
  - brokenmachine 2mo ago
    Even for my tiny toy projects, I notice a bug and I tell the LLM and it corrects it, most of the time, but I shudder at the amount of code it generates or changes. If it has some subtle data corruption bug then you're hosed. It's just crazy to me to think about running AI changes in production on a real product.
- taurath 2mo ago
  > Or are we quickly approaching an apex where the people running these companies realize AI cannot completely replace human developers? The boosters are loud and the failures are made silent, until the failures become impossible to ignore. Tens to hundreds of millions of dollars spent on AI in enterprise software companies need to show RoI at some point, and the promises of executives leaning on AI's productivity improvements to make their accelerated release schedules need to actually make the releases. Its currently a very dangerous career move to publically apply critical thinking to LLM capabilities. I think the math sort of does itself.

FrustratedMonky 2mo ago

" skill decline is real" Is this -> More people that are not devs, are using AI to create things, and are skilling up to a mediocre level? or Is this -> Existing experts, are actually loosing there skills? Using AI is reducing someone already known skills.

jselysianeagle 2mo ago
> More people that are not devs, are using AI to create things, and are skilling up to a mediocre level? What skills are being developed by non-devs prompting AI to churn out code they don't understand? Note that I'm not debating if the code is good or works or whatever, I'm asking if any real skills are being developed merely by prompting AI towards some goal.
- FrustratedMonky 2mo ago
  I don't think it is zero. If they are producing code, there is some knowledge being absorbed. Nobody is using AI to create an app is learning zero. Its just what they are learning, is not as much as they think, and really random, not-structured. Like reading just a few random chapter of a book across multiple classes. They get a smattering of ad-hoc tidbits of knowledge. I am really worried if using AI is reducing someone's already gained knowledge. Moving someone backwards. That has me a bit more scared.
  - bluefirebrand 2mo ago
    > If they are producing code, there is some knowledge being absorbed. Nobody is using AI to create an app is learning zero Maybe not actually absolutely zero, but it's much closer to zero than what they would learn creating the app themselves
  - jplusequalt 2mo ago
    >I am really worried if using AI is reducing someone's already gained knowledge. Moving someone backwards. That has me a bit more scared. It absolutely is. It removes a lot of friction from pathways that kept previous knowledge firm. This isn't unique to AI. When calculators came around, people had a new tool that decreased the need to do arithmetic in your head. This has lead to a decline in the percentage of people who knew arithmetic, but can no longer do it adequately without the external tool. Another example would be assembly skills declining amongst software engineers after compiled languages become dominant. What I would argue is unique to AI is two-fold. The first being that the scale of things being automated is enormous. Keeping with just software, AI isn't only a tool that decreases the need to write code, it's also a tool that decreases the need to do your own research, debugging, system design, version control, etc. The list of tasks a regular engineer is responsible for that have not been consumed by AI is small. To make matters more frightening, this is just a single industry. The same applies to multiple other knowledge heavy fields (e.g. mathematics). The second thing I'd argue is unique about AI is the addictive nature of it. These LLMs are deliberately coerced into being sycophants that blow smoke up your ass unless you deliberately tell the model not too (and it may still do it anyways!). A calculator didn't compliment your insight for asking it to calculate the square root of -1. Furthermore, many people have talked about the resemblance of LLMs to slot machines--there's a feeling that anything could be possible if you only prompt the model correctly. Your first prompt is wildly off the mark, but your next prompt is better though still not quite right, so you continue forward. You keep iterating your instructions, your word choices, your tone, all in search of the desired response. In this way, an LLM is a slot machine, your prompt the lever you pull, and the output is the jackpot. All this taken together, we have a machine that decreases the need to engage with the critical faculties of your own brain, which has been trained to keep your attention through flattery, and whose probabilistic nature preys on our love for gambling.

sushusshs 2mo ago

I don’t think it’s as clear cut as you make it out to be. Maintenance <> growth Once you got a skill it’s quite hard to actually lose it. It’s like riding a bike as they say. I find all the reading of those walls of code exercises my mind better than the writing did. I’m sure I lost memories of how to read a file by hand or balance a tree but who cares? I am thinking about architecture and design a lot more these days and every decision I make has to be actually argued for even to myself. I can no longer lean back and say “that’s how we always do it” or “too expensive to change now” as I find many people in practice actually do. They were just coasting on premade architectural choices and their “skills” consisted of knowing arcane incantations and syntactic details completely unrelated to the (business) problem at hand. I am not convinced many developers actually have the skills they think they have. They could wrestle syntax and mess around with tooling, but could they abstract properly? Define clear semantic boundaries? Have proper civil discussions about responsibilities and where they should lie on the right level of abstraction? Nothing has changed in that regard. If anything that part has been amplified. (“taste”)

ndriscoll 2mo ago

It doesn't seem clear to me at all. e.g. I've been having a fun time learning Godot this last week with codex laying down groundwork for logic, generating assets, etc. This gives me something concrete that I can work with and ground myself within instead of starting from a blank canvas. I don't much care for the actual scripting logic because it's all basically trivial to me with a few decades of programming experience, and I'm not going to be able to develop my artistic abilities to a useful capacity anytime within the next few years, so I can instead focus on learning architecture/organization patterns and game systems that I'm interested in while I read through docs. So basically, if you use it to wave away things you've mastered or explicitly don't want to learn at the moment, you can stay focused on things you are figuring out. I don't see it as any different from writing "by induction" in a math proof without writing all the details because you and the reader know you could easily work them out. In this way I can learn things that I simply would not have the time to dig into before, increasing my skillset. As is was before AI, metacognition is the most widely useful skill to have.

doug_durham 2mo ago

I don't see it being "crystal clear" that skill decline is real. People are doing work they wouldn't have been able to do before. They are getting a more broad exposure to technologies than they have before.

nomel 2mo ago
Seems like pretty clear breadth vs depth tradeoff, with a reduced total area. People rarely cared about the details to begin with. Now those that did care can no longer justify the time that was previously required.