7 ms·

I wonder if the study includes the technical debt that more experienced developers had to tackle after the less experienced devs have contributed their AI-drive

I wonder if the study includes the technical debt that more experienced developers had to tackle after the less experienced devs have contributed their AI-driven efforts. Because my personal experience has involved a lot of that in one of the companies listed in the study. Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them though.

deleted 2y ago
[deleted]
anonzzzies 2y ago
> had to tackle after the less experienced devs have contributed their AI-driven efforts. So, like before AI then? I haven't seen AI deliver illogical nonsense that I couldn't even decipher like I have seen some outsourcing companies deliver.
- htrp 2y ago
  AI seems to give more consistent results than outsourcing (not necessarily better, but at least more predictable failure modes)
- trashtester 2y ago
  > that I couldn't even decipher This is one of the challenges of being a tech lead. Sometimes code is hard to comprehend. In my experience, AI delivered code is no worse than entry level developer code.
  - anonzzzies 2y ago
    But that is a HN bubble thing: I work and worked with seniors with 10-15 years under their belt who have no logic bone in their body; the worst (and an AI does not do that) is when there is a somewhat 'busy' function that has an if or switch statement and, over time, to add features or fix bugs, ifs were added. Now after 5+ years, this function is 15000 lines and is somewhat of a trained neural network adjacent thing; 100s of nested ifs, pages long, that cannot be read and cannot be followed even if you have a brain. This is made by senior staff of, usually, outsourcing companies and I have seen it very many times over the past 40 years. Not entry level, not small companies either. I know a gov tax system which was partly maintained by a very large and well known outsourcing company which has numerous of these puppies in production that no one dares to touch. AI doesn't do stuff like because it could not, which to me, is a good thing. When it gets better, it might start to do it, I don't know. People here live in a bubble where they think the world is full of people who read 'beautiful code', make tests, use git or something instead of zip$date and know how DeMorgan works; by far, most don't, not juniors, not seniors.
    - nottorp 2y ago
      > to add features or fix bugs, ifs were added. "Back to that two page function. Yes, I know, it’s just a simple function to display a window, but it has grown little hairs and stuff on it and nobody knows why. Well, I’ll tell you why: those are bug fixes. One of them fixes that bug that Nancy had when she tried to install the thing on a computer that didn’t have Internet Explorer. Another one fixes that bug that occurs in low memory conditions. Another one fixes that bug that occurred when the file is on a floppy disk and the user yanks out the disk in the middle. " https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-... Of course, you can try to split the 15000 line function in logical blocks or something, but don't assume the ifs in it are useless.
- nottorp 2y ago
  > I haven't seen AI deliver illogical nonsense I have. If you're doing niche-er stuff it doesn't have enough data and hallucinates. The worst is when it spits two screens of code instead of 'this cannot be done at the level you want it'. > that I couldn't even decipher That's unrelated to code quality. Especially with C++ which has become as write only as perl.
sgarland 2y ago
> Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. Thank you, this says what I have been struggling to describe. The day I lost part of my soul was when I asked a dev if I could give them feedback on a DB schema, they said yes, and then cut me off a few minutes in with, “yeah, I don’t really care [about X].” You don’t care? I’m telling you as the SME for this exactly what can be improved, how to do it, and why you should do so, but you don’t care. Cool. Cloud was a mistake; it’s inculcated people with the idea that chasing efficiency and optimization doesn’t matter, because you can always scale up or out. I’m not even talking about doing micro-benchmarks (though you should…), I’m talking about dead-simple stuff like “maybe use this data structure instead of that one.”
- dartos 2y ago
  I’ve noticed this more and more since 2020. A lot of people who entered the field in the past 6 or so years are here for the money, obviously. Nothing wrong with that at all, but as someone with a long time programming and technology passion, it’s sad to see that change.
  - sgarland 2y ago
    There’s certainly nothing wrong with enjoying the high pay, no – I definitely do. But yeah, it’s upsetting to find out how few people care. Even moreso when they double down and say that you shouldn’t care either, because it’s been abstracted away, blah blah blah. Who do you think is going to continue creating these abstractions for you?
    - ebiester 2y ago
      ...Someone else who thinks of actual web applications as an abstraction. In the olden days, we used to throw it over to "Ops" and say, "your problem now." And Junior developers have always been overwhelmed with the details and under pressure to deliver enough to keep their job. None of this is new! I'm a graybeard now, but I remember seniors having the same complaints back then. "Kids these days" never gets old.
nerdjon 2y ago
I am curious about this also. I have now seen multiple PR's that I had to review that a method was clearly completely modified by AI with no good reason and when asked why something was changed we just got silence. Not exaggerating, literal silence and then trying to ignore the question and explain the thing we were first asking them to do. Clearly having no idea what is actually in this PR. This was done because we asked for a minor change to be done (talking maybe 5 lines of code) and tested. So now not only are we dealing with new debt, we are dealing with code that no one can explain why it was completely changed (and some of the changes were changes for the sake of change), and we are dealing with those of us that manage this code now looking at completely foreign code. I keep seeing this with people that are using these tools and they are not higher level engineers. We finally got to the point of denying these PR's and saying to go back and do it again. Loosing any of the time that was theoretically gained from doing it in the first place. Not saying these tools don't have a place. But people are using it without understanding what it is putting out and not understanding the long term effects it will have on a code base.
- okwhateverdude 2y ago
  > Not saying these tools don't have a place. But people are using it without understanding what it is putting out and not understanding the long term effects it will have on a code base. It is worse than that. We're all maintaining in our heads the mental sand castle that is the system the code base represents. The abuse of the autocoder erodes that sand castle because the intentions of the changes, which are crucial for mentally updating the sand castle, are not communicated (because they are unknowable). This is same thing with poor commit messages, or poor documentation around requirements/business processes. With enough erosion, plus expected turn over in staff, the sand castle is actually gone.
  - rongenre 2y ago
    There needs to be a pattern for including AI-generated code, recording all the context which led to the generation.
    - 2y ago
anonzzzies 2y ago
> Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. I found this too. But I also found the opposite, including here on HN; people who are interested in technology have almost an aversion against using AI. I personally love tech and I would and do write software for fun, but even that is objectively more fun for me with AI. It makes me far more productive (very much more than what the article states) and, more importantly, it removes the procrastination; whenever I am stuck or procrastinating getting to work or start, I start talking with Aider and before I know it, another task was done that I probably wouldn't have done that day without. That way I now launch bi weekly open and closed source projects while before that would take months to years. And the cost of having this team of fast experienced devs sitting with me is max a few $ per day.
- andybak 2y ago
  Yeah. There is a psychological benefit to using AI that I find very beneficial. A lot of tasks that I would have avoided or wasted time doing task avoidance on suddenly become tractable. I think Simon Willison said something similar.
  - jcgrillo 2y ago
    Are you just working on personal projects or in a shared codebase with tens or hundreds of other devs? If the latter, how do you keep your AI generated content from turning the whole thing into an incomprehensible superfund site of tech debt? I've gotten a lot of mileage in my career so far by paying attention when something felt tedious or mundane, because that's a signal you need some combination of refactoring, tooling, or automation. If instead you just lean on an LLM to brute-force your way through, sure, that accomplishes the short term goal of shipping your agile sprint deliverable or whatever, but what of the long term cost?
    - andybak 2y ago
      > Are you just working on personal projects or in a shared codebase with tens or hundreds of other devs? Like - I presume almost everyone - somewhere in the middle? That was a helluva dichotomy to offer me... > how do you keep your AI generated content from turning the whole thing into an incomprehensible superfund site of tech debt? By reading it, thinking about it and testing it? Did I somehow give the impression I'm cutting and pasting huge globs of code straight from ChatGPT into a git commit? There's a weird gulf of incomprehension between people that use AI to help them code and those that don't. I'm sure you're as confused by this exchange as I am.
infecto 2y ago
I hear this but I don't think this is a new issue that AI brought, it simply magnified it. That's a company culture issue. It reminds me of a talk Raymond Hettinger put on a while ago about rearranging the flowers in the garden. There is a tendency from new developers to rearrange for no good reason, AI makes it even easier now. This comes down to a culture problem to me, AI is simply the tool but the driver is the human (at least for now).
noobermin 2y ago
It doesn't, it looks at PRs, commits, and builds only. They remark at this lacking context.
fer 2y ago
Disclaimer: I work at a company who sells coding AI (among many other things). We use it internally and the technical debt is an enormous threat that IMO hasn't been properly gauged. It's very very useful to carpet bomb code with APIs and patterns you're not familiar with, but it also leads to insane amounts of code duplication and unwieldy boilerplate if you're not careful, because: 1. One of the two big bias of the models is the fact that the training data is StackOverflow-type training data, which are examples and don't take context and constraints into account. 2. The other is the existing codebase, and it tends to copy/repeat things instead of suggesting you to refactor. The first is mitigated by, well, doing your job and reviewing/editing what the LLM spat out. The second can only be mitigated once diffs/commit history become part of the training data, and that's a much harder dataset to handle and tag, as some changes are good (refactorings) but other might be not (bugs that get corrected in subsequent commits) and no clear distinction as commit messages are effectively lies (nobody ever writes: bug introduced). Not only that, merges/rebases/squashes alter/remove/add spurious meanings to the history, making everything blurrier.
- trashtester 2y ago
  If you're in a company that is valued at $10M+ per developer, the technical debt is not a big concern. Either you will go bust, OR you will be able to hire enough people to pay those debts, once you get traction in the market.
- dboreham 2y ago
  Consider myself very fortunate to have lived long enough that I'm reading a thread where the subject is the quality of the code generated by software. Decades of keeping that lollypop ready to be given, and now look where we are!
  - samatman 2y ago
    The rate at which I quote Perlis 93 is a roughly quadratic function of the number of quarters since coding LLMs were first deployed.
- disconcision 2y ago
kraftman 2y ago
> Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them though. Is this a bad thing? Maybe I'm misunderstanding it, but even when I'm working on my own projects, I'm usually trying to solve a problem, and the technology is a means to an end to solving that problem (delivering). I care that it works, and is maintainable, I don't care that much about the technology.
- never_inline 2y ago
  Read it as "closing a ticket" that makes sense. They don't care if the code explodes after the sprint.
- layer8 2y ago
  Caring that it works and is maintainable is caring about the technology.
  - collyw 2y ago
    No code solutions are often superior. Around 15 years ago we were shipping terabyte hard disks as it was faster than the internet (until one got stuck in customs)
add-sub-mul-div 2y ago
Maintaining bad AI code is the new maintaining bad offshore code.
acedTrex 2y ago
> Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them though. Bingo, this, so much this. Every dev i know who loves AI stuff was a dev that I had very little technical respect for pre AI. They got some stuff done but there was no craft or quality to it.
- chasd00 2y ago
  i'm certainly skeptical of genai but your argument sounds very familiar to assembler dev feelings towards c++ devs back in the day.
  - acedTrex 2y ago
    I am not necessarily arguing against GenAI. I am sure it will have somewhat similar effects to how the explosion in popularity of garbage collected languages et al had on software back in the 90s. More stuff will get done, the barrier of entry will be lower etc. The craft of programming took a significant quality/care hit when it transitioned from "only people who care enough to learn the ins and outs of memory management can feasibly do this" to "now anyone with a technical brain and a business use case can do it". Which makes sense, the code was no longer the point. The C++ devs rightly felt superior to the new java devs in the narrow niche of "ability to craft code." But that feeling doesn't move the needle business wise in the vast majority of circumstances. Which is always the schism between large technology leaps. Basically, the argument of "its worse" is not WRONG. Just, the same as it did not really matter in the mid 90s. Does not matter as much now, compared to the ability to "just get something that kinda works."
- wahnfrieden 2y ago
  You're playing status games
  - acedTrex 2y ago
    I mean... in a way yes. The status of being someone who cares about the craft of programming. In the scheme of things however, that status hardly matters compared to the "ability to get something shipped quickly" which is what the vast majority of people are paid to do. So while I might judge those people for not meeting my personal standards or bar. In many cases that does not actually matter. They got something out there, thats all that matters.
jacobsenscott 2y ago
> Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them though. You get what you measure. Nobody measure software quality.
- wahnfrieden 2y ago
  Maybe not at your workplaces, but at mine, we measured bugs, change failure rate, uptime, "critical functionality" uptime, regressions, performance, CSAT, etc. in addition to qualitative research on quality in-team and with customers
  - jacobsenscott 2y ago
    That's 0.001% of the workplaces out there. But good for them. Do you think it actually improved software quality, or was it all busy work?
    - wahnfrieden 2y ago
      Yah
prophesi 2y ago
From my experience, most of it is quickly caught in code review. And after a while it occurs less and less, granted that the junior developer puts in the effort to learn why their PRs aren't getting approved. So, pretty similar to how it was before. Except that motivated junior developers will improve incredibly fast. But that's also kind of always been the case in software development these past two decades?
- YeGoblynQueenne 2y ago
  It's interesting to see your comment currently right next to that of nerdjon here: https://news.ycombinator.com/item?id=41465827 https://news.ycombinator.com/item?id=41465827 (the two comments might have moved apart by the time you read this). Edit: yep, they just did.
Kiro 2y ago
I don't think it's that clear cut. I personally think the AI often delivers a better solution than the one I had in mind. It always contains a lot more safe guards against edge cases and other "boring" stuff that the AI has no problem adding but others find tedious.
- trashtester 2y ago
  If you're building a code base where AI is delivering on the the details of it, it's generally a bad thing if the code provided by AI provide safeguards WITHIN your code base. Those kinds of safeguards should instead be part of the framework you're using. If you need to prevent SQL injection, you need to make sure that all access to the SQL type database pass through a layer that prevents that. If you are worried about the security of your point of access (like an API facing the public), you need to apply safeguards as close to the point of entry as possible, and so on. I'm a big believer in AI generated code (over a long horizon), but I'm not sure the edge case robustness is the main selling point.
  - Kiro 2y ago
    Sounds like we're talking about different kind of safeguards. I mean stuff like a character in a game ending up in a broken case due to something that is theoretically possible but very unlikely or where the consequence is not worth the effort. An AI has no problem taking those into account and write tedious safeguards, while I skip it.
godelski 2y ago
> I wonder if the study includes the technical debt that more experienced developers had to tackle after the less experienced devs have contributed their AI-driven efforts. It does not You also may find this post from the other day more illuminating[0], as I believe the actual result strongly hints at what you're guessing. The study is high schoolers doing math. While GPT only has an 8% error rate for the final answer, it gets the steps wrong half the time. And with coding (like math), the steps are the important bits. But I think people evaluate very poorly when there's ill defined metrics but some metric exists. They over inflate it's value since it's concrete. Like completing a ticket doesn't mean you made progress. Introducing technical debt would mean taking a step back. A step forward in a very specific direction but away from the actual end goal. You're just outsourcing work to a future person and I think we like to pretend this doesn't exist because it's hard to measure. [0] https://news.ycombinator.com/item?id=41453300 https://news.ycombinator.com/item?id=41453300
mrighele 2y ago
> Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them though. For them programming is a means to an end, and I think it is fine, in a way. But you cannot just ask an AI to write you tiktok clone and expect to get the finished product. Writing software is an iterative process, and LLMs currently used are not good enough for that, because they need not only to answer the questions, but at the very minimum to start asking questions: "why do want to do that ?" "do you prefer this or that", etc., so that they can actually extract all the specification details that the user happily didn't even know he needed before producing an appropriate output. (It's not too different from how some independent developers has to handle their clients, isn't it ?). Probably we will get there, but not too soon. I also doubt that current tools can keep a project architecturally sound long-term, but that is just an hunch. I admit though that I may be biased because I don't like much tools like copilot: when I write software, I have in my mind a model of the software that I am writing/I want to write, the AI has another model "in mind" and I need to spend mental energy understanding what it is "thinking". Even if 99/100 it is what I wanted, the remaining 1% is enough to hold me back from trusting it. Maybe I am using it the wrong way, who knows. The AI tool that work for me would be a "voice controller AI powered pair programmer": I write my code, then from time to time I ask him questions on how to do something, and I can get either an contextual answer depending on the code I am working on, or generate the actual code if I wish so". Are there already plugins working that way for vscode/idea/etc ?
- liminalsunset 2y ago
  I've been playing with Cursor (albeit with a very small toy codebase) and it does seem like it could do some of what you said - it has a number of features, not all of which necessarily generate code. You can ask questions about the code, about documentation, and other things, and it can optionally suggest code that you can either accept, accept parts of, or decline. It's more of a fork of vscode than a plugin right now though. It is very nice in that it gives you a handy diffing tool before you accept, and it very much feels like it puts me in control.
  - mrighele 2y ago
creativeSlumber 2y ago
This. Even without AI, we have inexperienced developers rolling out something that "just works" without thinking about many of the scaling/availability issues. Then you have to spend 10x the time fixing those issues.
- Sakos 2y ago
  If inexperienced developers can commit code like that to master, then there's a culture issue, a process issue or both. This isn't a problem with AI.
deleted 2y ago
[deleted]
theflyinghorse 2y ago
> I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. If I don't deliver my startup burns in a year. In my previous role if I didn't deliver the people who were my reports did not get their bonuses. The incentives are very clear, and have always been clear - deliver.
- rurp 2y ago
  Succesful companies aren't built in a sprint. I doubt there has ever been a successful startup that didn't have at least some competent people thinking a number of steps ahead. Piling up tech debt to hit some short term arbitrary goals is not a good plan for real success.
  - theflyinghorse 2y ago
    These are platitudes. Here's a real example of delivering something now without worrying about being the best engineer I can. I have 2 CSRs, They are swamped with work and we're weeks away from bringing another CSR on board. I find a couple of time consuming tasks that are easy to automate and build those out separately as one-off jobs that work well enough. Instantly it's a solid time gain & stress reducer to CSRs. Are my one-off automation tasks a long term solution? No. Do I care? Not at the moment, and my ego can take a hit for the time being.
jonnycomputer 2y ago
Code quality is the hardest thing to measure. Seems like they were measuring commits, pull-requests, builds, and build success rate. This sort of gets at that, but is probably inadequate. The few attempts I've made at using genAI to make large-scale changes to code have been failures, and left me in the dark about the changes that were made in ways that were not helpful. I needed suggestions to be in much smaller chunks. paragraph sized. Right now I limit myself to using the genAI line completion suggestions in Pycharm. It very often guesses my intentions and so actually is helpful, particularly when laboriously typing out lots of long literals, e.g. keys in a dictionary.
- collyw 2y ago
  WTFs per minute is the standard way of measuring code quality. Lower being better.
elric 2y ago
I don't remember who said it, but "AI generated code turns every developer into a legacy code maintainer". It's pithy and a bit of an exaggeration, but there's a grain of truth in there that resonates with me.
EvkoGS 2y ago
>Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them though. You are not a fucking priest in the temple of engineering, go to fucking CS dep at the local uni and be the one and preach it there. You are worker of the company with customers, which pays you a salary from customers money.

deleted 2y ago

[deleted]

anonzzzies 2y ago

> had to tackle after the less experienced devs have contributed their AI-driven efforts. So, like before AI then? I haven't seen AI deliver illogical nonsense that I couldn't even decipher like I have seen some outsourcing companies deliver.

htrp 2y ago
AI seems to give more consistent results than outsourcing (not necessarily better, but at least more predictable failure modes)
trashtester 2y ago
> that I couldn't even decipher This is one of the challenges of being a tech lead. Sometimes code is hard to comprehend. In my experience, AI delivered code is no worse than entry level developer code.
- anonzzzies 2y ago
  But that is a HN bubble thing: I work and worked with seniors with 10-15 years under their belt who have no logic bone in their body; the worst (and an AI does not do that) is when there is a somewhat 'busy' function that has an if or switch statement and, over time, to add features or fix bugs, ifs were added. Now after 5+ years, this function is 15000 lines and is somewhat of a trained neural network adjacent thing; 100s of nested ifs, pages long, that cannot be read and cannot be followed even if you have a brain. This is made by senior staff of, usually, outsourcing companies and I have seen it very many times over the past 40 years. Not entry level, not small companies either. I know a gov tax system which was partly maintained by a very large and well known outsourcing company which has numerous of these puppies in production that no one dares to touch. AI doesn't do stuff like because it could not, which to me, is a good thing. When it gets better, it might start to do it, I don't know. People here live in a bubble where they think the world is full of people who read 'beautiful code', make tests, use git or something instead of zip$date and know how DeMorgan works; by far, most don't, not juniors, not seniors.
  - nottorp 2y ago
    > to add features or fix bugs, ifs were added. "Back to that two page function. Yes, I know, it’s just a simple function to display a window, but it has grown little hairs and stuff on it and nobody knows why. Well, I’ll tell you why: those are bug fixes. One of them fixes that bug that Nancy had when she tried to install the thing on a computer that didn’t have Internet Explorer. Another one fixes that bug that occurs in low memory conditions. Another one fixes that bug that occurred when the file is on a floppy disk and the user yanks out the disk in the middle. " https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-... Of course, you can try to split the 15000 line function in logical blocks or something, but don't assume the ifs in it are useless.
nottorp 2y ago
> I haven't seen AI deliver illogical nonsense I have. If you're doing niche-er stuff it doesn't have enough data and hallucinates. The worst is when it spits two screens of code instead of 'this cannot be done at the level you want it'. > that I couldn't even decipher That's unrelated to code quality. Especially with C++ which has become as write only as perl.

sgarland 2y ago

> Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. Thank you, this says what I have been struggling to describe. The day I lost part of my soul was when I asked a dev if I could give them feedback on a DB schema, they said yes, and then cut me off a few minutes in with, “yeah, I don’t really care [about X].” You don’t care? I’m telling you as the SME for this exactly what can be improved, how to do it, and why you should do so, but you don’t care. Cool. Cloud was a mistake; it’s inculcated people with the idea that chasing efficiency and optimization doesn’t matter, because you can always scale up or out. I’m not even talking about doing micro-benchmarks (though you should…), I’m talking about dead-simple stuff like “maybe use this data structure instead of that one.”

dartos 2y ago
I’ve noticed this more and more since 2020. A lot of people who entered the field in the past 6 or so years are here for the money, obviously. Nothing wrong with that at all, but as someone with a long time programming and technology passion, it’s sad to see that change.
- sgarland 2y ago
  There’s certainly nothing wrong with enjoying the high pay, no – I definitely do. But yeah, it’s upsetting to find out how few people care. Even moreso when they double down and say that you shouldn’t care either, because it’s been abstracted away, blah blah blah. Who do you think is going to continue creating these abstractions for you?
  - ebiester 2y ago
    ...Someone else who thinks of actual web applications as an abstraction. In the olden days, we used to throw it over to "Ops" and say, "your problem now." And Junior developers have always been overwhelmed with the details and under pressure to deliver enough to keep their job. None of this is new! I'm a graybeard now, but I remember seniors having the same complaints back then. "Kids these days" never gets old.

nerdjon 2y ago

I am curious about this also. I have now seen multiple PR's that I had to review that a method was clearly completely modified by AI with no good reason and when asked why something was changed we just got silence. Not exaggerating, literal silence and then trying to ignore the question and explain the thing we were first asking them to do. Clearly having no idea what is actually in this PR. This was done because we asked for a minor change to be done (talking maybe 5 lines of code) and tested. So now not only are we dealing with new debt, we are dealing with code that no one can explain why it was completely changed (and some of the changes were changes for the sake of change), and we are dealing with those of us that manage this code now looking at completely foreign code. I keep seeing this with people that are using these tools and they are not higher level engineers. We finally got to the point of denying these PR's and saying to go back and do it again. Loosing any of the time that was theoretically gained from doing it in the first place. Not saying these tools don't have a place. But people are using it without understanding what it is putting out and not understanding the long term effects it will have on a code base.

okwhateverdude 2y ago
> Not saying these tools don't have a place. But people are using it without understanding what it is putting out and not understanding the long term effects it will have on a code base. It is worse than that. We're all maintaining in our heads the mental sand castle that is the system the code base represents. The abuse of the autocoder erodes that sand castle because the intentions of the changes, which are crucial for mentally updating the sand castle, are not communicated (because they are unknowable). This is same thing with poor commit messages, or poor documentation around requirements/business processes. With enough erosion, plus expected turn over in staff, the sand castle is actually gone.
- rongenre 2y ago
  There needs to be a pattern for including AI-generated code, recording all the context which led to the generation.
  - 2y ago

anonzzzies 2y ago

> Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. I found this too. But I also found the opposite, including here on HN; people who are interested in technology have almost an aversion against using AI. I personally love tech and I would and do write software for fun, but even that is objectively more fun for me with AI. It makes me far more productive (very much more than what the article states) and, more importantly, it removes the procrastination; whenever I am stuck or procrastinating getting to work or start, I start talking with Aider and before I know it, another task was done that I probably wouldn't have done that day without. That way I now launch bi weekly open and closed source projects while before that would take months to years. And the cost of having this team of fast experienced devs sitting with me is max a few $ per day.

andybak 2y ago
Yeah. There is a psychological benefit to using AI that I find very beneficial. A lot of tasks that I would have avoided or wasted time doing task avoidance on suddenly become tractable. I think Simon Willison said something similar.
- jcgrillo 2y ago
  Are you just working on personal projects or in a shared codebase with tens or hundreds of other devs? If the latter, how do you keep your AI generated content from turning the whole thing into an incomprehensible superfund site of tech debt? I've gotten a lot of mileage in my career so far by paying attention when something felt tedious or mundane, because that's a signal you need some combination of refactoring, tooling, or automation. If instead you just lean on an LLM to brute-force your way through, sure, that accomplishes the short term goal of shipping your agile sprint deliverable or whatever, but what of the long term cost?
  - andybak 2y ago
    > Are you just working on personal projects or in a shared codebase with tens or hundreds of other devs? Like - I presume almost everyone - somewhere in the middle? That was a helluva dichotomy to offer me... > how do you keep your AI generated content from turning the whole thing into an incomprehensible superfund site of tech debt? By reading it, thinking about it and testing it? Did I somehow give the impression I'm cutting and pasting huge globs of code straight from ChatGPT into a git commit? There's a weird gulf of incomprehension between people that use AI to help them code and those that don't. I'm sure you're as confused by this exchange as I am.

infecto 2y ago

I hear this but I don't think this is a new issue that AI brought, it simply magnified it. That's a company culture issue. It reminds me of a talk Raymond Hettinger put on a while ago about rearranging the flowers in the garden. There is a tendency from new developers to rearrange for no good reason, AI makes it even easier now. This comes down to a culture problem to me, AI is simply the tool but the driver is the human (at least for now).

noobermin 2y ago

It doesn't, it looks at PRs, commits, and builds only. They remark at this lacking context.

fer 2y ago

Disclaimer: I work at a company who sells coding AI (among many other things). We use it internally and the technical debt is an enormous threat that IMO hasn't been properly gauged. It's very very useful to carpet bomb code with APIs and patterns you're not familiar with, but it also leads to insane amounts of code duplication and unwieldy boilerplate if you're not careful, because: 1. One of the two big bias of the models is the fact that the training data is StackOverflow-type training data, which are examples and don't take context and constraints into account. 2. The other is the existing codebase, and it tends to copy/repeat things instead of suggesting you to refactor. The first is mitigated by, well, doing your job and reviewing/editing what the LLM spat out. The second can only be mitigated once diffs/commit history become part of the training data, and that's a much harder dataset to handle and tag, as some changes are good (refactorings) but other might be not (bugs that get corrected in subsequent commits) and no clear distinction as commit messages are effectively lies (nobody ever writes: bug introduced). Not only that, merges/rebases/squashes alter/remove/add spurious meanings to the history, making everything blurrier.

trashtester 2y ago
If you're in a company that is valued at $10M+ per developer, the technical debt is not a big concern. Either you will go bust, OR you will be able to hire enough people to pay those debts, once you get traction in the market.
dboreham 2y ago
Consider myself very fortunate to have lived long enough that I'm reading a thread where the subject is the quality of the code generated by software. Decades of keeping that lollypop ready to be given, and now look where we are!
- samatman 2y ago
  The rate at which I quote Perlis 93 is a roughly quadratic function of the number of quarters since coding LLMs were first deployed.
disconcision 2y ago

kraftman 2y ago

> Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them though. Is this a bad thing? Maybe I'm misunderstanding it, but even when I'm working on my own projects, I'm usually trying to solve a problem, and the technology is a means to an end to solving that problem (delivering). I care that it works, and is maintainable, I don't care that much about the technology.

never_inline 2y ago
Read it as "closing a ticket" that makes sense. They don't care if the code explodes after the sprint.
layer8 2y ago
Caring that it works and is maintainable is caring about the technology.
- collyw 2y ago
  No code solutions are often superior. Around 15 years ago we were shipping terabyte hard disks as it was faster than the internet (until one got stuck in customs)

add-sub-mul-div 2y ago

Maintaining bad AI code is the new maintaining bad offshore code.

acedTrex 2y ago

> Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them though. Bingo, this, so much this. Every dev i know who loves AI stuff was a dev that I had very little technical respect for pre AI. They got some stuff done but there was no craft or quality to it.

chasd00 2y ago
i'm certainly skeptical of genai but your argument sounds very familiar to assembler dev feelings towards c++ devs back in the day.
- acedTrex 2y ago
  I am not necessarily arguing against GenAI. I am sure it will have somewhat similar effects to how the explosion in popularity of garbage collected languages et al had on software back in the 90s. More stuff will get done, the barrier of entry will be lower etc. The craft of programming took a significant quality/care hit when it transitioned from "only people who care enough to learn the ins and outs of memory management can feasibly do this" to "now anyone with a technical brain and a business use case can do it". Which makes sense, the code was no longer the point. The C++ devs rightly felt superior to the new java devs in the narrow niche of "ability to craft code." But that feeling doesn't move the needle business wise in the vast majority of circumstances. Which is always the schism between large technology leaps. Basically, the argument of "its worse" is not WRONG. Just, the same as it did not really matter in the mid 90s. Does not matter as much now, compared to the ability to "just get something that kinda works."
wahnfrieden 2y ago
You're playing status games
- acedTrex 2y ago
  I mean... in a way yes. The status of being someone who cares about the craft of programming. In the scheme of things however, that status hardly matters compared to the "ability to get something shipped quickly" which is what the vast majority of people are paid to do. So while I might judge those people for not meeting my personal standards or bar. In many cases that does not actually matter. They got something out there, thats all that matters.

jacobsenscott 2y ago

> Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them though. You get what you measure. Nobody measure software quality.

wahnfrieden 2y ago
Maybe not at your workplaces, but at mine, we measured bugs, change failure rate, uptime, "critical functionality" uptime, regressions, performance, CSAT, etc. in addition to qualitative research on quality in-team and with customers
- jacobsenscott 2y ago
  That's 0.001% of the workplaces out there. But good for them. Do you think it actually improved software quality, or was it all busy work?
  - wahnfrieden 2y ago
    Yah

prophesi 2y ago

From my experience, most of it is quickly caught in code review. And after a while it occurs less and less, granted that the junior developer puts in the effort to learn why their PRs aren't getting approved. So, pretty similar to how it was before. Except that motivated junior developers will improve incredibly fast. But that's also kind of always been the case in software development these past two decades?

YeGoblynQueenne 2y ago
It's interesting to see your comment currently right next to that of nerdjon here: https://news.ycombinator.com/item?id=41465827 https://news.ycombinator.com/item?id=41465827 (the two comments might have moved apart by the time you read this). Edit: yep, they just did.

Kiro 2y ago

I don't think it's that clear cut. I personally think the AI often delivers a better solution than the one I had in mind. It always contains a lot more safe guards against edge cases and other "boring" stuff that the AI has no problem adding but others find tedious.

trashtester 2y ago
If you're building a code base where AI is delivering on the the details of it, it's generally a bad thing if the code provided by AI provide safeguards WITHIN your code base. Those kinds of safeguards should instead be part of the framework you're using. If you need to prevent SQL injection, you need to make sure that all access to the SQL type database pass through a layer that prevents that. If you are worried about the security of your point of access (like an API facing the public), you need to apply safeguards as close to the point of entry as possible, and so on. I'm a big believer in AI generated code (over a long horizon), but I'm not sure the edge case robustness is the main selling point.
- Kiro 2y ago
  Sounds like we're talking about different kind of safeguards. I mean stuff like a character in a game ending up in a broken case due to something that is theoretically possible but very unlikely or where the consequence is not worth the effort. An AI has no problem taking those into account and write tedious safeguards, while I skip it.

godelski 2y ago

> I wonder if the study includes the technical debt that more experienced developers had to tackle after the less experienced devs have contributed their AI-driven efforts. It does not You also may find this post from the other day more illuminating[0], as I believe the actual result strongly hints at what you're guessing. The study is high schoolers doing math. While GPT only has an 8% error rate for the final answer, it gets the steps wrong half the time. And with coding (like math), the steps are the important bits. But I think people evaluate very poorly when there's ill defined metrics but some metric exists. They over inflate it's value since it's concrete. Like completing a ticket doesn't mean you made progress. Introducing technical debt would mean taking a step back. A step forward in a very specific direction but away from the actual end goal. You're just outsourcing work to a future person and I think we like to pretend this doesn't exist because it's hard to measure. [0] https://news.ycombinator.com/item?id=41453300 https://news.ycombinator.com/item?id=41453300

mrighele 2y ago

> Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them though. For them programming is a means to an end, and I think it is fine, in a way. But you cannot just ask an AI to write you tiktok clone and expect to get the finished product. Writing software is an iterative process, and LLMs currently used are not good enough for that, because they need not only to answer the questions, but at the very minimum to start asking questions: "why do want to do that ?" "do you prefer this or that", etc., so that they can actually extract all the specification details that the user happily didn't even know he needed before producing an appropriate output. (It's not too different from how some independent developers has to handle their clients, isn't it ?). Probably we will get there, but not too soon. I also doubt that current tools can keep a project architecturally sound long-term, but that is just an hunch. I admit though that I may be biased because I don't like much tools like copilot: when I write software, I have in my mind a model of the software that I am writing/I want to write, the AI has another model "in mind" and I need to spend mental energy understanding what it is "thinking". Even if 99/100 it is what I wanted, the remaining 1% is enough to hold me back from trusting it. Maybe I am using it the wrong way, who knows. The AI tool that work for me would be a "voice controller AI powered pair programmer": I write my code, then from time to time I ask him questions on how to do something, and I can get either an contextual answer depending on the code I am working on, or generate the actual code if I wish so". Are there already plugins working that way for vscode/idea/etc ?

liminalsunset 2y ago
I've been playing with Cursor (albeit with a very small toy codebase) and it does seem like it could do some of what you said - it has a number of features, not all of which necessarily generate code. You can ask questions about the code, about documentation, and other things, and it can optionally suggest code that you can either accept, accept parts of, or decline. It's more of a fork of vscode than a plugin right now though. It is very nice in that it gives you a handy diffing tool before you accept, and it very much feels like it puts me in control.
- mrighele 2y ago

creativeSlumber 2y ago

This. Even without AI, we have inexperienced developers rolling out something that "just works" without thinking about many of the scaling/availability issues. Then you have to spend 10x the time fixing those issues.

Sakos 2y ago
If inexperienced developers can commit code like that to master, then there's a culture issue, a process issue or both. This isn't a problem with AI.

deleted 2y ago

[deleted]

theflyinghorse 2y ago

> I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. If I don't deliver my startup burns in a year. In my previous role if I didn't deliver the people who were my reports did not get their bonuses. The incentives are very clear, and have always been clear - deliver.

rurp 2y ago
Succesful companies aren't built in a sprint. I doubt there has ever been a successful startup that didn't have at least some competent people thinking a number of steps ahead. Piling up tech debt to hit some short term arbitrary goals is not a good plan for real success.
- theflyinghorse 2y ago
  These are platitudes. Here's a real example of delivering something now without worrying about being the best engineer I can. I have 2 CSRs, They are swamped with work and we're weeks away from bringing another CSR on board. I find a couple of time consuming tasks that are easy to automate and build those out separately as one-off jobs that work well enough. Instantly it's a solid time gain & stress reducer to CSRs. Are my one-off automation tasks a long term solution? No. Do I care? Not at the moment, and my ego can take a hit for the time being.

jonnycomputer 2y ago

Code quality is the hardest thing to measure. Seems like they were measuring commits, pull-requests, builds, and build success rate. This sort of gets at that, but is probably inadequate. The few attempts I've made at using genAI to make large-scale changes to code have been failures, and left me in the dark about the changes that were made in ways that were not helpful. I needed suggestions to be in much smaller chunks. paragraph sized. Right now I limit myself to using the genAI line completion suggestions in Pycharm. It very often guesses my intentions and so actually is helpful, particularly when laboriously typing out lots of long literals, e.g. keys in a dictionary.

collyw 2y ago
WTFs per minute is the standard way of measuring code quality. Lower being better.

elric 2y ago

I don't remember who said it, but "AI generated code turns every developer into a legacy code maintainer". It's pithy and a bit of an exaggeration, but there's a grain of truth in there that resonates with me.

EvkoGS 2y ago

>Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them though. You are not a fucking priest in the temple of engineering, go to fucking CS dep at the local uni and be the one and preach it there. You are worker of the company with customers, which pays you a salary from customers money.