5 ms·

Except... no one validates the generated tickets, and it's full of inaccuracies. And then someone copy pastes it into Claude and now those inaccuracies become

Except... no one validates the generated tickets, and it's full of inaccuracies. And then someone copy pastes it into Claude and now those inaccuracies become part of the code and tests.

satvikpendem 5mo ago
The PMs validate it, why do you think they don't read over it to make sure it fits what they want? You might say "well they're lazy, look why they didn't write enough detail to start off with" but for lots of people, reviewing something to make sure it's close to what they want and then tweaking it is much easier than writing it from scratch. It's the equivalent of writer's block and is why a common advice given to writers is to put anything they can onto the page then edit it later.
- BugsJustFindMe 5mo ago
  > The PMs validate it, why do you think they don't read over it to make sure it fits what they want? Hahahahahaha. Sorry, I couldn't help myself; this reads like satire. The answer is "real life experience says otherwise".
  - ethin 5mo ago
    Yeah I was so tempted to ask if this person has ever actually met a project/product manager...
    - satvikpendem 5mo ago
      Maybe you both just have bad PMs, because just like good devs they should also be reviewing their work. My point was that it is more likely for PMs to review and edit a generated ticket than to have to write it all themselves which they often won't do.
      - BugsJustFindMe 5mo ago
        > My point was that it is more likely for PMs to I feel compelled to point out to you that this is a completely unsustainable, unsupportable, unsubstantiable claim. You have met ~0% of PMs, and of the ones you've met maybe you've experienced a non-zero percentage of their work, but statistically that's also very unlikely. If you think you can say what most PMs do or what PMs are likely to do, then, I'm sorry, but you are not even thinking like an engineer. You're thinking, actually, a lot more like a PM to many of us. > just like good devs I'm so sorry, my sides just can't handle the starry-eyed nature of these takes. This is just too much for me. To many of us this reads like you've never met people before. But who knows, maybe you live in Lake Wobegon, where all the women are strong, all the men are good-looking, and all the children are above average! If so then we're jealous, but you still should be more careful about how unrigorous your mental model is because it will make you a worse engineer. Experience with different PMs and developers aside, the older you get in the profession the more you will hopefully realize that none of your quality effort fantasy matters. Sales happen and money rolls in independently of whether you think the PMs or the people who call themselves engineers do a "good job". Businesses thrive on sales and marketing, not engineering.
        satvikpendem 5mo ago
        What a strange response. By your logic you've met ~0% of developers too yet I assume you can distinguish good development practices from bad. I also mentioned good PMs which by definition review and write good tickets with a clear explanation of the problem and what they want the solution to be. If personally meeting millions of people is the epistemic standard you have to know something then I'm not sure how you know anything at all. As to your latter point, not sure why you think I think business doesn't continue on even with bad employees, of course it does and I didn't say otherwise. But that does not mean they're doing a good job, those two are orthogonal concepts. And I'm not sure how we even got to this, the original point was that I personally as a dev can physically see PM productivity increasing with AI, even as other devs in this thread seem not to. For a competent PM, a tool that automates a detailed first draft fundamentally changes the psychology of ticket creation. If your argument is just "bad PMs will still be bad," then sure, I agree, but that doesn't really engage with how the tooling changes the workflow for everyone else.
- majormajor 5mo ago
  > The PMs validate it, why do you think they don't read over it to make sure it fits what they want? The PM has historically often not had a detailed enough mental model of the implementation to spot the hard parts in advance or a detailed enough mental model of the customer desires to know if it's gonna be the right thing or not. Those are the things that killed waterfall. You can use LLM tools to help you improve both those areas. Synthesizing large amounts of text and looking for inconsistencies. But the 80th-percentile-or-lower person who was already not working hard to try to get ahead of those things still isn't going to work any harder than the next person and so won't gain much of a real edge.
  - Avicebron 5mo ago
    I'm glad you mentioned it and TFA briefly mentioned waterfall. The second graph shown in the article with documentation overlapping the dev cycle, it's like the worse of both agile and waterfall. It's supposedly real-time waterfall. Normally waterfall works where the scope is extremely-well defined and articulated in design plans. Which shortens dev time because prior to AI code was mostly deterministic. Here we have to do waterfall level of documentation while iterating on a non-deterministic solution (code gen) to non-deterministic requirements (per usual). It's bonkers. I still think the technology is cool though. And to answer the questioner.. Have you worked with a PM? Most of the ones I've worked with try to be simultaneously in charge yet not responsible for anything. Validating something implies skill and responsibility.
  - satvikpendem 5mo ago
    Then they're just bad PMs and don't deserve to have the job. That can be said in any profession, devs or lawyers or doctors who blindly accept LLM output without review are bad employees.
    - BugsJustFindMe 5mo ago
      > Then they're just bad PMs and don't deserve to have the job. Nobody "deserves" anything. They do have the jobs though. Thinking that the world isn't full of people doing what they need to do to get by who don't give a shit about fitting a fantasy ideal is wild.
- zxornand 5mo ago
  I think validating a fully generated novel of a ticket, is much harder than thinking through the problem in the first place and creating your own ticket. We see it with code too right? It’s harder to review code than to write it. On top of that the LLM can work so fast that the amount of things that need validating grows! This is where humans get lazy and the problems come in IMO. Whether its a PM not validating their ticket, or a dev doing a bad code review. Add on to that that the incentives currently are to move fast and trust the AI. It becomes clear to me that a lot of that review work either won’t be done at all, or won’t be nearly thorough enough.
  - paulhebert 5mo ago
    I agree. I hate getting tickets like this because they’ve often gone down the wrong path and I have to work backwards to understand the actual problem and the right way to solve it
  - satvikpendem 5mo ago
    The tickets are not "novel"-length, they are about a few bulleted lists of the sections I mentioned above. In that case it is indeed way easier to review that a ticket only saying "do X with Y data." Reviewing code is harder than reviewing text because code does something and has interdependencies and therefore must be correct in its function, do not mix the two. This is like saying an editor reviewing an article or novel is harder than actually writing the novel which is blatantly incorrect.
    - zxornand 5mo ago
      Most real tickets are more complicated than “Do x with Y data” and also have many interdependencies throughout the business
      - satvikpendem 5mo ago
        Most? That's doubtful especially when a lot of tickets are simply CRUD which are fine being generated by an LLM. Those that are more complex require more review and interdependency management, sure, but to say that that is most tickets is simply not correct.
- mrbombastic 5mo ago
  just this week i pushed back on some requirements in a very detailed product spec I was implementing to speed up time to ship. The pm had no idea what I was talking about because the requirements were invented by an LLM. This is not a bad PM, discipline doesn't scale.
resters 5mo ago
This failure is human laziness, not an issue with the technology. People who use AI because they are trying to avoid doing work fall into a completely different category than people using AI as a force multiplier and for skills/capabilities enhancements / quality improvement.
- OptionOfT 5mo ago
  It's also the only way to get those massive increases in productivity.
- danaris 5mo ago
  This is very much a "you're holding it wrong" response. If your technology relies on humans using it in ways that go against the ways they are inclined to use them, then that is an issue with the technology.
  - majormajor 5mo ago
    I don't think that works as a critique of LLMs because it's far too broadly applicable to well-accepted tools. Are advanced calculators bad because a student could use the CAS to ace calculus homework, exams or the SAT without actually learning the material? Is copy/paste bad because a person could use it to copy/paste code from one place to another without noticing some of the areas they need to update in the new location, adding bugs and missing a chance to learn some more subtleties of the system? Is Git bad because a manager could use it to just measure performance by number of lines of code committed instead of doing more work to actually understand everyone's performance? Many tools can be used lazily in ways that will directly work against a long term goal of improving knowledge and productivity.
    - convolvatron 5mo ago
      but in this case that's exactly what AI is doing, and no more. its filling in the gaps with some plausible sounding goo so that the person doesn't have to worry about the details. ok, so for some of the jobs we're doing plausible sounding goo is just fine. and that's kinda sad. but the 'just playing around' case is fine for PSG, this isn't a serious effort but just seeing how things might work out without much effort. taking the remainder, where understanding and intent are important, the role of the ai is produce PSG, but the intentional person now goes through everything and plucks out all the nonsense. this may take more or less time than simply writing it, but we should understand this is resulting in less real engagement by the ultimate author. where this is actually interesting is a parallel to Burrough's cutup method - where source text and audio were randomly scrambled and sometimes really clever and novel stuff pops out. but to say the current model of vibe coding has much to offer in the second case is really quite unclear. to the extent to which coding is the production of boilerplate is really a problem with APIs and abstraction design. if we can get LLMs to mitigate some of that I the short term without causing too much distraction, that's fine, but we should really be using that to inform the solution to the fundamental problem. so for me what's missing in your model is how LLMs are supposed to be used 'properly'. I don't think laziness is really the right cut here, make-work is make-work, and there's plenty of real work to be done. but in what sense does LLM usage for code actually improve our understanding of these systems and get us more agency?

Except... no one validates the generated tickets, and it's full of inaccuracies. And then someone copy pastes it into Claude and now those inaccuracies become part of the code and tests.

satvikpendem 5mo ago

The PMs validate it, why do you think they don't read over it to make sure it fits what they want? You might say "well they're lazy, look why they didn't write enough detail to start off with" but for lots of people, reviewing something to make sure it's close to what they want and then tweaking it is much easier than writing it from scratch. It's the equivalent of writer's block and is why a common advice given to writers is to put anything they can onto the page then edit it later.

BugsJustFindMe 5mo ago
> The PMs validate it, why do you think they don't read over it to make sure it fits what they want? Hahahahahaha. Sorry, I couldn't help myself; this reads like satire. The answer is "real life experience says otherwise".
- ethin 5mo ago
  Yeah I was so tempted to ask if this person has ever actually met a project/product manager...
  - satvikpendem 5mo ago
    Maybe you both just have bad PMs, because just like good devs they should also be reviewing their work. My point was that it is more likely for PMs to review and edit a generated ticket than to have to write it all themselves which they often won't do.
    - BugsJustFindMe 5mo ago
      > My point was that it is more likely for PMs to I feel compelled to point out to you that this is a completely unsustainable, unsupportable, unsubstantiable claim. You have met ~0% of PMs, and of the ones you've met maybe you've experienced a non-zero percentage of their work, but statistically that's also very unlikely. If you think you can say what most PMs do or what PMs are likely to do, then, I'm sorry, but you are not even thinking like an engineer. You're thinking, actually, a lot more like a PM to many of us. > just like good devs I'm so sorry, my sides just can't handle the starry-eyed nature of these takes. This is just too much for me. To many of us this reads like you've never met people before. But who knows, maybe you live in Lake Wobegon, where all the women are strong, all the men are good-looking, and all the children are above average! If so then we're jealous, but you still should be more careful about how unrigorous your mental model is because it will make you a worse engineer. Experience with different PMs and developers aside, the older you get in the profession the more you will hopefully realize that none of your quality effort fantasy matters. Sales happen and money rolls in independently of whether you think the PMs or the people who call themselves engineers do a "good job". Businesses thrive on sales and marketing, not engineering.
      - satvikpendem 5mo ago
        What a strange response. By your logic you've met ~0% of developers too yet I assume you can distinguish good development practices from bad. I also mentioned good PMs which by definition review and write good tickets with a clear explanation of the problem and what they want the solution to be. If personally meeting millions of people is the epistemic standard you have to know something then I'm not sure how you know anything at all. As to your latter point, not sure why you think I think business doesn't continue on even with bad employees, of course it does and I didn't say otherwise. But that does not mean they're doing a good job, those two are orthogonal concepts. And I'm not sure how we even got to this, the original point was that I personally as a dev can physically see PM productivity increasing with AI, even as other devs in this thread seem not to. For a competent PM, a tool that automates a detailed first draft fundamentally changes the psychology of ticket creation. If your argument is just "bad PMs will still be bad," then sure, I agree, but that doesn't really engage with how the tooling changes the workflow for everyone else.
majormajor 5mo ago
> The PMs validate it, why do you think they don't read over it to make sure it fits what they want? The PM has historically often not had a detailed enough mental model of the implementation to spot the hard parts in advance or a detailed enough mental model of the customer desires to know if it's gonna be the right thing or not. Those are the things that killed waterfall. You can use LLM tools to help you improve both those areas. Synthesizing large amounts of text and looking for inconsistencies. But the 80th-percentile-or-lower person who was already not working hard to try to get ahead of those things still isn't going to work any harder than the next person and so won't gain much of a real edge.
- Avicebron 5mo ago
  I'm glad you mentioned it and TFA briefly mentioned waterfall. The second graph shown in the article with documentation overlapping the dev cycle, it's like the worse of both agile and waterfall. It's supposedly real-time waterfall. Normally waterfall works where the scope is extremely-well defined and articulated in design plans. Which shortens dev time because prior to AI code was mostly deterministic. Here we have to do waterfall level of documentation while iterating on a non-deterministic solution (code gen) to non-deterministic requirements (per usual). It's bonkers. I still think the technology is cool though. And to answer the questioner.. Have you worked with a PM? Most of the ones I've worked with try to be simultaneously in charge yet not responsible for anything. Validating something implies skill and responsibility.
- satvikpendem 5mo ago
  Then they're just bad PMs and don't deserve to have the job. That can be said in any profession, devs or lawyers or doctors who blindly accept LLM output without review are bad employees.
  - BugsJustFindMe 5mo ago
    > Then they're just bad PMs and don't deserve to have the job. Nobody "deserves" anything. They do have the jobs though. Thinking that the world isn't full of people doing what they need to do to get by who don't give a shit about fitting a fantasy ideal is wild.
zxornand 5mo ago
I think validating a fully generated novel of a ticket, is much harder than thinking through the problem in the first place and creating your own ticket. We see it with code too right? It’s harder to review code than to write it. On top of that the LLM can work so fast that the amount of things that need validating grows! This is where humans get lazy and the problems come in IMO. Whether its a PM not validating their ticket, or a dev doing a bad code review. Add on to that that the incentives currently are to move fast and trust the AI. It becomes clear to me that a lot of that review work either won’t be done at all, or won’t be nearly thorough enough.
- paulhebert 5mo ago
  I agree. I hate getting tickets like this because they’ve often gone down the wrong path and I have to work backwards to understand the actual problem and the right way to solve it
- satvikpendem 5mo ago
  The tickets are not "novel"-length, they are about a few bulleted lists of the sections I mentioned above. In that case it is indeed way easier to review that a ticket only saying "do X with Y data." Reviewing code is harder than reviewing text because code does something and has interdependencies and therefore must be correct in its function, do not mix the two. This is like saying an editor reviewing an article or novel is harder than actually writing the novel which is blatantly incorrect.
  - zxornand 5mo ago
    Most real tickets are more complicated than “Do x with Y data” and also have many interdependencies throughout the business
    - satvikpendem 5mo ago
      Most? That's doubtful especially when a lot of tickets are simply CRUD which are fine being generated by an LLM. Those that are more complex require more review and interdependency management, sure, but to say that that is most tickets is simply not correct.
mrbombastic 5mo ago
just this week i pushed back on some requirements in a very detailed product spec I was implementing to speed up time to ship. The pm had no idea what I was talking about because the requirements were invented by an LLM. This is not a bad PM, discipline doesn't scale.

resters 5mo ago

This failure is human laziness, not an issue with the technology. People who use AI because they are trying to avoid doing work fall into a completely different category than people using AI as a force multiplier and for skills/capabilities enhancements / quality improvement.

OptionOfT 5mo ago
It's also the only way to get those massive increases in productivity.
danaris 5mo ago
This is very much a "you're holding it wrong" response. If your technology relies on humans using it in ways that go against the ways they are inclined to use them, then that is an issue with the technology.
- majormajor 5mo ago
  I don't think that works as a critique of LLMs because it's far too broadly applicable to well-accepted tools. Are advanced calculators bad because a student could use the CAS to ace calculus homework, exams or the SAT without actually learning the material? Is copy/paste bad because a person could use it to copy/paste code from one place to another without noticing some of the areas they need to update in the new location, adding bugs and missing a chance to learn some more subtleties of the system? Is Git bad because a manager could use it to just measure performance by number of lines of code committed instead of doing more work to actually understand everyone's performance? Many tools can be used lazily in ways that will directly work against a long term goal of improving knowledge and productivity.
  - convolvatron 5mo ago
    but in this case that's exactly what AI is doing, and no more. its filling in the gaps with some plausible sounding goo so that the person doesn't have to worry about the details. ok, so for some of the jobs we're doing plausible sounding goo is just fine. and that's kinda sad. but the 'just playing around' case is fine for PSG, this isn't a serious effort but just seeing how things might work out without much effort. taking the remainder, where understanding and intent are important, the role of the ai is produce PSG, but the intentional person now goes through everything and plucks out all the nonsense. this may take more or less time than simply writing it, but we should understand this is resulting in less real engagement by the ultimate author. where this is actually interesting is a parallel to Burrough's cutup method - where source text and audio were randomly scrambled and sometimes really clever and novel stuff pops out. but to say the current model of vibe coding has much to offer in the second case is really quite unclear. to the extent to which coding is the production of boilerplate is really a problem with APIs and abstraction design. if we can get LLMs to mitigate some of that I the short term without causing too much distraction, that's fine, but we should really be using that to inform the solution to the fundamental problem. so for me what's missing in your model is how LLMs are supposed to be used 'properly'. I don't think laziness is really the right cut here, make-work is make-work, and there's plenty of real work to be done. but in what sense does LLM usage for code actually improve our understanding of these systems and get us more agency?