4 ms·

I don't think "IDK Claude did that" is a valid excuse. Immediate rejection. AI may be multi-threaded, but there's still a human, global interpreter lock in pla

by aantix 1y ago

I don't think "IDK Claude did that" is a valid excuse. Immediate rejection. AI may be multi-threaded, but there's still a human, global interpreter lock in place. :D If you put the code up for review, regardless of the source, you should fundamentally understand how it works. This raises a broader point about AI and productivity: while AI promises parallelism, there's still the human in the middle who is responsible for the code. The promise of "parallelism" is overstated. 100's of PRs should not be trusted. Or at least not without the c-suite understanding such risks. Maybe you're a small startup looking to get out the door as quickly as possible, so.. YOLO. But it's going to be a hot mess. A "clean up in aisle nine" level mess.

ryandrake 1y ago
It's insane that any company would just be OK with "IDK Claude did that" any more than a 2010 version of that company would be OK with "IDK I copy pasted from StackOverflow." Have engineering managers actually drank this Kool-aid to the point where they're actually OK with their direct reports just chucking PRs over the wall that they don't even understand?
- pinoy420 1y ago
  [dead]
- throwawaysleep 1y ago
  Depends on your incentives. People anecdotally seem far more impressed with buggy stuff shipped fast than good stuff shipped slowly. Lots of companies just accept bugs as something that happens.
  - dontlikeyoueith 1y ago
    Depends on your problem space too. Calendar app for local social clubs? Ship it and fix it later. B2B payments software that triggers funds transfers? JFC I hope you PIP people for that.
- Imustaskforhelp 1y ago
  It is even more funnier when you realize that because Claude and all AI models are trained on data including stackoverflow. So I guess if you asked Claude why it did that, the truth of it might be "IDK I copy pasted from StackOverflow" The same stuff pasted with a different sticker. Looks good to me.
  - jaredcwhite 1y ago
    Ha, I genuinely laughed at that. Thank you!
- liveoneggs 1y ago
  Of course they are okay with it. They changed the job function to be just that with forced(!) AI adoption.
- f1shy 1y ago
  This is exactly how I see it. Is not about the tool, is how it is used. In 1990 that would have been “IDK I got it from a BBS” and in 1980 “got if from a magazine“. It doesn’t matter how you get there, you have to understand it. BTW I had a similar problem as I was manager in HW development, where the value of a resistor had no documented calculation. I would ask: where does it came from? If the answer was “I tried and it worked”, or “tested in lab until I found it” or in the 2000 “I run many simulations and was the best value” I would reject and ask for proper calculations, with WCA.
  - Andrex 1y ago
    As vibe coding becomes more commonplace you'll see these historical safeguards erode. That is the danger IMO. You're right, saying you got something off SO would get you laughed out of programming circles back in the day. We should be applying the same shame to people who vibe code, not encourage it, if we want human-parseable and maintainable software.
    - vkou 1y ago
      > That is the danger IMO. For whom is this a danger for? If we're paid to dig ditches and fill them, who are we to question our supreme leaders? They control the purse strings, so of course they know best.
      - nemesisithetic 1y ago
        > If we're paid to dig ditches and fill them This is a very cruel punishment sometimes used in forced labor camps. You are describing torture.
- j45 1y ago
  The same developers submitting Claude submissions can take 1-2 minutes asking for an explanation of what they're submitting and how it works. Might even learn. Stack Overflow had enough provenance of copying and pasting. Models may not. Provenance remains a thing or it can add risk to the code.
- joseda-hg 1y ago
  I don't think it's common, but I've definitely seen it I've also seen "Ask ChatGPT if you're doing X right?", and basically signing off whatever it recommends without checking At this point I'm pretty confident I could trojan horse whatever decision I want from certain people by sending enough screenshots of ChatGPT agreeing with me
- calebkaiser 1y ago
  I don't think this is an AI specific thing. I work in the field, and so I'm around some of the most enthusiastic adopters of LLMs, and from what I see, engineering cultures surrounding LLM usage typically match the org's previous general engineering culture. So, for example, by and large the orgs I've seen chucking Claude PRs over the wall with little review were previously chucking 100% human written PRs over the wall with little review. Similarly, the teams I see effectively using test suites to guide their code generation are the same teams that effectively use test suites to guide their general software engineering workflows.
  - throwanem 1y ago
    How long are you spending with a given team, and where per se in their "AI lifecycle?" I would expect (for example) a sales engineer to see this differently than a support engineer, if support engineers still existed.
- siva7 1y ago
  If it pushes some nice velocity metric, most managers would be ok. Though you have to word it a bit differently of course.
- throwanem 1y ago
  "Look, the build is green and CI belongs to another team, how perfectionist do you need us to be about this?" is the sort of response I would generally expect, and also in the case where AI was used.
- danielmarkbruce 1y ago
  What about "the compiler did that" ?
SoftTalker 1y ago
As someone who doesn't use AI for writing code, why can't you just ask Claude to write up an explanation of each change for code review? Then at least you can look at whether the explanation seems sane.
- threetonesun 1y ago
  It will fairly confidently state changes are "correct" for whatever reason it makes up. This becomes more of an issue with things that might be edge cases or vague requirements, in which case it's better to have AI write tests instead of the code.
- ahoef 1y ago
  Claude also doesn't know, because Claude dreamt up changes that didn't work, then "fixed" them, "fixed" them again and in the process left swathes of code that isn't reached.
- thegeomaster 1y ago
  This can be dangerous, because Claude doesn't truly understand why it did something. Whatever it writes a post-hoc justification which may or may not be accurate to the "intent". This is because these are still autoregressive models --- they have only the context to go on, not prior intent.
  - zahlman 1y ago
    Indeed. Watching it (well, Anthropic, really) cheat at Baba Is You and then try to give a rationalization for how it came up with the solution (qv. https://news.ycombinator.com/item?id=44473615 https://news.ycombinator.com/item?id=44473615) is quite instructive.
- zahlman 1y ago
  Because the explanations will often not be sane; when they are sane, they will focus on irrelevant details and be maddeningly padded out unless you put inordinate effort into trying to control the AI's writing style. Ask pretty much any FOSS developer who has received AI-generated (both code and explanations) PRs on GitHub (and when you complain about these, the author will almost always use the same AI to generate responses) about their experiences. It's a huge time sink if you don't cut them off. There are plenty of projects out there now that have explicit policy documentation against such submissions and even boilerplate messages for rejecting them.
NegativeLatency 1y ago
> I don't think "IDK Claude did that" is a valid excuse. It's not, and yet I have seen that offered as an excuse several times.
- grogenaut 1y ago
  Did you push back?
f1shy 1y ago
At least I would not accept from my team. Is borderline infuriating. And I would promptly insinuate, if that is the answer, next time I do not need you, I will ask directly Claude, you can stay home!
- Herring 1y ago
  Maybe check what's their workload otherwise. Most engineers I've worked with want to do a good job and ship something useful. It's possible they're offloading work to the LLM because they're under a lot of pressure. (And in this case you can't make them stay home)
corytheboyd 1y ago
> The promise of "parallelism" is overstated. 100% my takeaway after trying to parallelize using worktrees. While Claude has no problem managing more than one context instance, I sure as hell do. It’s exhausting, to the point of slowing me down.
- vouaobrasil 1y ago
  That's an intended effect. It doesn't matter to those in power who know what AI is really for. Once you get so exhausted that you can't work any more, there will be a hundred bright-eyed naïve programmers who will step into your place and who think they can do better. Until they burn out in a few years time.
  - corytheboyd 1y ago
    I have been wondering when I would start to feel aged out of the tech industry… gosh is it here already?
    - vouaobrasil 1y ago
      I don't know if it is but I'm certainly glad I left tech a long time ago...
ToucanLoucan 1y ago
> If you put the code up for review, regardless of the source, you should fundamentally understand how it works. Inb4 the chorus of whining from AI hypists accusing you of being an coastal elitist intellectual jerk for daring to ask that they might want to LEARN something. I am so over this anti-intellectual garbage. It's gotten to such a ridiculous place in our society and is literally going to get tons of people killed.
- zahlman 1y ago
  I understand and agree with your frustration, but this is not what discourse here is supposed to look like.
j-bos 1y ago
> I don't think "IDK Claude did that" is a valid excuse. Immediate rejection. I strongly agree, however manager^x do not and want see report the massive "productivity" gains.
- Izikiel43 1y ago
  You tell them clippy’s revengeance pr caused an outage worth millions of dollars because of push for productivity and they shouldn’t bother you for a couple of months.
- dingnuts 1y ago
  Sane CTOs think "Claude did that" is invalid. I assure you: those leaders exist. Refuse to work for idiots who think bots can be held accountable. You must must understand every line of code yourself. "Claude did that" is functionally equivalent to "idk I copied that from r/programming" and is totally unacceptable for a professional
  - derf_ 1y ago
    > You must must understand every line of code yourself. I have never seen this standard reached for any real codebase of any size. Even in projects with a reputation for a strong review culture, people know who the "easy" reviewers are and target them for the dicey stuff (they are often the most overloaded... which only causes them to get more overloaded). I've seen people explicitly state they are just "rubber stamping" PRs. Literally no one reviews every line of third-party dependencies, and especially not when they are updated routinely. I've seen over a million lines of security-sensitive third-party code integrated and pushed out to hundreds of millions of users by a handful of developers in a matter of months. I've seen developers write their new green-field code as a third-party library to circumvent the review process that would have been applied if it had been developed as a series of first-party PRs. None of that had anything to do with AI. It all predated AI coding tools. That is how humans behave. Does this create ticking time-bombs? It absolutely does. You do the best you can. You triage and deal with the most important things according to your best judgment, and circle back to the rest as time and attention allow. If your judgment is good, it's mostly okay. Some day it might not be. But I do not think that you can argue that the optimal level of risk is zero, outside of a few specialized contexts like space shuttles and nuclear reactors. I know. It hurts my soul, too. But reality isn't pretty, and worse is better.
kibwen 1y ago
> I don't think "IDK Claude did that" is a valid excuse. Immediate rejection. That will work, but only until the people filing these PRs go crying to their managers that you refuse to merge any of their code, at which point you'll be given a stern reprimand from your betters to stop being so picky. Have fun vibe-reviewing.
pkdpic 1y ago
Yeah in a perfect world I absolutely agree. But the reality I'm observing is that everything continues to always be behind schedule (not a new phenomena) and if anything expectations from project leads / management are just getting less realistic leaving Jr devs in even more of a position of "no time to be curious or learn deeply just get it done" and teamleads reviewing PRs in a maybe even worse position of "no time to get into a deep review / mentorship session just figure out if it breaks anything or not". And ultimately clients are still just living in fantasy land in terms of expectations both in terms of build out time for basic features / patches and also how much AI razzle-dazzle they expect for new project proposals. Nothing can move fast enough to keep up with these hype-fueled TED talk expectations all the way up the chain. I don't know if there's any solution and I'm sure it's not like this everywhere but I'm also sure I'm not alone. At this point I'm just trying to keep my feet wet on "AI" related projects until the hype dust settles so I can reassess what this industry even is anymore. Maybe it's not too late to get a single subject credential and go teach math or finger painting or something.

by aantix 1y ago

ryandrake 1y ago

It's insane that any company would just be OK with "IDK Claude did that" any more than a 2010 version of that company would be OK with "IDK I copy pasted from StackOverflow." Have engineering managers actually drank this Kool-aid to the point where they're actually OK with their direct reports just chucking PRs over the wall that they don't even understand?

pinoy420 1y ago
[dead]
throwawaysleep 1y ago
Depends on your incentives. People anecdotally seem far more impressed with buggy stuff shipped fast than good stuff shipped slowly. Lots of companies just accept bugs as something that happens.
- dontlikeyoueith 1y ago
  Depends on your problem space too. Calendar app for local social clubs? Ship it and fix it later. B2B payments software that triggers funds transfers? JFC I hope you PIP people for that.
Imustaskforhelp 1y ago
It is even more funnier when you realize that because Claude and all AI models are trained on data including stackoverflow. So I guess if you asked Claude why it did that, the truth of it might be "IDK I copy pasted from StackOverflow" The same stuff pasted with a different sticker. Looks good to me.
- jaredcwhite 1y ago
  Ha, I genuinely laughed at that. Thank you!
liveoneggs 1y ago
Of course they are okay with it. They changed the job function to be just that with forced(!) AI adoption.
f1shy 1y ago
This is exactly how I see it. Is not about the tool, is how it is used. In 1990 that would have been “IDK I got it from a BBS” and in 1980 “got if from a magazine“. It doesn’t matter how you get there, you have to understand it. BTW I had a similar problem as I was manager in HW development, where the value of a resistor had no documented calculation. I would ask: where does it came from? If the answer was “I tried and it worked”, or “tested in lab until I found it” or in the 2000 “I run many simulations and was the best value” I would reject and ask for proper calculations, with WCA.
- Andrex 1y ago
  As vibe coding becomes more commonplace you'll see these historical safeguards erode. That is the danger IMO. You're right, saying you got something off SO would get you laughed out of programming circles back in the day. We should be applying the same shame to people who vibe code, not encourage it, if we want human-parseable and maintainable software.
  - vkou 1y ago
    > That is the danger IMO. For whom is this a danger for? If we're paid to dig ditches and fill them, who are we to question our supreme leaders? They control the purse strings, so of course they know best.
    - nemesisithetic 1y ago
      > If we're paid to dig ditches and fill them This is a very cruel punishment sometimes used in forced labor camps. You are describing torture.
j45 1y ago
The same developers submitting Claude submissions can take 1-2 minutes asking for an explanation of what they're submitting and how it works. Might even learn. Stack Overflow had enough provenance of copying and pasting. Models may not. Provenance remains a thing or it can add risk to the code.
joseda-hg 1y ago
I don't think it's common, but I've definitely seen it I've also seen "Ask ChatGPT if you're doing X right?", and basically signing off whatever it recommends without checking At this point I'm pretty confident I could trojan horse whatever decision I want from certain people by sending enough screenshots of ChatGPT agreeing with me
calebkaiser 1y ago
I don't think this is an AI specific thing. I work in the field, and so I'm around some of the most enthusiastic adopters of LLMs, and from what I see, engineering cultures surrounding LLM usage typically match the org's previous general engineering culture. So, for example, by and large the orgs I've seen chucking Claude PRs over the wall with little review were previously chucking 100% human written PRs over the wall with little review. Similarly, the teams I see effectively using test suites to guide their code generation are the same teams that effectively use test suites to guide their general software engineering workflows.
- throwanem 1y ago
  How long are you spending with a given team, and where per se in their "AI lifecycle?" I would expect (for example) a sales engineer to see this differently than a support engineer, if support engineers still existed.
siva7 1y ago
If it pushes some nice velocity metric, most managers would be ok. Though you have to word it a bit differently of course.
throwanem 1y ago
"Look, the build is green and CI belongs to another team, how perfectionist do you need us to be about this?" is the sort of response I would generally expect, and also in the case where AI was used.
danielmarkbruce 1y ago
What about "the compiler did that" ?

SoftTalker 1y ago

As someone who doesn't use AI for writing code, why can't you just ask Claude to write up an explanation of each change for code review? Then at least you can look at whether the explanation seems sane.

threetonesun 1y ago
It will fairly confidently state changes are "correct" for whatever reason it makes up. This becomes more of an issue with things that might be edge cases or vague requirements, in which case it's better to have AI write tests instead of the code.
ahoef 1y ago
Claude also doesn't know, because Claude dreamt up changes that didn't work, then "fixed" them, "fixed" them again and in the process left swathes of code that isn't reached.
thegeomaster 1y ago
This can be dangerous, because Claude doesn't truly understand why it did something. Whatever it writes a post-hoc justification which may or may not be accurate to the "intent". This is because these are still autoregressive models --- they have only the context to go on, not prior intent.
- zahlman 1y ago
  Indeed. Watching it (well, Anthropic, really) cheat at Baba Is You and then try to give a rationalization for how it came up with the solution (qv. https://news.ycombinator.com/item?id=44473615 https://news.ycombinator.com/item?id=44473615) is quite instructive.
zahlman 1y ago
Because the explanations will often not be sane; when they are sane, they will focus on irrelevant details and be maddeningly padded out unless you put inordinate effort into trying to control the AI's writing style. Ask pretty much any FOSS developer who has received AI-generated (both code and explanations) PRs on GitHub (and when you complain about these, the author will almost always use the same AI to generate responses) about their experiences. It's a huge time sink if you don't cut them off. There are plenty of projects out there now that have explicit policy documentation against such submissions and even boilerplate messages for rejecting them.

NegativeLatency 1y ago

> I don't think "IDK Claude did that" is a valid excuse. It's not, and yet I have seen that offered as an excuse several times.

grogenaut 1y ago
Did you push back?

f1shy 1y ago

At least I would not accept from my team. Is borderline infuriating. And I would promptly insinuate, if that is the answer, next time I do not need you, I will ask directly Claude, you can stay home!

Herring 1y ago
Maybe check what's their workload otherwise. Most engineers I've worked with want to do a good job and ship something useful. It's possible they're offloading work to the LLM because they're under a lot of pressure. (And in this case you can't make them stay home)

corytheboyd 1y ago

> The promise of "parallelism" is overstated. 100% my takeaway after trying to parallelize using worktrees. While Claude has no problem managing more than one context instance, I sure as hell do. It’s exhausting, to the point of slowing me down.

vouaobrasil 1y ago
That's an intended effect. It doesn't matter to those in power who know what AI is really for. Once you get so exhausted that you can't work any more, there will be a hundred bright-eyed naïve programmers who will step into your place and who think they can do better. Until they burn out in a few years time.
- corytheboyd 1y ago
  I have been wondering when I would start to feel aged out of the tech industry… gosh is it here already?
  - vouaobrasil 1y ago
    I don't know if it is but I'm certainly glad I left tech a long time ago...

ToucanLoucan 1y ago

> If you put the code up for review, regardless of the source, you should fundamentally understand how it works. Inb4 the chorus of whining from AI hypists accusing you of being an coastal elitist intellectual jerk for daring to ask that they might want to LEARN something. I am so over this anti-intellectual garbage. It's gotten to such a ridiculous place in our society and is literally going to get tons of people killed.

zahlman 1y ago
I understand and agree with your frustration, but this is not what discourse here is supposed to look like.

j-bos 1y ago

> I don't think "IDK Claude did that" is a valid excuse. Immediate rejection. I strongly agree, however manager^x do not and want see report the massive "productivity" gains.

Izikiel43 1y ago
You tell them clippy’s revengeance pr caused an outage worth millions of dollars because of push for productivity and they shouldn’t bother you for a couple of months.
dingnuts 1y ago
Sane CTOs think "Claude did that" is invalid. I assure you: those leaders exist. Refuse to work for idiots who think bots can be held accountable. You must must understand every line of code yourself. "Claude did that" is functionally equivalent to "idk I copied that from r/programming" and is totally unacceptable for a professional
- derf_ 1y ago
  > You must must understand every line of code yourself. I have never seen this standard reached for any real codebase of any size. Even in projects with a reputation for a strong review culture, people know who the "easy" reviewers are and target them for the dicey stuff (they are often the most overloaded... which only causes them to get more overloaded). I've seen people explicitly state they are just "rubber stamping" PRs. Literally no one reviews every line of third-party dependencies, and especially not when they are updated routinely. I've seen over a million lines of security-sensitive third-party code integrated and pushed out to hundreds of millions of users by a handful of developers in a matter of months. I've seen developers write their new green-field code as a third-party library to circumvent the review process that would have been applied if it had been developed as a series of first-party PRs. None of that had anything to do with AI. It all predated AI coding tools. That is how humans behave. Does this create ticking time-bombs? It absolutely does. You do the best you can. You triage and deal with the most important things according to your best judgment, and circle back to the rest as time and attention allow. If your judgment is good, it's mostly okay. Some day it might not be. But I do not think that you can argue that the optimal level of risk is zero, outside of a few specialized contexts like space shuttles and nuclear reactors. I know. It hurts my soul, too. But reality isn't pretty, and worse is better.

kibwen 1y ago

> I don't think "IDK Claude did that" is a valid excuse. Immediate rejection. That will work, but only until the people filing these PRs go crying to their managers that you refuse to merge any of their code, at which point you'll be given a stern reprimand from your betters to stop being so picky. Have fun vibe-reviewing.

pkdpic 1y ago

Yeah in a perfect world I absolutely agree. But the reality I'm observing is that everything continues to always be behind schedule (not a new phenomena) and if anything expectations from project leads / management are just getting less realistic leaving Jr devs in even more of a position of "no time to be curious or learn deeply just get it done" and teamleads reviewing PRs in a maybe even worse position of "no time to get into a deep review / mentorship session just figure out if it breaks anything or not". And ultimately clients are still just living in fantasy land in terms of expectations both in terms of build out time for basic features / patches and also how much AI razzle-dazzle they expect for new project proposals. Nothing can move fast enough to keep up with these hype-fueled TED talk expectations all the way up the chain. I don't know if there's any solution and I'm sure it's not like this everywhere but I'm also sure I'm not alone. At this point I'm just trying to keep my feet wet on "AI" related projects until the hype dust settles so I can reassess what this industry even is anymore. Maybe it's not too late to get a single subject credential and go teach math or finger painting or something.