13 ms·

When everyone has AI and the company still learns nothing

390 points by youngbrioche 6mo ago

blitzar 6mo ago
> Where is the ROI for the 2 mio € we paid Anthropic last year? The CEO has a youtube style platinum token plaque for their office.
nati0n 6mo ago
[dead]
rob74 6mo ago
One more point I noticed: since AI adoption is being promoted by companies, collaboration between developers could suffer. Why wait for a more experienced developer to have the time to explain some aspect of the codebase to you (and at the same time confess your ignorance), when AI can do it right away in a competent-sounding way (and most of the time it will probably be right, too)?
- homeonthemtn 6mo ago
  That's a valid point. Dev/team member isolation, not a great environment to build
  - reaperducer 6mo ago
    Dev/team member isolation, not a great environment to build Gone are the days of mandatory corporate "synergy" and after-work bar gatherings to promote "team building." AI is showing people in the tech industry that they're just interchangeable cogs. AI is bringing the offshored Indian work environment to Silicon Valley.
- b112 6mo ago
  I think you hit the nail on the head, it's probably right, most of the time. Or, maybe 89% right, 91% of the time. The more I use AI, the more I see mistakes. I've noticed others see these same mistakes, correct them, then when queried say "Oh, it gets it right all of the time!". No, having to point out "you got this wrong, re-write that last bit" isn't "getting it right". And it's not that the code is wrong overtly, it's subtle. Not using a function correctly, not passing something through it should (and the default happens to just work -- during testing), and more. LLMs are great at subtle bugs. So moving forward with this isolation you mention, ensures that maybe the guy in the company, the 'answer guy' about a thing, never actually appears. Maybe, he doesn't even get to know his own code well enough to be the answer guy. And so when an LLM writes a weird routine, instead of being able to say "No, re-write that last bit", you'll have to shrug and say "the code looks fine, right?", because you, and the answer guy, if he exists, don't know the code well enough to see the subtle mistakes.
  - skydhash 6mo ago
    I’ve noticed that when I was implementing a build pipeline for a project. My changes introduced a runtime bug (I only tested that the thing was building), but then another developer broke the pipeline while fixing the runtime bug. While it was a failure of mine to introduce the runtime bug, I don’t think I can publish a fix for a bug without investigating why a bug appeared in the first place. Because code is all about assumptions and contracts, and if something that was working break, that means something else has changed and you need to be aware of it.
- rogerthis 6mo ago
  That already happens here. I am old dev who was the goto guy for people with certain business and technical questions. Not anymore (which is part good, as I'm interrupted much less, and part bad, as sometimes they regard the wrong answer as truth).
  - cadamsdotcom 6mo ago
    You could vibe yourself up an AMA tool where people can submit questions, an agent goes to work on them, then the question and agent answer sit in a queue waiting for you to provide a review and give your weigh-in.
    - i_think_so 6mo ago
      I'm pretty sure this is the best idea I've ever heard of for this technology. You should build that tool and it should become mandatory throughout the tech world. Can we get some enabling legislation? A UN resolution perhaps?
      - cadamsdotcom 6mo ago
        Despite the snark I’ll engage. The “get an immediate agent answer then a human expert’s fast-follow” is I think a great idea for many domains - imagine if you could get legal advice this way; the agent will have already explained the basics and the human expert just has to provide corrections - way less typing by humans. Also, the corrections are now documented and could become future grounding for the agent.
        i_think_so 6mo ago
        Absolutely zero snark. I'm serious. (About the serious part; obviously not the joke part.) > a great idea for many domains I completely agree. This is a great idea. If you don't do something with it I'm stealing it. ;-)
        Terr_ 6mo ago
        I expect the time-limited expert will actually end up being tasked with more pain per request. They won't just need to understand what problem the requestor has (or thinks they have) but also validate that the "immediate" feedback wasn't subtly horribly wrong.
- user34283 6mo ago
  In a large codebase it‘s probably next to impossible to get people who fully understand the code to explain it to you with unerring accuracy. AI can get a pretty good picture, near instantly, whenever you need it. It’s not just competent-sounding, it is reasonably competent, and certainly very useful for tasks like that.
cyanydeez 6mo ago
I think if these companies first adopted local models with fewer token outs and the learners got to watch the tokens get made, there'd be a lot more understanding.
olsondv 6mo ago
The post hits the nail on the head with the messy middle. There is simply no motivation to develop this sort of intelligence loop as a dev who has their own responsibilities which their job depend on. Management can ask as nicely as they want, but I’m not going to selflessly share my productivity gains with the broader company for free. I might share a tool if it’s useful. All the learning of how to wrangle AI or set up agents is better kept to myself if there is no recognition for sharing. My company set up a “prompt of the week” award and brown-bag sessions to help spread adoption. We also have teams meant to develop these workflows. Clearly, they set these events up to play it off as their own productivity. Without a real (read “monetary”) incentive or job security, the risk and cost of spreading the knowledge falls squarely on the developer.
- ravenstine 6mo ago
  It kinda racks my brain how a lot of people don't think this way. For example, way before the current state of AI, I wrote my own CLI to make aspects of my job easier and easier to write scripts to automate; some colleagues have noticed my tool and said I should share it, and my diplomatically worded answer is no. I don't share it with anyone because of the negative return in both supporting it and everyone else being able to be as productive as I am. Moreover, leadership will not recognize my ingenuity as an asset, hence no added job security. No way am I going to help my company out of the goodness of my heart to be potentially let go anyway in the near future. If developers are worried about their jobs with the way the market currently is, they should treat their personal workflows as trade secrets. My example was not specific to AI, but it applies just as much to AI workflows. In a worker's market, it was sometimes fun to share that kind of knowledge with an organization. In an employer's market, they can pay me if they want access to my personal choices.
  - anonymars 6mo ago
    What are your thoughts on open source? Seems like the same problem writ large
    - ravenstine 6mo ago
pards 6mo ago
In my large enterprise world, AI adoption hasn't made it outside of the development teams - only developers have access to Github Copilot. Code takes 6-12 months to make it from commit to production. Development speed was never the bottleneck; it's all the other processes that take time: infra provisioning, testing, sign-offs, change management, deployment scheduling etc. AI makes these post-development bottlenecks worse. Changes are now piling up at the door waiting to get on a release train. Large enterprises need to learn how to ship software faster if they want to lock in ROI on their token spend. Unshipped code is a liability, not an asset.
- embedding-shape 6mo ago
  > Large enterprises need to learn how to ship software faster They haven't even learned that "less code is better" yet, I wouldn't hold my breathe waiting for them to suddenly learn "more advanced" things like that before they learn the basics.
  - forinti 6mo ago
    More code means more support and more maintenance. If your team is already overloaded or if it's going to be reduced because of AI, things are going to get tough.
    - lxgr 6mo ago
      My bigger concern is actually that, if a company isn't careful, the bloat (complexity, amount of code, other artifacts etc.) will just balloon and largely cancel out any gains. Feedback is often only considered once something is already on fire (financially, functionally, or literally).
      - nkrisc 6mo ago
        That’s the game plan for the AI companies: once companies have massive codebases of critical AI generated code and a skeleton crew of prompt engineers they’re going to be locked in to the AI product to develop anything new. They’re not even selling shovels, they’re selling subscriptions for shovels.
deleted 6mo ago
[deleted]
simoncion 6mo ago
> There is another pressure building underneath all this. AI usage will become more visibly metered. The current enterprise feeling of “everyone has access, don’t worry too much about the bill” will not hold forever, at least not in the form people are getting used to. ... > I do not want to make this a cost panic story, that would be the least interesting way to think about “rented intelligence”. The question is not how to minimize token spend in the abstract, any more than the question of software delivery was ever how to minimize keystrokes. If tokens were as cheap as keystrokes -that is, effectively free- then "How do we minimize token spend?" wouldn't be a question that anyone asks. It's because keystrokes are effectively free that you only ask "How do we minimize the number of keys pressed during the software development process?" if you're looking for an entertaining weekend project. If keystrokes cost as much per unit of work done as the -currently heavily subsidized- cost of tokens from OpenAI and Anthropic, you'd see a lot of focus on golfing everything under the sun all the damn time.
- fallpeak 6mo ago
  Tokens _are_ as cheap as keystrokes. A single keypress by a full-time SWE averages out to $0.005-$0.02 (depending on typing speed and TC). The relationship is obscured because the keystrokes are usually part of a fixed-price subscription plan but they absolutely have a cost. Prior to AI this was in fact a large reason everyone pontificated about concise programming languages and elegantly factoring problems and DRY and...
  - simoncion 6mo ago
    > Tokens _are_ as cheap as keystrokes. extremely loud incorrect buzzer I can set up a drinking-bird-style device to press keys on a keyboard. My per-year operational cost is the same whether or not the bird presses keys. Programmer time costs money. Nearly all of a programmer's time is spent understanding the system being maintained, the reasons for its existence, and the reasons for the proposed changes to it. If you're honestly such a fool as to think that DRY, "elegant" [0] design, and related things are about reducing the number of keys pressed rather than the time spent understanding the system and how to change it, then... well, I think you make an awfully good "AI Booster". [0] Read as "easy to reason about, once you understand the core concepts"
Schlagbohrer 6mo ago
[dead]
Cthulhu_ 6mo ago
On the first part of the article, I believe it describes how individual productivity gains do not seem to translate to business / larger scale productivity. I think this is expected; individual developer productivity, code volume, LOC/day never was a valuable metric on a company scale. Number of delivered features might be one, but ultimately, revenue and customer growth etc are. While I do believe higher developer productivity can lead to faster reacting to market forces or more A/B testing, that won't necessarily lead to a successful business. Because ultimately it rarely is the software that's the issue there.
jt654 6mo ago
This is a great article. It helps you realize that the feedback loop is the goal but it won't just happen and traditional methodologies don't really support it. Has anyone here found a good way that promotes teams in a company to focus on the loop instead of productivity hack?
woodydesign 6mo ago
Great article. The part that stood out to me is the shift in how organizations define work. In the old model, performance and OKRs were anchored in disciplines, job titles, and role-specific expectations. In the AI era, those boundaries are starting to collapse. The deeper issue is psychological and organizational: people are constantly negotiating the line between “this is my job” and “this is not my responsibility.” That creates a key adoption problem: what is the upside of being visibly recognized as an expert AI user? If people learn that I can do faster, better, and more cross-functional work, why would I reveal that unless the company also creates a clear system for recognition, compensation, or career growth?
- ohnei 6mo ago
  If they create a system to compensate expert AI users wouldn't that career have a problem in that anyone (enticed by the new careers existence and) integrating their advice on any company particulars with a (weeks) more modern approach is basically putting them in the role of domain expert being eliminated.
  - woodydesign 6mo ago
    The part I push back on is the idea that expertise is easy to learn in just a few weeks. Take Andrej Karpathy as an example. Even if I knew exactly what tools he uses and what his workflow looks like, I still would not be able to produce anything close to what he can produce in a few weeks. And he is not standing still either—he is evolving at the same time. A lot of real expertise is not in the visible/system-able workflow. It is in someone’s experience, taste, judgment, and wisdom. You can copy the artifact, but you cannot easily copy the thinking behind it: the principles, the decision-making, and the ability to apply those principles across many different/subtle situations. But I do agree with the concern behind the argument. People may worry that sharing what they know could weaken their own position. And the more uncomfortable question is about peers: if someone’s role can be “retired” because others absorbed their knowledge and skills, then it is hard not to ask, “Am I next?”
    - ohnei 6mo ago
zidoo 6mo ago
Once people try to increase quality instead of speed they will see how LLMs are powerful. Everything else is just sales pitch by Nvidia and friends.
- wongarsu 6mo ago
  Even if LLMs write more buggy code they can still bring up software quality in the short to medium term by allowing you to clear out a lot of the backlog of bugs and UI issues that are known but never had enough priority to be fixed Debugging and developing first fixes is also one of the spaces where current LLMs are the biggest force multipliers. Especially if you have reproduction cases the LLM can test on its own But long-term it might look very different as more and more of the code becomes LLM written
- MyHonestOpinon 6mo ago
  Make sense to me. I can see how LLMs can help you make better systems. I don't have a christal ball but I can see how focusing on speed (or more precisely volume) can have a lot of unintended consequences.
cadamsdotcom 6mo ago
AI by itself isn’t that useful. An agent forgets and makes enough mistakes that you have to check all its work, which can be net productivity negative. It really comes into its own when you treat it as a tool that can build other tools. For example, having it build tools that force it to keep going until its work reaches a certain quality, or runs compliance checks on its outputs and tells it where it needs to fix things. Then and only then, can you trust its work. Right now most current roles & workflows are designed around wrangling the tools you’re given to do a certain job. In that regime AI can only slide in at the edges.
- SadErn 6mo ago
  [dead]
i_think_so 6mo ago
> one team uses Copilot as autocomplete and calls it a day. Another team runs Claude Code in tight loops, with tests, reviews, and constant steering. A product owner suddenly prototypes real software instead of mocking screens in Figma. A senior engineer delegates a root-cause analysis to an agent and comes back to the valid solution in under an hour; this would’ve taken him two weeks without AI. A junior person produces polished code but has no idea which architectural assumptions got smuggled into the system. A support team quietly turns recurring tickets into workflow automation, because they know exactly where the work hurts and nobody in the Center of Excellence ever asked the right question. This is just sales copy for various AI companies, laundered through an "influencer". It might as well be the CIA sending their article to be published in Daily Post Nigeria, so that the NYT can quote it as "sources". The title is just clickbait. The rest of the content are fluffy bunnies and rainbows. It's all summed up as "continue to consume product, but remember to also do X". Sales copy + HBR MBA bait. The closest thing to an honest, less-than-rosy example is the "junior person" who has no idea about the code they committed. What about the "senior person" who has no idea about the code they committed? What about the CISO who doesn't understand that pasting proprietary documents willy nilly into the LLM's gaping maw might have legal/security/common sense implications, and that it is his job to set policy on such behavior? What about the middle manager who doesn't even try to retain the most experienced dev in the company because "we don't need the headcount anymore, now that Claude is so fast"? What about the company eating its own seed corn because every single junior position has been eliminated and there are no plans for the future anymore? What about the filesystem developer who fell in love with his chatbot girlfriend and is crashing out on Discord? Oh wait, scratch that last one. He left the company and is crashing out on his own. Carry on, then.
- Supermancho 6mo ago
  Indeed. Any developer who has used copilot knows you can't rely on it 100% The post's head image immediately bothered me. Copilot's strength is not on patching SDLC but to speed up the catching of typos and minor oversights. If you're using it as an integral part of SDLC, it causes problems immediately. So why posit the strawman? Marketing.
- lbrito 6mo ago
alaudet 6mo ago
As a 3 year retired Systems Analyst I feel bad for my younger colleagues. In 2023 I was one of the first in my team to use AI to untangle some legacy code that did something mission critical with Perl and whose original author had long ago left and apparently didn't understand anything about actually commenting code or documentation. We were all in awe of this new technology that got us out of a bind. But more and more it looks less like a tool that is available to you instead of something that is being _done_ to you. Nobody asked for this. At what point is inspiration and thought just devalued and worthless in the name of doing things instantly. The work has no soul.
- deleted 6mo ago
  [deleted]
ch_ase 6mo ago
It’s been helpful for me to look at the promise of AI by comparing with the dotcom boom. Lots of similarities. But the internet was a simpler concept for businesses. Basically it was you can now sell to people from their computers. AI’s promise is what? It can approximate reasoning about things? This is much more challenging implementation puzzle to truly solve. I don’t know that I’ve seen anything of real substance outside coding tasks yet.
cmiles8 6mo ago
There are some improvements on coding and speed of developers, but more broadly in the enterprise AI is just producing a lot of slop that folks are getting fed up with. AI content has a look and feel people sense immediately. It’s amazing to see how quickly things shifted from “wow this is so cool, AI is going to change everything” to folks calling out “you lazy bum, this just looks like some slop you threw together with AI… let’s get some real thinking please.” We are firmly heading into “trough of disillusionment” territory on the hype cycle.
netcan 6mo ago
Path dependencies between invention and utilization... are complicated and hard to fathom. Our mental models of developments like the industrial revolution, literacy, printing or suchlike tend to be a lot more straightforward than how things play out in practice. When a bottleneck is eliminated... you tend to shortly find the next bottleneck. Meanwhile, there is an underlying assumption everyone seems to make that "more software, more value" is the basic reality. But... I'm skeptical. To do lists, wishlists, buglists and road maps may be full of stuff but... Visa or Salesforce have already exploited all their immediate "more software, more money" opportunities. The ones in a position to easily leverage AI are upstarts. They're starting with nothing. No code. No features. No software. With Ai, presumably, they can produce more software and make value. Also... I think overextended market rationalism leads people to see everything as an industrial revolution...which irl is much more of an exception. The networked personal computing revokution put a pc one every desk. It digitized everything. Do we have way better administration for less cost? Not really. Most administrations have grown. Did law fundamentally change dues to dugital efficiency? No. Not really. If you work on a terrible enterprise codebase... it's very possible that software quality/quantity isn't actually that important to your organization.
- cyanydeez 6mo ago
  >If you work on a terrible enterprise codebase... it's very possible that software quality/quantity isn't actually that important to your organization. It's possible capitalism will drive all enterprise to terrible codebases.
dominictorresmo 6mo ago
It's just ass to work in this area now. In the company I work, the bosses let everyone use it, even non-developers. I really want to quit and work in another area but unfortunately where I live a beginning salary can't pay a rent and I'm getting old
luodaint 6mo ago
[flagged]
threecheese 6mo ago
TIL my $company has used the same consultants as this guy. We started with Training and Champions, to Leadership/Lab/Crowd with a CoE/brown bags. We are definitely struggling with the same issues author describes, but even worse the leaders down at the Crowd level have some perverse need to achieve reuse across their teams, rather than letting their Crowd experiment. One team does something interesting, we must stop and get that thing out to all teams in that group, so everyone “benefits”. This is a scarcity mindset, which made sense pre-AI where code was costly and ideas were more valuable. At the same time, everyone not only has to do their work, they need to be 25% more efficient from AI (new KPIs), and so their own learnings slow to a halt, and the team with the cool idea has to give presentations instead of hacking.
romaniv 6mo ago
> "Where is the ROI for the 2 mio € we paid Anthropic last year?" The bias in the assumptions here is absolutely bonkers. Problem: GenAI is not generating any visible return on investment. "Solution": rearrange your entire development organization around the technology and start inventing new tooling. What's entirely obvious is that the point of such articles is not the stuff they purportedly discuss, but the normalization of assumptions those discussions are based on.
- paodealho 6mo ago
  LLMs can't fail, they can only be failed ... by you!
  - Terr_ 6mo ago
    Hence OpenAI sinking $4b into "The Deployment Company", to pump up the number of Very Helpful Consultants which can offer to help your company overcome is tragic failure to adopt and buy, for a small fee...
aykutseker 6mo ago
The Hub captures decisions humans made. It can't see the ones they didn't, which is most of what AI ships. You instrument the deliberate half and inherit the undeliberate one.
raffael_de 6mo ago
My biggest gripe with language models is that technical and conceptual discussions which used to be led organically (with people having to think about what others wrote and decide what to reply themselves) now turned into AI slop avalanches with participants just copy pasting obviously generated text into the discussion. And those texts are always very long and super weird to respond to because they usually are overall correct enough so you can't just reject them but are flawed all over the place, missing the point, lacking depth where it would be important, skipping over important steps. This is a huge time waste. Funnily, many people have no idea how obvious it is that their texts are generated and rubbish at that.
ncr100 6mo ago
In many midsize companies I've worked as an engineer, developing a growth plan is something I have been asked to do, and to share it with my manager. Then at the end of the quarter I go over the growth plan and the manager agrees or disagrees with my actual growth, and gives me a performance bonus befitting my success.. I propose employees create self-training byproducts as a result of any AI interaction. And then they also work with their Cuban manager to make sure that these self-training byproducts are a part of their growth plan. This can guarantee growth without losing that opportunity To interact with the intelligent AI system (on topics that are relevant to the company's short, mid, and long-term strategic advantage,).
nelfrancis 6mo ago
[flagged]
conorliu 6mo ago
[flagged]
Knowledgee_KZA 6mo ago
[flagged]
chrisjj 6mo ago
> The company may still learn almost nothing. Not a problem if the hired "AI" now does that job. /i
dakiol 6mo ago
If you're a regular engineer like me, there's no real upside to using AI in a company setting. They're boiling us. Of course, the HN elite (investors, execs, celebrities, and top-tier engineers) will say otherwise because "how can you be against innovation man?" AI/LLMs aren't innovation the way TCP/IP, linux, or postgres were. To be clear: claude/codex/gemini/grok/whatever exist for profit, to squeeze the last drop of productivity out of you until there's nothing left, and then you're disposable (laid off). If you like AI, use open source models, use them in your side projects.
- asdfman123 6mo ago
  1) The game is not ending, it's changing. AI can sling a lot of code but we still need engineers that actually understand what the hell is going on. That's always been the bottleneck. It could eliminate junior positions, but seniors are fine for now. 2) It's been a hard lesson for me to learn because I'm naturally a contrarian, but you are hired to do what management wants you to do. If you resist, your best bet is to hope they don't notice or care, but it's not going to change much.
  - herpderperator 6mo ago
    If you eliminate juniors over the next few years, there will be no seniors for the future when the current seniors retire.
    - sanderjd 6mo ago
      Yep, I'm a fan of the current crop of AI tools because they are incredibly useful to me, but I have deep concerns about the pipeline problem.
    - Our_Benefactors 6mo ago
      Then pay will go up again for those mid-level developers who still remain, and companies will again overhire and overtrain like we saw during COVID years. “We won’t have any seniors in ten years!!1!” is a handwringy problem that self solves by the free market.
woeirua 6mo ago
The hype is extreme right now and everyone is still trying to figure out how to use the tools. People who are further along the bleeding edge are trying to tear down all the process that we used to have to further improve velocity. After people go all the way to "dark factories" most companies will realize that they don't actually have any good ideas for what to build, and honestly never did. They've been coasting for years, someone else can replicate their product now and its just a race to the bottom. At that point, token budgets are going to collapse. I'm staunchly pro-AI as a technology, but I do think the bubble is going to pop in the next year or two just because the business value won't materialize for most companies fast enough.
- a96 6mo ago
  Enshittification of the AI clouds will also pick up speed. Tokens will get more costly, terms and tiers will turn worse etc. And there will be more security incidents on all fronts.
trenchgun 6mo ago
Article is mostly LLM written slop.
scotttaylor 6mo ago
[flagged]
theuniverseson 6mo ago
[flagged]
nickchen778 6mo ago
[flagged]
OutOfHere 6mo ago
Link is dead.
lbrito 6mo ago
>This cannot become employee surveillance...If people believe the organization is measuring whether they used enough AI, they will game the signals. It already has; ship has sailed. https://blog.pragmaticengineer.com/the-pulse-tokenmaxxing-as-a-weird-new-trend/ https://blog.pragmaticengineer.com/the-pulse-tokenmaxxing-as...
throw_m239339 6mo ago
I love these posts because they are missing the elephant in the room. The company learns nothing, it develops no knowledge, however, OpenAI, Anthropic and co services are absolutely learning from the people that use these tools. You are training your own replacement.
Amber-chen 6mo ago
[flagged]
hnisforrtards 6mo ago
[dead]
Roark66 6mo ago
In the current "big co" I'm working for it's a bit ridiculous. While AI tools have been provided pretty quickly (over a year ago, I initially used gemini cli, then copilot once it added anthropic models) the management is absolutely clueless about it. The top wants agents. Every team is asked few times a week "what autonomous agents will you build next" and answers the current AI lacks agency required not to mess up critical long running tasks and generate even more work are falling on deaf ears. (also ideas such as, why don't we setup a wiki page were teams can post their repetitive tasks and we can use AI to script them, are considered "not fast enough" - just build it... but we are the automation team, we automated everything we do years ago :-) Middle managers on the other hand suddenly started giving juniors senior's work and asking seniors "tell them (juniors) how to prompt it". Seriously? How about I prompt it myself instead? Oh, but it makes a shit load of architectural errors and boobie traps the junior will fail to find... So now instead of a cursory glance I have to spend an hour reviewing a small PR from them. And any questions about "why are you creating a new X for this instead of extending the existing one?" are met with blank panicked stares... The essence of this BS is contained in my description of the recent "Copilot Review" incident. We sometimes merge the same Github workflow files (10 line files) to dozens of repos, we have to obtain approvals for the PRs from a bunch of teams working in different timezones, but the merge has to be done everywhere at once and it has to be coordinated with other work. As we were on a day of such task some "helpful hand" enabled Copilot PR reviews for the whole org. Copilot helpfully opened 7 or 8 discussions on each PR giving us such precious advice as "your concurrency group uses the commit sha as a differentiating factor, this will allow multiple runs to proceed concurrently" to which one is tempted to answer "no shit sherlock". We suddenly had almost 200 conversations to "resolve" an hour before the merge and a bunch of approvers didn't give their approvals because "there is a discussion". Thankfully we had copilot that wrote us a script in 5 minutes to resolve the problem caused by itself... Maybe our next overnight agent can go over all our open PRs and close Copilot Review conversations with appropriate messages?

390 points by youngbrioche 6mo ago

> Where is the ROI for the 2 mio € we paid Anthropic last year? The CEO has a youtube style platinum token plaque for their office.

nati0n 6mo ago

[dead]

rob74 6mo ago

One more point I noticed: since AI adoption is being promoted by companies, collaboration between developers could suffer. Why wait for a more experienced developer to have the time to explain some aspect of the codebase to you (and at the same time confess your ignorance), when AI can do it right away in a competent-sounding way (and most of the time it will probably be right, too)?

homeonthemtn 6mo ago
That's a valid point. Dev/team member isolation, not a great environment to build
- reaperducer 6mo ago
  Dev/team member isolation, not a great environment to build Gone are the days of mandatory corporate "synergy" and after-work bar gatherings to promote "team building." AI is showing people in the tech industry that they're just interchangeable cogs. AI is bringing the offshored Indian work environment to Silicon Valley.
b112 6mo ago
I think you hit the nail on the head, it's probably right, most of the time. Or, maybe 89% right, 91% of the time. The more I use AI, the more I see mistakes. I've noticed others see these same mistakes, correct them, then when queried say "Oh, it gets it right all of the time!". No, having to point out "you got this wrong, re-write that last bit" isn't "getting it right". And it's not that the code is wrong overtly, it's subtle. Not using a function correctly, not passing something through it should (and the default happens to just work -- during testing), and more. LLMs are great at subtle bugs. So moving forward with this isolation you mention, ensures that maybe the guy in the company, the 'answer guy' about a thing, never actually appears. Maybe, he doesn't even get to know his own code well enough to be the answer guy. And so when an LLM writes a weird routine, instead of being able to say "No, re-write that last bit", you'll have to shrug and say "the code looks fine, right?", because you, and the answer guy, if he exists, don't know the code well enough to see the subtle mistakes.
- skydhash 6mo ago
  I’ve noticed that when I was implementing a build pipeline for a project. My changes introduced a runtime bug (I only tested that the thing was building), but then another developer broke the pipeline while fixing the runtime bug. While it was a failure of mine to introduce the runtime bug, I don’t think I can publish a fix for a bug without investigating why a bug appeared in the first place. Because code is all about assumptions and contracts, and if something that was working break, that means something else has changed and you need to be aware of it.
rogerthis 6mo ago
That already happens here. I am old dev who was the goto guy for people with certain business and technical questions. Not anymore (which is part good, as I'm interrupted much less, and part bad, as sometimes they regard the wrong answer as truth).
- cadamsdotcom 6mo ago
  You could vibe yourself up an AMA tool where people can submit questions, an agent goes to work on them, then the question and agent answer sit in a queue waiting for you to provide a review and give your weigh-in.
  - i_think_so 6mo ago
    I'm pretty sure this is the best idea I've ever heard of for this technology. You should build that tool and it should become mandatory throughout the tech world. Can we get some enabling legislation? A UN resolution perhaps?
    - cadamsdotcom 6mo ago
      Despite the snark I’ll engage. The “get an immediate agent answer then a human expert’s fast-follow” is I think a great idea for many domains - imagine if you could get legal advice this way; the agent will have already explained the basics and the human expert just has to provide corrections - way less typing by humans. Also, the corrections are now documented and could become future grounding for the agent.
      - i_think_so 6mo ago
        Absolutely zero snark. I'm serious. (About the serious part; obviously not the joke part.) > a great idea for many domains I completely agree. This is a great idea. If you don't do something with it I'm stealing it. ;-)
      - Terr_ 6mo ago
        I expect the time-limited expert will actually end up being tasked with more pain per request. They won't just need to understand what problem the requestor has (or thinks they have) but also validate that the "immediate" feedback wasn't subtly horribly wrong.
user34283 6mo ago
In a large codebase it‘s probably next to impossible to get people who fully understand the code to explain it to you with unerring accuracy. AI can get a pretty good picture, near instantly, whenever you need it. It’s not just competent-sounding, it is reasonably competent, and certainly very useful for tasks like that.

cyanydeez 6mo ago

I think if these companies first adopted local models with fewer token outs and the learners got to watch the tokens get made, there'd be a lot more understanding.

olsondv 6mo ago

The post hits the nail on the head with the messy middle. There is simply no motivation to develop this sort of intelligence loop as a dev who has their own responsibilities which their job depend on. Management can ask as nicely as they want, but I’m not going to selflessly share my productivity gains with the broader company for free. I might share a tool if it’s useful. All the learning of how to wrangle AI or set up agents is better kept to myself if there is no recognition for sharing. My company set up a “prompt of the week” award and brown-bag sessions to help spread adoption. We also have teams meant to develop these workflows. Clearly, they set these events up to play it off as their own productivity. Without a real (read “monetary”) incentive or job security, the risk and cost of spreading the knowledge falls squarely on the developer.

ravenstine 6mo ago
It kinda racks my brain how a lot of people don't think this way. For example, way before the current state of AI, I wrote my own CLI to make aspects of my job easier and easier to write scripts to automate; some colleagues have noticed my tool and said I should share it, and my diplomatically worded answer is no. I don't share it with anyone because of the negative return in both supporting it and everyone else being able to be as productive as I am. Moreover, leadership will not recognize my ingenuity as an asset, hence no added job security. No way am I going to help my company out of the goodness of my heart to be potentially let go anyway in the near future. If developers are worried about their jobs with the way the market currently is, they should treat their personal workflows as trade secrets. My example was not specific to AI, but it applies just as much to AI workflows. In a worker's market, it was sometimes fun to share that kind of knowledge with an organization. In an employer's market, they can pay me if they want access to my personal choices.
- anonymars 6mo ago
  What are your thoughts on open source? Seems like the same problem writ large
  - ravenstine 6mo ago

pards 6mo ago

In my large enterprise world, AI adoption hasn't made it outside of the development teams - only developers have access to Github Copilot. Code takes 6-12 months to make it from commit to production. Development speed was never the bottleneck; it's all the other processes that take time: infra provisioning, testing, sign-offs, change management, deployment scheduling etc. AI makes these post-development bottlenecks worse. Changes are now piling up at the door waiting to get on a release train. Large enterprises need to learn how to ship software faster if they want to lock in ROI on their token spend. Unshipped code is a liability, not an asset.

embedding-shape 6mo ago
> Large enterprises need to learn how to ship software faster They haven't even learned that "less code is better" yet, I wouldn't hold my breathe waiting for them to suddenly learn "more advanced" things like that before they learn the basics.
- forinti 6mo ago
  More code means more support and more maintenance. If your team is already overloaded or if it's going to be reduced because of AI, things are going to get tough.
  - lxgr 6mo ago
    My bigger concern is actually that, if a company isn't careful, the bloat (complexity, amount of code, other artifacts etc.) will just balloon and largely cancel out any gains. Feedback is often only considered once something is already on fire (financially, functionally, or literally).
    - nkrisc 6mo ago
      That’s the game plan for the AI companies: once companies have massive codebases of critical AI generated code and a skeleton crew of prompt engineers they’re going to be locked in to the AI product to develop anything new. They’re not even selling shovels, they’re selling subscriptions for shovels.

deleted 6mo ago

[deleted]

simoncion 6mo ago

> There is another pressure building underneath all this. AI usage will become more visibly metered. The current enterprise feeling of “everyone has access, don’t worry too much about the bill” will not hold forever, at least not in the form people are getting used to. ... > I do not want to make this a cost panic story, that would be the least interesting way to think about “rented intelligence”. The question is not how to minimize token spend in the abstract, any more than the question of software delivery was ever how to minimize keystrokes. If tokens were as cheap as keystrokes -that is, effectively free- then "How do we minimize token spend?" wouldn't be a question that anyone asks. It's because keystrokes are effectively free that you only ask "How do we minimize the number of keys pressed during the software development process?" if you're looking for an entertaining weekend project. If keystrokes cost as much per unit of work done as the -currently heavily subsidized- cost of tokens from OpenAI and Anthropic, you'd see a lot of focus on golfing everything under the sun all the damn time.

fallpeak 6mo ago
Tokens _are_ as cheap as keystrokes. A single keypress by a full-time SWE averages out to $0.005-$0.02 (depending on typing speed and TC). The relationship is obscured because the keystrokes are usually part of a fixed-price subscription plan but they absolutely have a cost. Prior to AI this was in fact a large reason everyone pontificated about concise programming languages and elegantly factoring problems and DRY and...
- simoncion 6mo ago
  > Tokens _are_ as cheap as keystrokes. extremely loud incorrect buzzer I can set up a drinking-bird-style device to press keys on a keyboard. My per-year operational cost is the same whether or not the bird presses keys. Programmer time costs money. Nearly all of a programmer's time is spent understanding the system being maintained, the reasons for its existence, and the reasons for the proposed changes to it. If you're honestly such a fool as to think that DRY, "elegant" [0] design, and related things are about reducing the number of keys pressed rather than the time spent understanding the system and how to change it, then... well, I think you make an awfully good "AI Booster". [0] Read as "easy to reason about, once you understand the core concepts"

Schlagbohrer 6mo ago

[dead]

Cthulhu_ 6mo ago

On the first part of the article, I believe it describes how individual productivity gains do not seem to translate to business / larger scale productivity. I think this is expected; individual developer productivity, code volume, LOC/day never was a valuable metric on a company scale. Number of delivered features might be one, but ultimately, revenue and customer growth etc are. While I do believe higher developer productivity can lead to faster reacting to market forces or more A/B testing, that won't necessarily lead to a successful business. Because ultimately it rarely is the software that's the issue there.

jt654 6mo ago

This is a great article. It helps you realize that the feedback loop is the goal but it won't just happen and traditional methodologies don't really support it. Has anyone here found a good way that promotes teams in a company to focus on the loop instead of productivity hack?

woodydesign 6mo ago

Great article. The part that stood out to me is the shift in how organizations define work. In the old model, performance and OKRs were anchored in disciplines, job titles, and role-specific expectations. In the AI era, those boundaries are starting to collapse. The deeper issue is psychological and organizational: people are constantly negotiating the line between “this is my job” and “this is not my responsibility.” That creates a key adoption problem: what is the upside of being visibly recognized as an expert AI user? If people learn that I can do faster, better, and more cross-functional work, why would I reveal that unless the company also creates a clear system for recognition, compensation, or career growth?

ohnei 6mo ago
If they create a system to compensate expert AI users wouldn't that career have a problem in that anyone (enticed by the new careers existence and) integrating their advice on any company particulars with a (weeks) more modern approach is basically putting them in the role of domain expert being eliminated.
- woodydesign 6mo ago
  The part I push back on is the idea that expertise is easy to learn in just a few weeks. Take Andrej Karpathy as an example. Even if I knew exactly what tools he uses and what his workflow looks like, I still would not be able to produce anything close to what he can produce in a few weeks. And he is not standing still either—he is evolving at the same time. A lot of real expertise is not in the visible/system-able workflow. It is in someone’s experience, taste, judgment, and wisdom. You can copy the artifact, but you cannot easily copy the thinking behind it: the principles, the decision-making, and the ability to apply those principles across many different/subtle situations. But I do agree with the concern behind the argument. People may worry that sharing what they know could weaken their own position. And the more uncomfortable question is about peers: if someone’s role can be “retired” because others absorbed their knowledge and skills, then it is hard not to ask, “Am I next?”
  - ohnei 6mo ago

zidoo 6mo ago

Once people try to increase quality instead of speed they will see how LLMs are powerful. Everything else is just sales pitch by Nvidia and friends.

wongarsu 6mo ago
Even if LLMs write more buggy code they can still bring up software quality in the short to medium term by allowing you to clear out a lot of the backlog of bugs and UI issues that are known but never had enough priority to be fixed Debugging and developing first fixes is also one of the spaces where current LLMs are the biggest force multipliers. Especially if you have reproduction cases the LLM can test on its own But long-term it might look very different as more and more of the code becomes LLM written
MyHonestOpinon 6mo ago
Make sense to me. I can see how LLMs can help you make better systems. I don't have a christal ball but I can see how focusing on speed (or more precisely volume) can have a lot of unintended consequences.

cadamsdotcom 6mo ago

AI by itself isn’t that useful. An agent forgets and makes enough mistakes that you have to check all its work, which can be net productivity negative. It really comes into its own when you treat it as a tool that can build other tools. For example, having it build tools that force it to keep going until its work reaches a certain quality, or runs compliance checks on its outputs and tells it where it needs to fix things. Then and only then, can you trust its work. Right now most current roles & workflows are designed around wrangling the tools you’re given to do a certain job. In that regime AI can only slide in at the edges.

SadErn 6mo ago
[dead]

i_think_so 6mo ago

> one team uses Copilot as autocomplete and calls it a day. Another team runs Claude Code in tight loops, with tests, reviews, and constant steering. A product owner suddenly prototypes real software instead of mocking screens in Figma. A senior engineer delegates a root-cause analysis to an agent and comes back to the valid solution in under an hour; this would’ve taken him two weeks without AI. A junior person produces polished code but has no idea which architectural assumptions got smuggled into the system. A support team quietly turns recurring tickets into workflow automation, because they know exactly where the work hurts and nobody in the Center of Excellence ever asked the right question. This is just sales copy for various AI companies, laundered through an "influencer". It might as well be the CIA sending their article to be published in Daily Post Nigeria, so that the NYT can quote it as "sources". The title is just clickbait. The rest of the content are fluffy bunnies and rainbows. It's all summed up as "continue to consume product, but remember to also do X". Sales copy + HBR MBA bait. The closest thing to an honest, less-than-rosy example is the "junior person" who has no idea about the code they committed. What about the "senior person" who has no idea about the code they committed? What about the CISO who doesn't understand that pasting proprietary documents willy nilly into the LLM's gaping maw might have legal/security/common sense implications, and that it is his job to set policy on such behavior? What about the middle manager who doesn't even try to retain the most experienced dev in the company because "we don't need the headcount anymore, now that Claude is so fast"? What about the company eating its own seed corn because every single junior position has been eliminated and there are no plans for the future anymore? What about the filesystem developer who fell in love with his chatbot girlfriend and is crashing out on Discord? Oh wait, scratch that last one. He left the company and is crashing out on his own. Carry on, then.

Supermancho 6mo ago
Indeed. Any developer who has used copilot knows you can't rely on it 100% The post's head image immediately bothered me. Copilot's strength is not on patching SDLC but to speed up the catching of typos and minor oversights. If you're using it as an integral part of SDLC, it causes problems immediately. So why posit the strawman? Marketing.
lbrito 6mo ago

alaudet 6mo ago

As a 3 year retired Systems Analyst I feel bad for my younger colleagues. In 2023 I was one of the first in my team to use AI to untangle some legacy code that did something mission critical with Perl and whose original author had long ago left and apparently didn't understand anything about actually commenting code or documentation. We were all in awe of this new technology that got us out of a bind. But more and more it looks less like a tool that is available to you instead of something that is being _done_ to you. Nobody asked for this. At what point is inspiration and thought just devalued and worthless in the name of doing things instantly. The work has no soul.

deleted 6mo ago
[deleted]

ch_ase 6mo ago

It’s been helpful for me to look at the promise of AI by comparing with the dotcom boom. Lots of similarities. But the internet was a simpler concept for businesses. Basically it was you can now sell to people from their computers. AI’s promise is what? It can approximate reasoning about things? This is much more challenging implementation puzzle to truly solve. I don’t know that I’ve seen anything of real substance outside coding tasks yet.

cmiles8 6mo ago

There are some improvements on coding and speed of developers, but more broadly in the enterprise AI is just producing a lot of slop that folks are getting fed up with. AI content has a look and feel people sense immediately. It’s amazing to see how quickly things shifted from “wow this is so cool, AI is going to change everything” to folks calling out “you lazy bum, this just looks like some slop you threw together with AI… let’s get some real thinking please.” We are firmly heading into “trough of disillusionment” territory on the hype cycle.

netcan 6mo ago

Path dependencies between invention and utilization... are complicated and hard to fathom. Our mental models of developments like the industrial revolution, literacy, printing or suchlike tend to be a lot more straightforward than how things play out in practice. When a bottleneck is eliminated... you tend to shortly find the next bottleneck. Meanwhile, there is an underlying assumption everyone seems to make that "more software, more value" is the basic reality. But... I'm skeptical. To do lists, wishlists, buglists and road maps may be full of stuff but... Visa or Salesforce have already exploited all their immediate "more software, more money" opportunities. The ones in a position to easily leverage AI are upstarts. They're starting with nothing. No code. No features. No software. With Ai, presumably, they can produce more software and make value. Also... I think overextended market rationalism leads people to see everything as an industrial revolution...which irl is much more of an exception. The networked personal computing revokution put a pc one every desk. It digitized everything. Do we have way better administration for less cost? Not really. Most administrations have grown. Did law fundamentally change dues to dugital efficiency? No. Not really. If you work on a terrible enterprise codebase... it's very possible that software quality/quantity isn't actually that important to your organization.

cyanydeez 6mo ago
>If you work on a terrible enterprise codebase... it's very possible that software quality/quantity isn't actually that important to your organization. It's possible capitalism will drive all enterprise to terrible codebases.

dominictorresmo 6mo ago

It's just ass to work in this area now. In the company I work, the bosses let everyone use it, even non-developers. I really want to quit and work in another area but unfortunately where I live a beginning salary can't pay a rent and I'm getting old

luodaint 6mo ago

[flagged]

threecheese 6mo ago

TIL my $company has used the same consultants as this guy. We started with Training and Champions, to Leadership/Lab/Crowd with a CoE/brown bags. We are definitely struggling with the same issues author describes, but even worse the leaders down at the Crowd level have some perverse need to achieve reuse across their teams, rather than letting their Crowd experiment. One team does something interesting, we must stop and get that thing out to all teams in that group, so everyone “benefits”. This is a scarcity mindset, which made sense pre-AI where code was costly and ideas were more valuable. At the same time, everyone not only has to do their work, they need to be 25% more efficient from AI (new KPIs), and so their own learnings slow to a halt, and the team with the cool idea has to give presentations instead of hacking.

romaniv 6mo ago

> "Where is the ROI for the 2 mio € we paid Anthropic last year?" The bias in the assumptions here is absolutely bonkers. Problem: GenAI is not generating any visible return on investment. "Solution": rearrange your entire development organization around the technology and start inventing new tooling. What's entirely obvious is that the point of such articles is not the stuff they purportedly discuss, but the normalization of assumptions those discussions are based on.

paodealho 6mo ago
LLMs can't fail, they can only be failed ... by you!
- Terr_ 6mo ago
  Hence OpenAI sinking $4b into "The Deployment Company", to pump up the number of Very Helpful Consultants which can offer to help your company overcome is tragic failure to adopt and buy, for a small fee...

aykutseker 6mo ago

The Hub captures decisions humans made. It can't see the ones they didn't, which is most of what AI ships. You instrument the deliberate half and inherit the undeliberate one.

raffael_de 6mo ago

My biggest gripe with language models is that technical and conceptual discussions which used to be led organically (with people having to think about what others wrote and decide what to reply themselves) now turned into AI slop avalanches with participants just copy pasting obviously generated text into the discussion. And those texts are always very long and super weird to respond to because they usually are overall correct enough so you can't just reject them but are flawed all over the place, missing the point, lacking depth where it would be important, skipping over important steps. This is a huge time waste. Funnily, many people have no idea how obvious it is that their texts are generated and rubbish at that.

ncr100 6mo ago

In many midsize companies I've worked as an engineer, developing a growth plan is something I have been asked to do, and to share it with my manager. Then at the end of the quarter I go over the growth plan and the manager agrees or disagrees with my actual growth, and gives me a performance bonus befitting my success.. I propose employees create self-training byproducts as a result of any AI interaction. And then they also work with their Cuban manager to make sure that these self-training byproducts are a part of their growth plan. This can guarantee growth without losing that opportunity To interact with the intelligent AI system (on topics that are relevant to the company's short, mid, and long-term strategic advantage,).

nelfrancis 6mo ago

[flagged]

conorliu 6mo ago

[flagged]

Knowledgee_KZA 6mo ago

[flagged]

chrisjj 6mo ago

> The company may still learn almost nothing. Not a problem if the hired "AI" now does that job. /i

dakiol 6mo ago

If you're a regular engineer like me, there's no real upside to using AI in a company setting. They're boiling us. Of course, the HN elite (investors, execs, celebrities, and top-tier engineers) will say otherwise because "how can you be against innovation man?" AI/LLMs aren't innovation the way TCP/IP, linux, or postgres were. To be clear: claude/codex/gemini/grok/whatever exist for profit, to squeeze the last drop of productivity out of you until there's nothing left, and then you're disposable (laid off). If you like AI, use open source models, use them in your side projects.

asdfman123 6mo ago
1) The game is not ending, it's changing. AI can sling a lot of code but we still need engineers that actually understand what the hell is going on. That's always been the bottleneck. It could eliminate junior positions, but seniors are fine for now. 2) It's been a hard lesson for me to learn because I'm naturally a contrarian, but you are hired to do what management wants you to do. If you resist, your best bet is to hope they don't notice or care, but it's not going to change much.
- herpderperator 6mo ago
  If you eliminate juniors over the next few years, there will be no seniors for the future when the current seniors retire.
  - sanderjd 6mo ago
    Yep, I'm a fan of the current crop of AI tools because they are incredibly useful to me, but I have deep concerns about the pipeline problem.
  - Our_Benefactors 6mo ago
    Then pay will go up again for those mid-level developers who still remain, and companies will again overhire and overtrain like we saw during COVID years. “We won’t have any seniors in ten years!!1!” is a handwringy problem that self solves by the free market.

woeirua 6mo ago

The hype is extreme right now and everyone is still trying to figure out how to use the tools. People who are further along the bleeding edge are trying to tear down all the process that we used to have to further improve velocity. After people go all the way to "dark factories" most companies will realize that they don't actually have any good ideas for what to build, and honestly never did. They've been coasting for years, someone else can replicate their product now and its just a race to the bottom. At that point, token budgets are going to collapse. I'm staunchly pro-AI as a technology, but I do think the bubble is going to pop in the next year or two just because the business value won't materialize for most companies fast enough.

a96 6mo ago
Enshittification of the AI clouds will also pick up speed. Tokens will get more costly, terms and tiers will turn worse etc. And there will be more security incidents on all fronts.

trenchgun 6mo ago

Article is mostly LLM written slop.

scotttaylor 6mo ago

[flagged]

theuniverseson 6mo ago

[flagged]

nickchen778 6mo ago

[flagged]

OutOfHere 6mo ago

Link is dead.

lbrito 6mo ago

>This cannot become employee surveillance...If people believe the organization is measuring whether they used enough AI, they will game the signals. It already has; ship has sailed. https://blog.pragmaticengineer.com/the-pulse-tokenmaxxing-as-a-weird-new-trend/ https://blog.pragmaticengineer.com/the-pulse-tokenmaxxing-as...

throw_m239339 6mo ago

I love these posts because they are missing the elephant in the room. The company learns nothing, it develops no knowledge, however, OpenAI, Anthropic and co services are absolutely learning from the people that use these tools. You are training your own replacement.

Amber-chen 6mo ago

[flagged]

hnisforrtards 6mo ago

[dead]

Roark66 6mo ago

In the current "big co" I'm working for it's a bit ridiculous. While AI tools have been provided pretty quickly (over a year ago, I initially used gemini cli, then copilot once it added anthropic models) the management is absolutely clueless about it. The top wants agents. Every team is asked few times a week "what autonomous agents will you build next" and answers the current AI lacks agency required not to mess up critical long running tasks and generate even more work are falling on deaf ears. (also ideas such as, why don't we setup a wiki page were teams can post their repetitive tasks and we can use AI to script them, are considered "not fast enough" - just build it... but we are the automation team, we automated everything we do years ago :-) Middle managers on the other hand suddenly started giving juniors senior's work and asking seniors "tell them (juniors) how to prompt it". Seriously? How about I prompt it myself instead? Oh, but it makes a shit load of architectural errors and boobie traps the junior will fail to find... So now instead of a cursory glance I have to spend an hour reviewing a small PR from them. And any questions about "why are you creating a new X for this instead of extending the existing one?" are met with blank panicked stares... The essence of this BS is contained in my description of the recent "Copilot Review" incident. We sometimes merge the same Github workflow files (10 line files) to dozens of repos, we have to obtain approvals for the PRs from a bunch of teams working in different timezones, but the merge has to be done everywhere at once and it has to be coordinated with other work. As we were on a day of such task some "helpful hand" enabled Copilot PR reviews for the whole org. Copilot helpfully opened 7 or 8 discussions on each PR giving us such precious advice as "your concurrency group uses the commit sha as a differentiating factor, this will allow multiple runs to proceed concurrently" to which one is tempted to answer "no shit sherlock". We suddenly had almost 200 conversations to "resolve" an hour before the merge and a bunch of approvers didn't give their approvals because "there is a discussion". Thankfully we had copilot that wrote us a script in 5 minutes to resolve the problem caused by itself... Maybe our next overnight agent can go over all our open PRs and close Copilot Review conversations with appropriate messages?