5 ms·

The Economic Benefit of Refactoring

286 points by javaeeeee 2mo ago

defrim 2mo ago
Being a bit cheeky here -- the amount of comments on this post is a telltale sign of how common / desirable refactoring is for the majority of developers (xD). It isn't our fault though, maybe if those pesky managers read this article then they would understand..
- sltr 2mo ago
  It's been on HN for 45 minutes. It's before 9am on the west coast
imoverclocked 2mo ago
It’s interesting that cyclomatic complexity or cognitive complexity scales with token usage. A codebase that enforces upper bounds on one of these two also (potentially) helps AI agents stay efficient.
vaylian 2mo ago
Interesting take-away: > Claude is unable to look at code, look at refactorings in general and work out which are suitable to apply: a human needs to actively guide it. Claude is happy to produce a very large Rust file. But you need human guidance to make it smaller.
- vehemenz 2mo ago
  "I would have written a shorter letter but did not have the time"
sltr 2mo ago
the punchline: "Refactoring reduces token consumption" I appreciate the effort to quantify the benefit rather than pontificate. It's worth mentioning Martin Fowler wrote a whole book on refactoring [1], in which he states, "to refactor, the essential precondition is [...] solid tests", which I think is the real benefit here, AI or not. Good tests protect against regressions, whether human or robot. They also help encode the spec, which humans and robots can read. [1] https://www.oreilly.com/library/view/refactoring-improving-the/9780134757681/ https://www.oreilly.com/library/view/refactoring-improving-t...
- pmg101 2mo ago
  (Just to note that although the article is on martinfowler.com Martin is not the author. It's attributed to Thoughtworks CTO Giles Edwards-Alexander.)
  - sltr 2mo ago
    Thanks for catching that. Updated my post
  - benburton 2mo ago
    Oh. That's kind of disappointing.
- dash2 2mo ago
  I think the real punchline was that the value saved was on the order of cents! > Every single change that touches the data access layer from this point forward now costs significantly less. > How much of a saving? Assuming Sonnet 5 pricing at the time of writing of $3/MTok, 39.7 cents. Now consider OpenAI's price drop, and open models, and consider that in the long run tokens will get cheaper. And think that the refactor needs to be human guided at a price of what for a senior developer - $100/hour?
  - disgruntledphd2 2mo ago
    > I think the real punchline was that the value saved was on the order of cents! If the agents can read less tokens in the future, and make changes more effectively then this would add up over time.
theturtletalks 2mo ago
Make it work, make it right, make it fast, in that order Make it right is the refactoring
pmg101 2mo ago
It's very interesting to have some data on this. It matches my experience which is that LLMs greatly benefit from well factored code, but are not particularly adept at creating such code. Much like most human developers I suppose!
- jerf 2mo ago
  I've been budgeting myself explicit "slop removal" time. In fact I'm in it right now one window to my right here. I still get a big win from AI on the net, but you do need to budget some time to clean up. I'm still on team "read every line". In fact this is a case where I deliberately deferred some review because I was a blocker for another team. Now that I've got something to them I'm going back and I'm going to eat a bigger chunk of debt than I normally would, but it's worth it for unblocking the other team sooner. AI has made tech debt easier to take out, in all senses of that term. It is also pretty decent, in my experience, at being guided into how to fix tech debt. Some other people's experience varies: https://news.ycombinator.com/item?id=49035455 https://news.ycombinator.com/item?id=49035455 YMMV.
  - tengbretson 2mo ago
    It's totally worth doing. Especially since it only takes one or two manual improvements before you can just say "ok, replicate this new pattern throughout the codebase. Go."
- j45 2mo ago
  Out of the box, this is what I've found too. I've had some luck while refactoring by helping it shape how to refactor, which can improve how well factored code should look like. Providing examples of well factored code can go a long way, even if it's an open source repo of what to do / not to do.
danbruc 2mo ago
Interesting that the amount of code remained essentially unchanged. In my experience it is not unusual that refactoring messy code cuts the number of lines in half.
- stingraycharles 2mo ago
  It can go both ways. Lines of code is a terrible metric for pretty much any goal that refactorings are meant to tackle.
  - NichoPaolucci 2mo ago
    I don’t know. LOC to me is indispensable for certain refactoring goals in my opinion. Taking a component and turning it from 7K lines to 3K lines and maintaining functionality obviously means there’s less complexity introduced, less to go wrong now, and less overhead to modify in the future. Sure it can go the other way, the component needs to support something it might need, we need to adjust larger patterns, this function needs to be refactored into something more robust. But lines of code is a pretty decent metric of success for “trimming down and cleaning up” style refactoring, to me at least. It’s not everything of course, but it’s definitely an indicator.
    - j45 2mo ago
      LOC can be one measure, but not always the sole or best ones. Since LLMs are word generators, and have a propensity to generating words, they need to be shaped to understand simplest is best, more isn't more, and less isn't more always. Trimming down and cleaning up could be formatting, standardization, commenting, or even some basic re-architecting that was overdue. One of the biggest benefits of llms for refactoring I'm finding is reducing technical debt.
    - Catloafdev 2mo ago
      That's LOC in a single file, that's a bit different from total LOC as a metric. Total LOC is a garbage metric. Things like reducing line count in specific files or components is a big benefit, but those lines are often moved, not dropped.
paxys 2mo ago
I have been enforcing this myself through AGENTS files and other explicit instructions. What’s crazy is that none of the existing coding harnesses include such instructions by default. Two lines in Claude Code and Codex and the entire world would be writing better, more efficient code. Makes me believe in the conspiracy that these companies want more verbose code and longer files because it directly results in higher token usage.
- Ginger-Pickles 2mo ago
  Pray do share those two lines
firasd 2mo ago
I think this is one of those things where a human in the loop is indispensable An agentic refactoring pass does make sense cause one LLM reviewing work can spot things the ‘generator’ LLM missed while focused on the initial task output But can the reviewer agent ever actually have in mind what this project actually is? And how the code all comes together to do the work involved? In other words what parts of the code are redundant or can be made more elegant Asking coding agents to refactor your codebase is maybe like asking trauma surgeons to increase your exercise capacity. The agents are gonna need a really holistic POV to do this properly I guess part of my point is that just splitting big files into multiple files is only refactoring in a superficial sense without having a theory of what code belongs together and what can be extracted into utility functions etc. Is splitting files actually like decomposing factors or is it like splitting a larger number into smaller numbers that still eventually get added together A good example of what I mean is that agents often don’t ~actually~ understand the whole system anyway. They might implement a system to store and calculate something that is already being fetched via API. Humans often have a dual perspective — a holistic sense of the project and (when applying our mind to a task) a precise scalpel: ‘oh if we just look at this this JSON it has a key with this data already’
- j45 2mo ago
  I was able to largely refactor something that was going to take months in a week or so by being the narrating human in the loop. It was helped by having refactoring experience and approaches to codebases by others, and in my case, being the original architect and being able to speak to the original and current intents, where needed. This was using a less common, but capable and easy language for the LLM without a ton of dependancy brittleness to manage.. once the effort to remove javascript/python bias was in place, it became so powerful that once the lightbulb went on, it really got cruising. The project was playing in the world of JSR-223 languages, where you could script in many popular languages, but it all got to run in the JVM, which was an environmental requirement. https://en.wikipedia.org/wiki/Scripting_for_the_Java_Platform https://en.wikipedia.org/wiki/Scripting_for_the_Java_Platfor...
- zuzululu 2mo ago
  Your take is outdated. Agents are excellent at refactoring now. If you told me what you wrote during an interview, that would be grounds to not continue. It's very important that you are using coding agents with the latest frontier models and know exactly what it can't and can't do if you want to be hired in this market.
vehemenz 2mo ago
Opus/Fable 5 (or really, any thinking model) do a better job of refactoring than Sonnet would. I think the takeaways are still valid, just not as valid when using a more expensive model at a higher effort level.
BenoitEssiambre 2mo ago
The benefits go beyond reducing token consumption. Compact contexts also foster better reasoning, enable intelligence across more layers if you can load them in a single context, and writing software to enable this, results in more correct software, software that _generalizes_, that has higher probability of being correct not just for tested cases, but for the interpolations and extrapolations of these cases. Refactoring towards good abstractions is more powerful than people realize. There's information theoretic bayesian math to back this up. It's a bit of a divine coincidence that software that is more economically and energy efficient to process and run tends to also be more correct. It's all about reducing the entropy of your code. https://benoitessiambre.com/entropy.html https://benoitessiambre.com/entropy.html
- hammock 2mo ago
  > It's all about reducing the entropy of your code In all corners of our world and the universe at large, reducing entropy in anything can be thought of as “building.”
  - HappMacDonald 2mo ago
    All I heard was "compression is intelligence"
- golemotron 2mo ago
  Imagine what happens when human readability is discarded as a goal. LLMs are already very good at inferring meaning with little context. If the objective function is reduction of token consumption, it's hard to know where we will end up.
kattenelvis 2mo ago
You can tell this is BS because there's no way that app is actually functioning at all. Fully vibe-coded, all that? No, I don't believe it.
t2ance 2mo ago
Can't agree more. Feels like LLMs are not able to understand your requirements at a high level and always add complexity by default. Really need a handbook to guide them to do those cleanups. Obviously they know the knowledge in their weights but they just don't know how to apply it.
Natalia724 2mo ago
[flagged]
pragmatic 2mo ago
Agent code can only be read and understood by agents. We’ve reached the point where people just can’t comprehend these giant code blobs. Feature, bug or emergent property? I don’t think the distinction matters as much as the reality. We’re being locked into using the AI tooling bc the code was generated with AI tooling. These giant files of doom were being generated by humans anyway and were very hard to work with. With LLMs it’s at last manageable or feasible to edit, refactor etc. I honestly think LLMs are going to save us from ourselves as the codebases became too large and “messy” for humans to comprehend. (Mono repos of doom) On a personal level these giant files are abhorrent but that’s just personal taste and I don’t think any of the Martin Fowler refactor/cleanup stuff is going to matter at all anymore. Kinda sad on some level.
- deleted 2mo ago
  [deleted]
- Catloafdev 2mo ago
  > Agent code can only be read and understood by agents. This just fundamentally isn't true and if this is your perspective then you're using LLMs wrong.
  - ahalay-mahalay 2mo ago
    Fundamentally, the code can be understood by humans, but the volume of the generated code outpaces human capacity to understand it.
- apsurd 2mo ago
  There was always bad code. Extend that to the general challenge of a bad hire doing net-negative things for the company. AI is a new problem because it 1000x a bad hire's blast radius. Personally, I'd even state that AI tends to turn an average and sometimes-but-not-always-good hire into bad hires because finally they get to do the thing they've always wanted at lightning speed in a way that previously the company red-tape wouldn't allow.
- antonvs 2mo ago
  > These giant files of doom were being generated by humans anyway Right, I've never seen an agent produce code anywhere near as bad as some of the human-generated code that I've worked on. And, if your agent is producing huge files or functions, you can just tell it not to and it'll comply.
knighthacker 2mo ago
[flagged]
whats_a_quasar 2mo ago
This is such a nice piece, this is how people should write about AI. Specific, grounded to how the tools are actually being used, and quantitative. There is so much bad AI commentary that is incredibly vague, divorced from any actual use cases, or written by people who don't actually use the tools. It is good to see a critique that is "here is a thing AI is bad at and measurements to show it" rather than gesturing "here is why I think AI is problematic for society." It is rather different but another piece of research I liked for the same reason was this report that interviewed Boko Haram members about how they used AI to assist terrorism. You get these interminable online debates that are so unproductive and reporting that is specific is such a breath of fresh air. https://casp.ac/reports/ai-enabled-terrorism https://casp.ac/reports/ai-enabled-terrorism
- jimbokun 2mo ago
  Yes, more like this please, universe!
  - gga 2mo ago
    I’ll try to keep it up.
Viliam1234 2mo ago
I find it funny how the best practices for programmers, ignored in most IT companies, get reinvented as the best practices for AIs. Boring: The documentation should be in code, not in external Word documents uploaded to the company SharePoint server. Exciting: The documentation for the AI should be in code, not in external Word documents uploaded to the company SharePoint server. Boring: You should give your developers the big picture of the project, not just micromanage them using Jira tasks. Exciting: You should give your AI the big picture of the project in CLAUDE.md, not just micromanage it using prompts. Boring: Refactoring makes your developers more productive in long term. Exciting: Refactoring makes your AI more productive in long term.
- Ericson2314 2mo ago
  No this isn't mindless reinventing --- this is finally having clear empiric evidence for something that we knew the entire time. This is a huge relief! Next up is demonstrating the AI is more productive with better programming languages.
  - tikhonj 2mo ago
    I've already seen at least one promising experiment about how static checking helps LLMs: https://arxiv.org/abs/2606.01522 https://arxiv.org/abs/2606.01522 Key parts of the abstract: > This raises a question the programming-language community has not previously had reason to ask: should error-message detail be calibrated differently for AI agents than for humans? > We investigate this question through a controlled experiment using Shplait, an ML-style statically typed language. We construct a suite of programs containing a single deliberate type error each, and measure how often an AI agent repairs them under ablation: a detailed error context using the unification stack; a proximate error location; a minimal type error; and a dynamic (test suite) error only. An automated oracle uses a test suite to classify each repair attempt as a type error, semantically incorrect, or semantically correct. > We find concrete evidence that more detailed error messages generally improve an agent's ability to fix type errors. We also find that the presence of a type system appears to help more than only test suite failure reports.
  - kagevf 2mo ago
holtkam2 2mo ago
Great piece, but it misses the elephant in the room: the lion's share of economic benefit from refactoring will come from the fact that it makes it easier for humans to understand. That means 3am pages get resolved faster, fewer bugs will end up in prod, and your team can ship faster than your competition - gaining a leg up in the market. Most importantly, folks will be more comfortable accepting responsibility and ownership of a system when they understand it... this means that if/when something goes wrong, people will more quickly jump in and fix it, and when things could be better, folks will jump in and improve it.
gowld 2mo ago
> Claude.ai was better than Claude Code Why is this something users have to distinguish? Why can't these generative AIs choose good names for things?
ChulioZ 2mo ago
I can very much relate to the experiences described in the article. In my projects, I have multiple Claude skills and rules aiming at making my files (the code, but also the Claude files themselves) more token-effective. I have seen huge differences in token usage before and after invoking those skills, which I regularly do as part of having Claude audit my projects; although I haven't measured them. Those projects are (like the one in the article) fully developed by Claude Code, so I found such audits and mindful token usage very necessary.
janpeuker 2mo ago
> Claude is unable to look at code, look at refactorings in general and work out which are suitable to apply I am just sitting here waiting for Grady Booch to write "Architecture!"
jimbokun 2mo ago
> Add a new ItemWatchStore public async trait to the Firestore layer, following existing patterns exactly. The trait must have three methods: async fn watch_item(&self, item_id: &str, user_id: &str) -> Result<()> async fn unwatch_item(&self, item_id: &str, user_id: &str) -> Result<()> async fn watched_items_for_user(&self, user_id: &str) -> Result<Vec<String>> This shows the limitations of vibe coding. It takes someone with a long history of software development to prompt for something like this. Even though the model is writing 100% of the code, still needed someone with a lot of programming knowledge to write the prompt.
- skydhash 2mo ago
  After reading a fair amount of OSS code, it's rather glaring that the data structures and algorithms are more like atoms than molecules in software design. We do have some molecules in the Design patterns, but they are more suitable to the OOP universe, like the collection api (filter, map, take,...) when dealing with groups of objects or the reactive api for dealing with concurrent tasks. Most of software development is looking at some process and then decomposing it recursively until you get to those molecules/atoms of the computing world. Coding them is trivial, and while you can gain a certain boost from the AI, after a while you no longer have to write that much code. It will turn into a balancing act where the introduction of a new concept has to be done carefully.
persedes 2mo ago
Was hoping for a process diagram on how refactoring removes bottlenecks during software developement and allows you to ship faster etc :) Still a nice writeup and love how these meta analysises (presumeably) done via AI can now easily capture metrics that inform your workflow.
awsglkhj 2mo ago
I wonder, how close can we get to a reasonable simulator for testing software engineering practices before inflicting them onto the real world?
jmartrican 2mo ago
The author said that Claude was not good at refactoring. But the author was using Sonnet 5. I supect Opus 5 would have done a lot better at refactoring. It would have come up with a multstep approach, i suspect.
- gga 2mo ago
  I used Opus 4.8 to create the refactoring plans. I converted token savings using Sonnet 5 pricing at the time of writing as I'd expect to use that for more 'straightforward' feature additions.
- the__alchemist 2mo ago
  Opus 5 is old news and has lots of problems. Minuet 6.9 is the one to use, and fixes all the problems people have with LLMs.
jmartrican 2mo ago
I think the name of the game is going to be "how do we maximize output while keeping token counts low". For example, finding ways to replace AI workflows, even if minor ones, with scripts/code. Anthropic told me that I have till August 19th to be get my act together becuase they are going to reduce my token count by 50%. lol. I have been abusing my Pro Max allowance and need to start being less wasteful. Articles like this, can help us come up with ideas on how to do it.
nycticorax 2mo ago
This is interesting, but I think it would be even more interesting to see a comparison of the token cost of adding a new feature, in the original codebase vs the refactored one. You'd think/hope that the cost of adding the new feature would be lower when starting from the refactored codebase, indicating that refactoring makes economic sense in the long run by lowering the cost of adding new features.
- gga 2mo ago
  That's what I've shown in this article. The token consumption shown is for adding a new feature (the same feature each time -- in a sub-agent) after every refactoring is applied. I didn't actually capture the token cost of the refactoring. That was a miss, and I'm making sure I do that in the future.
  - nycticorax 2mo ago
    Kind of you to engage! Ahh, ok, I misunderstood. Very cool! I think maybe it would have helped me if you had given a one-or-two-sentence description of the "representative change" in the main text, to make it a bit more concrete up-front? But possibly just a 'me' issue...
sharpvik 2mo ago
[flagged]
neet_dev 2mo ago
[flagged]
okzgn 2mo ago
Great point. Using good software architecture and clean code will deliver direct and measurable economic value to projects and reduce AI costs right from the start, whether during refactoring or when planning a logical reorganization.
solarized 2mo ago
Putting this in agent.md didnt help? "always give ultimate decent code in term of maintanibility, cleaness, and perf".
Ozzie-D 2mo ago
[flagged]
wolttam 2mo ago
> and used tiktoken to approximate tokens, by dividing character count by four. This is just the type of thing that stands out. You used a tokenizer library to determine the number of tokens in your text by dividing its length by 4. That makes no sense!
rocky_raccoon 2mo ago
I really enjoy refactoring. Like, doing it by hand (5 miles uphill in the snow both ways) rather than AI. I don't even understand why I enjoy it, because if done right, there's no visible change. When people ask me why I'm so pumped up after having worked on the codebase all day long, I can't give them any sort of answer that makes sense (coming from a small business with a smaller dev team). It's just... "I'm future-proofing our website and we won't see any direct results today but things will be so much easier going forward..." There's something about the puzzle. Looking at my old, deranged coding workarounds that tried to solve problems that have already been solved a thousand times before with established paradigms; and then moving them toward said best practices; and doing it in a way that no NEW technical debt is created. It's just satisfying. I think one of the best learning experiences for me has been the fact that I created a bunch of sloppy shit by hand, auth and all, which forced me to learn things the hard way. All along the way, people were shouting from the rooftops: "use established libraries, dummy!", which is the same advice I would give to somebody today. But by doing things the hard way, I learned so much more about the inner workings. And, I've also given myself a decade's worth of refactoring work, which I really enjoy!
- nevster 2mo ago
  Probably it's just the same dopamine hits that make things like watching the Windows 98 Defragger soothing.
- HappMacDonald 2mo ago
  I'm curious how heavily you have test suites built up as guardrails while doing this, to prevent regressions and such?
  - mattmanser 2mo ago
    I have often done huge refactors over the last 20 years on systems where there are none. It's very satisfying turning a 1,000 line class file into 100 lines. But all my refactors have been in statically typed languages. To speculate on the reason why, which the GP asked, I think perhaps it is two things. It's the same satisfaction from tidying a room or your workbench. But it's also slightly narcissistic unfortunately. I think you've shown you're better than the other programmers. Much, much, better when you end up with massive reductions in TLOC. And with tests, the irony is that the resulting code is often so much easier to reason about, you start spotting really obvious bad assumptions in the original implementation. It's one of the reasons I've always been fairly skeptical about the true value of unit tests (integration, I get). That and the fact that projects I've worked on that did have unit tests catch like 1 bug a year. Maybe it's just the size of systems I traditionally work on (smaller teams, or even 1 person teams, so man-years worth of effort rather than decades or centuries).
hiAndrewQuinn 2mo ago
>The goal of refactoring an agentic code base is to spend tokens now in refactoring to make token consumption for future work lower. Couldn't have said it better myself. Deciding to refactor should be like a discounted cash flow analysis but for tokens!
4b11b4 2mo ago
I've been experimenting with not only refactoring but rewriting history. I'm always moving around commits, inserting, splitting things up. The amount of times I've instructed to "tell the story right the first time"... More quantitative and qualitative analysis to come
benrutter 2mo ago
Tangent, but I have a theory that refactoring is one of the best symptoms of a healthy dev team. It's partly that refactors themselves have benefits, but I think more that the benefits to refactoring aren't visible to something like product-owners, feature tickets, etc. If teams are refactoring to ensure the health of the overall software, it's a tell-tale sign that developers are happy making recommendations for good software, and that those recommendations are being taken seriously. I think Martin Folwer might have actually coined the term "software rot" - either way, as an issue it happens most severely when a team either aren't motivated or empowered to build their vision of high quality software. When a team can follow their judgement of excellence, that's usually a great sign! (and yes, obviously this can go to far, there are probably some teams who rewrote all their stuff in Ruby then Node then Rust and now something else to be "agent native", but in the coorporate world, I see a lot less of this than teams who just don't feel like they have permission to improve things)
L-patpat 2mo ago
[flagged]
MS8080 2mo ago
[flagged]
ukoki 2mo ago
I have a blanket, unignorable 'files must be <= 1000 lines' lint for my Rust projects for exactly this reason. Exploring large files is _costly_ for agents, so it's much better to lean into the filesystem hierarchy to a greater extent than you would normally with a team of human developers.
supportm 2mo ago
[flagged]

286 points by javaeeeee 2mo ago

Being a bit cheeky here -- the amount of comments on this post is a telltale sign of how common / desirable refactoring is for the majority of developers (xD). It isn't our fault though, maybe if those pesky managers read this article then they would understand..

sltr 2mo ago
It's been on HN for 45 minutes. It's before 9am on the west coast

imoverclocked 2mo ago

It’s interesting that cyclomatic complexity or cognitive complexity scales with token usage. A codebase that enforces upper bounds on one of these two also (potentially) helps AI agents stay efficient.

vaylian 2mo ago

Interesting take-away: > Claude is unable to look at code, look at refactorings in general and work out which are suitable to apply: a human needs to actively guide it. Claude is happy to produce a very large Rust file. But you need human guidance to make it smaller.

vehemenz 2mo ago
"I would have written a shorter letter but did not have the time"

sltr 2mo ago

the punchline: "Refactoring reduces token consumption" I appreciate the effort to quantify the benefit rather than pontificate. It's worth mentioning Martin Fowler wrote a whole book on refactoring [1], in which he states, "to refactor, the essential precondition is [...] solid tests", which I think is the real benefit here, AI or not. Good tests protect against regressions, whether human or robot. They also help encode the spec, which humans and robots can read. [1] https://www.oreilly.com/library/view/refactoring-improving-the/9780134757681/ https://www.oreilly.com/library/view/refactoring-improving-t...

pmg101 2mo ago
(Just to note that although the article is on martinfowler.com Martin is not the author. It's attributed to Thoughtworks CTO Giles Edwards-Alexander.)
- sltr 2mo ago
  Thanks for catching that. Updated my post
- benburton 2mo ago
  Oh. That's kind of disappointing.
dash2 2mo ago
I think the real punchline was that the value saved was on the order of cents! > Every single change that touches the data access layer from this point forward now costs significantly less. > How much of a saving? Assuming Sonnet 5 pricing at the time of writing of $3/MTok, 39.7 cents. Now consider OpenAI's price drop, and open models, and consider that in the long run tokens will get cheaper. And think that the refactor needs to be human guided at a price of what for a senior developer - $100/hour?
- disgruntledphd2 2mo ago
  > I think the real punchline was that the value saved was on the order of cents! If the agents can read less tokens in the future, and make changes more effectively then this would add up over time.

theturtletalks 2mo ago

Make it work, make it right, make it fast, in that order Make it right is the refactoring

pmg101 2mo ago

It's very interesting to have some data on this. It matches my experience which is that LLMs greatly benefit from well factored code, but are not particularly adept at creating such code. Much like most human developers I suppose!

jerf 2mo ago
I've been budgeting myself explicit "slop removal" time. In fact I'm in it right now one window to my right here. I still get a big win from AI on the net, but you do need to budget some time to clean up. I'm still on team "read every line". In fact this is a case where I deliberately deferred some review because I was a blocker for another team. Now that I've got something to them I'm going back and I'm going to eat a bigger chunk of debt than I normally would, but it's worth it for unblocking the other team sooner. AI has made tech debt easier to take out, in all senses of that term. It is also pretty decent, in my experience, at being guided into how to fix tech debt. Some other people's experience varies: https://news.ycombinator.com/item?id=49035455 https://news.ycombinator.com/item?id=49035455 YMMV.
- tengbretson 2mo ago
  It's totally worth doing. Especially since it only takes one or two manual improvements before you can just say "ok, replicate this new pattern throughout the codebase. Go."
j45 2mo ago
Out of the box, this is what I've found too. I've had some luck while refactoring by helping it shape how to refactor, which can improve how well factored code should look like. Providing examples of well factored code can go a long way, even if it's an open source repo of what to do / not to do.

danbruc 2mo ago

Interesting that the amount of code remained essentially unchanged. In my experience it is not unusual that refactoring messy code cuts the number of lines in half.

stingraycharles 2mo ago
It can go both ways. Lines of code is a terrible metric for pretty much any goal that refactorings are meant to tackle.
- NichoPaolucci 2mo ago
  I don’t know. LOC to me is indispensable for certain refactoring goals in my opinion. Taking a component and turning it from 7K lines to 3K lines and maintaining functionality obviously means there’s less complexity introduced, less to go wrong now, and less overhead to modify in the future. Sure it can go the other way, the component needs to support something it might need, we need to adjust larger patterns, this function needs to be refactored into something more robust. But lines of code is a pretty decent metric of success for “trimming down and cleaning up” style refactoring, to me at least. It’s not everything of course, but it’s definitely an indicator.
  - j45 2mo ago
    LOC can be one measure, but not always the sole or best ones. Since LLMs are word generators, and have a propensity to generating words, they need to be shaped to understand simplest is best, more isn't more, and less isn't more always. Trimming down and cleaning up could be formatting, standardization, commenting, or even some basic re-architecting that was overdue. One of the biggest benefits of llms for refactoring I'm finding is reducing technical debt.
  - Catloafdev 2mo ago
    That's LOC in a single file, that's a bit different from total LOC as a metric. Total LOC is a garbage metric. Things like reducing line count in specific files or components is a big benefit, but those lines are often moved, not dropped.

paxys 2mo ago

I have been enforcing this myself through AGENTS files and other explicit instructions. What’s crazy is that none of the existing coding harnesses include such instructions by default. Two lines in Claude Code and Codex and the entire world would be writing better, more efficient code. Makes me believe in the conspiracy that these companies want more verbose code and longer files because it directly results in higher token usage.

Ginger-Pickles 2mo ago
Pray do share those two lines

firasd 2mo ago

I think this is one of those things where a human in the loop is indispensable An agentic refactoring pass does make sense cause one LLM reviewing work can spot things the ‘generator’ LLM missed while focused on the initial task output But can the reviewer agent ever actually have in mind what this project actually is? And how the code all comes together to do the work involved? In other words what parts of the code are redundant or can be made more elegant Asking coding agents to refactor your codebase is maybe like asking trauma surgeons to increase your exercise capacity. The agents are gonna need a really holistic POV to do this properly I guess part of my point is that just splitting big files into multiple files is only refactoring in a superficial sense without having a theory of what code belongs together and what can be extracted into utility functions etc. Is splitting files actually like decomposing factors or is it like splitting a larger number into smaller numbers that still eventually get added together A good example of what I mean is that agents often don’t ~actually~ understand the whole system anyway. They might implement a system to store and calculate something that is already being fetched via API. Humans often have a dual perspective — a holistic sense of the project and (when applying our mind to a task) a precise scalpel: ‘oh if we just look at this this JSON it has a key with this data already’

j45 2mo ago
I was able to largely refactor something that was going to take months in a week or so by being the narrating human in the loop. It was helped by having refactoring experience and approaches to codebases by others, and in my case, being the original architect and being able to speak to the original and current intents, where needed. This was using a less common, but capable and easy language for the LLM without a ton of dependancy brittleness to manage.. once the effort to remove javascript/python bias was in place, it became so powerful that once the lightbulb went on, it really got cruising. The project was playing in the world of JSR-223 languages, where you could script in many popular languages, but it all got to run in the JVM, which was an environmental requirement. https://en.wikipedia.org/wiki/Scripting_for_the_Java_Platform https://en.wikipedia.org/wiki/Scripting_for_the_Java_Platfor...
zuzululu 2mo ago
Your take is outdated. Agents are excellent at refactoring now. If you told me what you wrote during an interview, that would be grounds to not continue. It's very important that you are using coding agents with the latest frontier models and know exactly what it can't and can't do if you want to be hired in this market.

vehemenz 2mo ago

Opus/Fable 5 (or really, any thinking model) do a better job of refactoring than Sonnet would. I think the takeaways are still valid, just not as valid when using a more expensive model at a higher effort level.

BenoitEssiambre 2mo ago

The benefits go beyond reducing token consumption. Compact contexts also foster better reasoning, enable intelligence across more layers if you can load them in a single context, and writing software to enable this, results in more correct software, software that _generalizes_, that has higher probability of being correct not just for tested cases, but for the interpolations and extrapolations of these cases. Refactoring towards good abstractions is more powerful than people realize. There's information theoretic bayesian math to back this up. It's a bit of a divine coincidence that software that is more economically and energy efficient to process and run tends to also be more correct. It's all about reducing the entropy of your code. https://benoitessiambre.com/entropy.html https://benoitessiambre.com/entropy.html

hammock 2mo ago
> It's all about reducing the entropy of your code In all corners of our world and the universe at large, reducing entropy in anything can be thought of as “building.”
- HappMacDonald 2mo ago
  All I heard was "compression is intelligence"
golemotron 2mo ago
Imagine what happens when human readability is discarded as a goal. LLMs are already very good at inferring meaning with little context. If the objective function is reduction of token consumption, it's hard to know where we will end up.

kattenelvis 2mo ago

You can tell this is BS because there's no way that app is actually functioning at all. Fully vibe-coded, all that? No, I don't believe it.

t2ance 2mo ago

Can't agree more. Feels like LLMs are not able to understand your requirements at a high level and always add complexity by default. Really need a handbook to guide them to do those cleanups. Obviously they know the knowledge in their weights but they just don't know how to apply it.

Natalia724 2mo ago

[flagged]

pragmatic 2mo ago

Agent code can only be read and understood by agents. We’ve reached the point where people just can’t comprehend these giant code blobs. Feature, bug or emergent property? I don’t think the distinction matters as much as the reality. We’re being locked into using the AI tooling bc the code was generated with AI tooling. These giant files of doom were being generated by humans anyway and were very hard to work with. With LLMs it’s at last manageable or feasible to edit, refactor etc. I honestly think LLMs are going to save us from ourselves as the codebases became too large and “messy” for humans to comprehend. (Mono repos of doom) On a personal level these giant files are abhorrent but that’s just personal taste and I don’t think any of the Martin Fowler refactor/cleanup stuff is going to matter at all anymore. Kinda sad on some level.

deleted 2mo ago
[deleted]
Catloafdev 2mo ago
> Agent code can only be read and understood by agents. This just fundamentally isn't true and if this is your perspective then you're using LLMs wrong.
- ahalay-mahalay 2mo ago
  Fundamentally, the code can be understood by humans, but the volume of the generated code outpaces human capacity to understand it.
apsurd 2mo ago
There was always bad code. Extend that to the general challenge of a bad hire doing net-negative things for the company. AI is a new problem because it 1000x a bad hire's blast radius. Personally, I'd even state that AI tends to turn an average and sometimes-but-not-always-good hire into bad hires because finally they get to do the thing they've always wanted at lightning speed in a way that previously the company red-tape wouldn't allow.
antonvs 2mo ago
> These giant files of doom were being generated by humans anyway Right, I've never seen an agent produce code anywhere near as bad as some of the human-generated code that I've worked on. And, if your agent is producing huge files or functions, you can just tell it not to and it'll comply.

knighthacker 2mo ago

[flagged]

whats_a_quasar 2mo ago

This is such a nice piece, this is how people should write about AI. Specific, grounded to how the tools are actually being used, and quantitative. There is so much bad AI commentary that is incredibly vague, divorced from any actual use cases, or written by people who don't actually use the tools. It is good to see a critique that is "here is a thing AI is bad at and measurements to show it" rather than gesturing "here is why I think AI is problematic for society." It is rather different but another piece of research I liked for the same reason was this report that interviewed Boko Haram members about how they used AI to assist terrorism. You get these interminable online debates that are so unproductive and reporting that is specific is such a breath of fresh air. https://casp.ac/reports/ai-enabled-terrorism https://casp.ac/reports/ai-enabled-terrorism

jimbokun 2mo ago
Yes, more like this please, universe!
- gga 2mo ago
  I’ll try to keep it up.

Viliam1234 2mo ago

I find it funny how the best practices for programmers, ignored in most IT companies, get reinvented as the best practices for AIs. Boring: The documentation should be in code, not in external Word documents uploaded to the company SharePoint server. Exciting: The documentation for the AI should be in code, not in external Word documents uploaded to the company SharePoint server. Boring: You should give your developers the big picture of the project, not just micromanage them using Jira tasks. Exciting: You should give your AI the big picture of the project in CLAUDE.md, not just micromanage it using prompts. Boring: Refactoring makes your developers more productive in long term. Exciting: Refactoring makes your AI more productive in long term.

Ericson2314 2mo ago
No this isn't mindless reinventing --- this is finally having clear empiric evidence for something that we knew the entire time. This is a huge relief! Next up is demonstrating the AI is more productive with better programming languages.
- tikhonj 2mo ago
  I've already seen at least one promising experiment about how static checking helps LLMs: https://arxiv.org/abs/2606.01522 https://arxiv.org/abs/2606.01522 Key parts of the abstract: > This raises a question the programming-language community has not previously had reason to ask: should error-message detail be calibrated differently for AI agents than for humans? > We investigate this question through a controlled experiment using Shplait, an ML-style statically typed language. We construct a suite of programs containing a single deliberate type error each, and measure how often an AI agent repairs them under ablation: a detailed error context using the unification stack; a proximate error location; a minimal type error; and a dynamic (test suite) error only. An automated oracle uses a test suite to classify each repair attempt as a type error, semantically incorrect, or semantically correct. > We find concrete evidence that more detailed error messages generally improve an agent's ability to fix type errors. We also find that the presence of a type system appears to help more than only test suite failure reports.
- kagevf 2mo ago

holtkam2 2mo ago

Great piece, but it misses the elephant in the room: the lion's share of economic benefit from refactoring will come from the fact that it makes it easier for humans to understand. That means 3am pages get resolved faster, fewer bugs will end up in prod, and your team can ship faster than your competition - gaining a leg up in the market. Most importantly, folks will be more comfortable accepting responsibility and ownership of a system when they understand it... this means that if/when something goes wrong, people will more quickly jump in and fix it, and when things could be better, folks will jump in and improve it.

gowld 2mo ago

> Claude.ai was better than Claude Code Why is this something users have to distinguish? Why can't these generative AIs choose good names for things?

ChulioZ 2mo ago

I can very much relate to the experiences described in the article. In my projects, I have multiple Claude skills and rules aiming at making my files (the code, but also the Claude files themselves) more token-effective. I have seen huge differences in token usage before and after invoking those skills, which I regularly do as part of having Claude audit my projects; although I haven't measured them. Those projects are (like the one in the article) fully developed by Claude Code, so I found such audits and mindful token usage very necessary.

janpeuker 2mo ago

> Claude is unable to look at code, look at refactorings in general and work out which are suitable to apply I am just sitting here waiting for Grady Booch to write "Architecture!"

jimbokun 2mo ago

> Add a new ItemWatchStore public async trait to the Firestore layer, following existing patterns exactly. The trait must have three methods: async fn watch_item(&self, item_id: &str, user_id: &str) -> Result<()> async fn unwatch_item(&self, item_id: &str, user_id: &str) -> Result<()> async fn watched_items_for_user(&self, user_id: &str) -> Result<Vec<String>> This shows the limitations of vibe coding. It takes someone with a long history of software development to prompt for something like this. Even though the model is writing 100% of the code, still needed someone with a lot of programming knowledge to write the prompt.

skydhash 2mo ago
After reading a fair amount of OSS code, it's rather glaring that the data structures and algorithms are more like atoms than molecules in software design. We do have some molecules in the Design patterns, but they are more suitable to the OOP universe, like the collection api (filter, map, take,...) when dealing with groups of objects or the reactive api for dealing with concurrent tasks. Most of software development is looking at some process and then decomposing it recursively until you get to those molecules/atoms of the computing world. Coding them is trivial, and while you can gain a certain boost from the AI, after a while you no longer have to write that much code. It will turn into a balancing act where the introduction of a new concept has to be done carefully.

persedes 2mo ago

Was hoping for a process diagram on how refactoring removes bottlenecks during software developement and allows you to ship faster etc :) Still a nice writeup and love how these meta analysises (presumeably) done via AI can now easily capture metrics that inform your workflow.

awsglkhj 2mo ago

I wonder, how close can we get to a reasonable simulator for testing software engineering practices before inflicting them onto the real world?

jmartrican 2mo ago

The author said that Claude was not good at refactoring. But the author was using Sonnet 5. I supect Opus 5 would have done a lot better at refactoring. It would have come up with a multstep approach, i suspect.

gga 2mo ago
I used Opus 4.8 to create the refactoring plans. I converted token savings using Sonnet 5 pricing at the time of writing as I'd expect to use that for more 'straightforward' feature additions.
the__alchemist 2mo ago
Opus 5 is old news and has lots of problems. Minuet 6.9 is the one to use, and fixes all the problems people have with LLMs.

jmartrican 2mo ago

I think the name of the game is going to be "how do we maximize output while keeping token counts low". For example, finding ways to replace AI workflows, even if minor ones, with scripts/code. Anthropic told me that I have till August 19th to be get my act together becuase they are going to reduce my token count by 50%. lol. I have been abusing my Pro Max allowance and need to start being less wasteful. Articles like this, can help us come up with ideas on how to do it.

nycticorax 2mo ago

This is interesting, but I think it would be even more interesting to see a comparison of the token cost of adding a new feature, in the original codebase vs the refactored one. You'd think/hope that the cost of adding the new feature would be lower when starting from the refactored codebase, indicating that refactoring makes economic sense in the long run by lowering the cost of adding new features.

gga 2mo ago
That's what I've shown in this article. The token consumption shown is for adding a new feature (the same feature each time -- in a sub-agent) after every refactoring is applied. I didn't actually capture the token cost of the refactoring. That was a miss, and I'm making sure I do that in the future.
- nycticorax 2mo ago
  Kind of you to engage! Ahh, ok, I misunderstood. Very cool! I think maybe it would have helped me if you had given a one-or-two-sentence description of the "representative change" in the main text, to make it a bit more concrete up-front? But possibly just a 'me' issue...

sharpvik 2mo ago

[flagged]

neet_dev 2mo ago

[flagged]

okzgn 2mo ago

Great point. Using good software architecture and clean code will deliver direct and measurable economic value to projects and reduce AI costs right from the start, whether during refactoring or when planning a logical reorganization.

solarized 2mo ago

Putting this in agent.md didnt help? "always give ultimate decent code in term of maintanibility, cleaness, and perf".

Ozzie-D 2mo ago

[flagged]

wolttam 2mo ago

> and used tiktoken to approximate tokens, by dividing character count by four. This is just the type of thing that stands out. You used a tokenizer library to determine the number of tokens in your text by dividing its length by 4. That makes no sense!

rocky_raccoon 2mo ago

I really enjoy refactoring. Like, doing it by hand (5 miles uphill in the snow both ways) rather than AI. I don't even understand why I enjoy it, because if done right, there's no visible change. When people ask me why I'm so pumped up after having worked on the codebase all day long, I can't give them any sort of answer that makes sense (coming from a small business with a smaller dev team). It's just... "I'm future-proofing our website and we won't see any direct results today but things will be so much easier going forward..." There's something about the puzzle. Looking at my old, deranged coding workarounds that tried to solve problems that have already been solved a thousand times before with established paradigms; and then moving them toward said best practices; and doing it in a way that no NEW technical debt is created. It's just satisfying. I think one of the best learning experiences for me has been the fact that I created a bunch of sloppy shit by hand, auth and all, which forced me to learn things the hard way. All along the way, people were shouting from the rooftops: "use established libraries, dummy!", which is the same advice I would give to somebody today. But by doing things the hard way, I learned so much more about the inner workings. And, I've also given myself a decade's worth of refactoring work, which I really enjoy!

nevster 2mo ago
Probably it's just the same dopamine hits that make things like watching the Windows 98 Defragger soothing.
HappMacDonald 2mo ago
I'm curious how heavily you have test suites built up as guardrails while doing this, to prevent regressions and such?
- mattmanser 2mo ago
  I have often done huge refactors over the last 20 years on systems where there are none. It's very satisfying turning a 1,000 line class file into 100 lines. But all my refactors have been in statically typed languages. To speculate on the reason why, which the GP asked, I think perhaps it is two things. It's the same satisfaction from tidying a room or your workbench. But it's also slightly narcissistic unfortunately. I think you've shown you're better than the other programmers. Much, much, better when you end up with massive reductions in TLOC. And with tests, the irony is that the resulting code is often so much easier to reason about, you start spotting really obvious bad assumptions in the original implementation. It's one of the reasons I've always been fairly skeptical about the true value of unit tests (integration, I get). That and the fact that projects I've worked on that did have unit tests catch like 1 bug a year. Maybe it's just the size of systems I traditionally work on (smaller teams, or even 1 person teams, so man-years worth of effort rather than decades or centuries).

hiAndrewQuinn 2mo ago

>The goal of refactoring an agentic code base is to spend tokens now in refactoring to make token consumption for future work lower. Couldn't have said it better myself. Deciding to refactor should be like a discounted cash flow analysis but for tokens!

4b11b4 2mo ago

I've been experimenting with not only refactoring but rewriting history. I'm always moving around commits, inserting, splitting things up. The amount of times I've instructed to "tell the story right the first time"... More quantitative and qualitative analysis to come

benrutter 2mo ago

Tangent, but I have a theory that refactoring is one of the best symptoms of a healthy dev team. It's partly that refactors themselves have benefits, but I think more that the benefits to refactoring aren't visible to something like product-owners, feature tickets, etc. If teams are refactoring to ensure the health of the overall software, it's a tell-tale sign that developers are happy making recommendations for good software, and that those recommendations are being taken seriously. I think Martin Folwer might have actually coined the term "software rot" - either way, as an issue it happens most severely when a team either aren't motivated or empowered to build their vision of high quality software. When a team can follow their judgement of excellence, that's usually a great sign! (and yes, obviously this can go to far, there are probably some teams who rewrote all their stuff in Ruby then Node then Rust and now something else to be "agent native", but in the coorporate world, I see a lot less of this than teams who just don't feel like they have permission to improve things)

L-patpat 2mo ago

[flagged]

MS8080 2mo ago

[flagged]

ukoki 2mo ago

I have a blanket, unignorable 'files must be <= 1000 lines' lint for my Rust projects for exactly this reason. Exploring large files is _costly_ for agents, so it's much better to lean into the filesystem hierarchy to a greater extent than you would normally with a team of human developers.

supportm 2mo ago

[flagged]