4 ms·

ImpactGate: A merge gate that scores the structural decay AI adds

34 points by sagenschneider 19d ago

sagenschneider 19d ago
[flagged]
hnacobsxph 19d ago
[dead]
appleappleapple 19d ago
Nice idea. Our new CTO brought in a tool he made for analyzing cyclomatic complexity and it’s been useful since we’re a heavily AI-forward shop. BTW, you can avoid your comments being flagged and killed by writing them yourself! I know it’s tempting to offshore it to AI (especially after you’ve vibe-coded a whole project) but some genuine human communication goes a long way.
- crab_galaxy 19d ago
  I know cyclomatic complexity has been heavily debated for a long time, but I do think it’s valuable. It’s really good at highlighting common annoyances like overly clever code, nested ternaries, dense functions with too many branches… The only thing is that these issues seem like human code problems and IME LLMs don’t really write code like this anymore. It’s almost the opposite in python, actually, where Claude leans on writing lots of 2-3 liner private utils which is a separate kind of complexity and organization problem. I still find it useful specifically for React where it’s frustratingly normalized to write many branches in your JSX though.
  - sagenschneider 19d ago
    The difference to previous CC use, is the the change impact formula looks at the complexity already in the class/file. Typical CC just looks at the function it is change and not the context of the change. The Change Impact formula incorporates that to avoid god class and god method issues. Plus multiplying by number of files punishes for non-cohesive code bases. For me it puts the intuition of high cohesion and low coupling into a measurable metric.
    - crab_galaxy 19d ago
      [dead]
- j_bum 19d ago
  Is it public? Would love to try it!
  - ecshafer 19d ago
    Sonarqube has existed for like 20 years. There's doezens of cyclomatic complexity, linter, and cve scanner tools though.
    - sagenschneider 18d ago
      Yep, not looking to replace cyclomatic complexity. Looking to use it in what I've researched is a novel new way. Instead of focusing solely on the complexity of the thing you change. Include the context around what you are changing also. This allows the Change Impact formula to differentiate between two different files of the same length. One is a complex god class with high CC. The other is an entity module that is simple but just has a lot of methods due to properties. Adding to the god class really should warrant a refactor. Adding to the entity class is just another field to store for the entity. The Change Impact formula can differentiate and point to the god classes.
VladVladikoff 19d ago
The overuse of “gate” in this post title makes me think the entire thing was vibe coded even the marketing.
- sagenschneider 19d ago
  It is called Impact"Gate" https://github.com/marketplace/actions/impact-gate https://github.com/marketplace/actions/impact-gate
- appleappleapple 19d ago
  Personally I’m ok with vibe coded projects - certainly feels like the future of things, and I think the line between vibe coded and “professionally” coded is increasingly blurring - but I completely agree on the marketing/communications piece. Ideally your communication about a project conveys real expertise and ownership, signaling that you really understand the problem you’re trying to solve and the tradeoffs you made in your approach to do so. I am very hesitant to use a project where it feels like the eng couldn’t pass a pop quiz about how it works + why.
  - sagenschneider 19d ago
    How the change impact formula works: https://blog.officefloor.net/2026/08/measuring-blast-radius-of-change.html https://blog.officefloor.net/2026/08/measuring-blast-radius-...
    - appleappleapple 19d ago
      Seems like the posts on this site are largely LLM-generated though? Correct me if I’m wrong.
      - pedrosbmartins 19d ago
        "That is a lot to ask of one number. Here is how it fits into one. Built one decision at a time." it sure sounds like Claude.
owebmaster 19d ago
A sloppy project to find slop in projects. It sure works well
philipwhiuk 19d ago
Interesting idea even for non-AI code.
- sagenschneider 19d ago
  Yes, I've run it against a bunch of open source projects with long histories (before AI) to see if it predicts bugs. Seems file size is still a better predictor. However, for the projects where good coding was strictly adhered to and others that were not, it showed the differences appropriately. So I've found it useful in general for Software erosion.
MichaelNolan 19d ago
Maybe I missed it, but it look like this has just a single metric. Maybe instead of making a new project, you could try to get this metric added to a existing tool like https://dekobon.github.io/big-code-analysis/index.html https://dekobon.github.io/big-code-analysis/index.html which already has dozens of metrics.
- sagenschneider 19d ago
  Yes, I'm doing my own research on AI augmented pipelines https://blog.officefloor.net https://blog.officefloor.net . I actually found most code quality tools look for bugs and complexity, but nothing much about cohesive erosion. The nice thing about this metric, is that it determine the files where the erosion is occurring. I turned it into a GitHub action to make it easier to access to get wider feedback on the metric. The GitHub action triggers on your merge request and tells you the files where erosion is occurring to refactor. This stops erosion before it gets too expensive to change (big refactors or rewrite). Yes, happy to work with others to get the metric into other tools.
  - catlifeonmars 19d ago
    What exactly is “cohesive erosion”?
    - sagenschneider 19d ago
      Comes from the basic Computer Science principals of High Cohesion and Low Coupling. High cohesion means the functionality of a component are closely related and focused on performing a single well defined task. Basically single classes for single purposes. Erosion of this is when classes start doing to many things, in the case of God classes. The Change Impact formula looks at a way of detecting when the cohesion is eroding and flagging it on a change (as the pull/merge request itself should generally be single focus cohesive change)
  - apercu 19d ago
    I've never encountered that term before (cohesive erosion) but I like it, if I'm interpreting it correctly. Do you mean like the hyper focus an LLM puts on the task in front of it so you end up with drift (duplicated concepts/multiple ways of doing things, terminology drift (e.g., now we have "customer" and "client"). That sort of thing?
samayashar 19d ago
Great way of detecting AI slop. Claude is pretty good at adding focused changes and if a fix is already present, then it correctly points it out rather than adding unnecessary refactors.
- sagenschneider 19d ago
  I'm not looking at building prototypes. Looking at ways to manage code bases after they've gone through hundreds if not thousands of changes.
EliasWatson 19d ago
What languages does this support? I'm guessing it's Python only, but it would be nice if that was mentioned somewhere.
- sagenschneider 19d ago
  It uses lizard for parsing, so Python, Java, JavaScript, TypeScript, C, C#, Go, Scala, and more.
Retr0id 19d ago
What's the structural decay score of this HN title? Apparently not high enough to be gated.
WD-42 19d ago
Kind of ironic that this thing purports to “gate” slop, yet the entire project is slop, including the README and even the authors comments here. I hate this timeline.
Lerc 19d ago
What score do we get for a group of fixes to single line bugs that are obvious once spotted? There is an implication that all AI changes add complexity and reduce quality, but it is obvious that the quality and complexity is a property of the code not the writer, so all equivalent changes should be equal no matter what the shource. How do clear improvements to make something simpler yet more functional score under this system?
- sagenschneider 19d ago
  Yes, the formula does make some approximations. The problem with cohesion is understanding what is "single purpose". Nothing can determine this without interpreting the code and making a judgement call on whether it needs to be refactored into two or more classes. So I'm looking to approximate this. Instead of looking for cohesion, Change Impact formula approximates this by looking at existing complexity in the file. If the change adds more complex functions to an already complex class, it scores high. Smell of it doing too many things. If the change adds a complex function to an empty file, it's likely just a complex single problem. If the change adds simple change to complex file, then if this happens too many times, we flag it for concern. Especially if it changes a lot of other files too. So it's not mean to measure your code exactly. It's meant to highlight smell that needs investigation. And the formula points out the classes that need looking at. Plus I'm not sure that all changes are equivalent. I'm not looking at single one off changes. I'm looking at 60 changes in a row. The first 20 might go through fine. However, after the changes pile on, there needs to be some refactoring to keep the code clean. This is trying to catch it before it becomes a mess.
  - eithed 19d ago
    > If the change adds a complex function to an empty file, it's likely just a complex single problem. How do you determine that this is correct = adding complexity to a single file vs adding less complexity to multiple files that this one file then orchestrates.
    - sagenschneider 19d ago
altcognito 19d ago
Isn't this just static code analysis? We have tools for this.
- sagenschneider 19d ago
  Most static code analysis looks for bugs or just too complex functions. The novelty of Change Impact is it looks at the context of the complexity. Adding complexity to something already complex should be flagged for review. I've not come across any static code analysis so far that has provided this. And in my own research of architectures, I came to this formula.
p32929 19d ago
[flagged]
TimByte 18d ago
[flagged]

sagenschneider 19d ago

[flagged]

hnacobsxph 19d ago

[dead]

appleappleapple 19d ago

Nice idea. Our new CTO brought in a tool he made for analyzing cyclomatic complexity and it’s been useful since we’re a heavily AI-forward shop. BTW, you can avoid your comments being flagged and killed by writing them yourself! I know it’s tempting to offshore it to AI (especially after you’ve vibe-coded a whole project) but some genuine human communication goes a long way.

crab_galaxy 19d ago
I know cyclomatic complexity has been heavily debated for a long time, but I do think it’s valuable. It’s really good at highlighting common annoyances like overly clever code, nested ternaries, dense functions with too many branches… The only thing is that these issues seem like human code problems and IME LLMs don’t really write code like this anymore. It’s almost the opposite in python, actually, where Claude leans on writing lots of 2-3 liner private utils which is a separate kind of complexity and organization problem. I still find it useful specifically for React where it’s frustratingly normalized to write many branches in your JSX though.
- sagenschneider 19d ago
  The difference to previous CC use, is the the change impact formula looks at the complexity already in the class/file. Typical CC just looks at the function it is change and not the context of the change. The Change Impact formula incorporates that to avoid god class and god method issues. Plus multiplying by number of files punishes for non-cohesive code bases. For me it puts the intuition of high cohesion and low coupling into a measurable metric.
  - crab_galaxy 19d ago
    [dead]
j_bum 19d ago
Is it public? Would love to try it!
- ecshafer 19d ago
  Sonarqube has existed for like 20 years. There's doezens of cyclomatic complexity, linter, and cve scanner tools though.
  - sagenschneider 18d ago
    Yep, not looking to replace cyclomatic complexity. Looking to use it in what I've researched is a novel new way. Instead of focusing solely on the complexity of the thing you change. Include the context around what you are changing also. This allows the Change Impact formula to differentiate between two different files of the same length. One is a complex god class with high CC. The other is an entity module that is simple but just has a lot of methods due to properties. Adding to the god class really should warrant a refactor. Adding to the entity class is just another field to store for the entity. The Change Impact formula can differentiate and point to the god classes.

VladVladikoff 19d ago

The overuse of “gate” in this post title makes me think the entire thing was vibe coded even the marketing.

sagenschneider 19d ago
It is called Impact"Gate" https://github.com/marketplace/actions/impact-gate https://github.com/marketplace/actions/impact-gate
appleappleapple 19d ago
Personally I’m ok with vibe coded projects - certainly feels like the future of things, and I think the line between vibe coded and “professionally” coded is increasingly blurring - but I completely agree on the marketing/communications piece. Ideally your communication about a project conveys real expertise and ownership, signaling that you really understand the problem you’re trying to solve and the tradeoffs you made in your approach to do so. I am very hesitant to use a project where it feels like the eng couldn’t pass a pop quiz about how it works + why.
- sagenschneider 19d ago
  How the change impact formula works: https://blog.officefloor.net/2026/08/measuring-blast-radius-of-change.html https://blog.officefloor.net/2026/08/measuring-blast-radius-...
  - appleappleapple 19d ago
    Seems like the posts on this site are largely LLM-generated though? Correct me if I’m wrong.
    - pedrosbmartins 19d ago
      "That is a lot to ask of one number. Here is how it fits into one. Built one decision at a time." it sure sounds like Claude.

owebmaster 19d ago

A sloppy project to find slop in projects. It sure works well

philipwhiuk 19d ago

Interesting idea even for non-AI code.

sagenschneider 19d ago
Yes, I've run it against a bunch of open source projects with long histories (before AI) to see if it predicts bugs. Seems file size is still a better predictor. However, for the projects where good coding was strictly adhered to and others that were not, it showed the differences appropriately. So I've found it useful in general for Software erosion.

MichaelNolan 19d ago

Maybe I missed it, but it look like this has just a single metric. Maybe instead of making a new project, you could try to get this metric added to a existing tool like https://dekobon.github.io/big-code-analysis/index.html https://dekobon.github.io/big-code-analysis/index.html which already has dozens of metrics.

sagenschneider 19d ago
Yes, I'm doing my own research on AI augmented pipelines https://blog.officefloor.net https://blog.officefloor.net . I actually found most code quality tools look for bugs and complexity, but nothing much about cohesive erosion. The nice thing about this metric, is that it determine the files where the erosion is occurring. I turned it into a GitHub action to make it easier to access to get wider feedback on the metric. The GitHub action triggers on your merge request and tells you the files where erosion is occurring to refactor. This stops erosion before it gets too expensive to change (big refactors or rewrite). Yes, happy to work with others to get the metric into other tools.
- catlifeonmars 19d ago
  What exactly is “cohesive erosion”?
  - sagenschneider 19d ago
    Comes from the basic Computer Science principals of High Cohesion and Low Coupling. High cohesion means the functionality of a component are closely related and focused on performing a single well defined task. Basically single classes for single purposes. Erosion of this is when classes start doing to many things, in the case of God classes. The Change Impact formula looks at a way of detecting when the cohesion is eroding and flagging it on a change (as the pull/merge request itself should generally be single focus cohesive change)
- apercu 19d ago
  I've never encountered that term before (cohesive erosion) but I like it, if I'm interpreting it correctly. Do you mean like the hyper focus an LLM puts on the task in front of it so you end up with drift (duplicated concepts/multiple ways of doing things, terminology drift (e.g., now we have "customer" and "client"). That sort of thing?

samayashar 19d ago

Great way of detecting AI slop. Claude is pretty good at adding focused changes and if a fix is already present, then it correctly points it out rather than adding unnecessary refactors.

sagenschneider 19d ago
I'm not looking at building prototypes. Looking at ways to manage code bases after they've gone through hundreds if not thousands of changes.

EliasWatson 19d ago

What languages does this support? I'm guessing it's Python only, but it would be nice if that was mentioned somewhere.

sagenschneider 19d ago
It uses lizard for parsing, so Python, Java, JavaScript, TypeScript, C, C#, Go, Scala, and more.

Retr0id 19d ago

What's the structural decay score of this HN title? Apparently not high enough to be gated.

WD-42 19d ago

Kind of ironic that this thing purports to “gate” slop, yet the entire project is slop, including the README and even the authors comments here. I hate this timeline.

Lerc 19d ago

What score do we get for a group of fixes to single line bugs that are obvious once spotted? There is an implication that all AI changes add complexity and reduce quality, but it is obvious that the quality and complexity is a property of the code not the writer, so all equivalent changes should be equal no matter what the shource. How do clear improvements to make something simpler yet more functional score under this system?

sagenschneider 19d ago
Yes, the formula does make some approximations. The problem with cohesion is understanding what is "single purpose". Nothing can determine this without interpreting the code and making a judgement call on whether it needs to be refactored into two or more classes. So I'm looking to approximate this. Instead of looking for cohesion, Change Impact formula approximates this by looking at existing complexity in the file. If the change adds more complex functions to an already complex class, it scores high. Smell of it doing too many things. If the change adds a complex function to an empty file, it's likely just a complex single problem. If the change adds simple change to complex file, then if this happens too many times, we flag it for concern. Especially if it changes a lot of other files too. So it's not mean to measure your code exactly. It's meant to highlight smell that needs investigation. And the formula points out the classes that need looking at. Plus I'm not sure that all changes are equivalent. I'm not looking at single one off changes. I'm looking at 60 changes in a row. The first 20 might go through fine. However, after the changes pile on, there needs to be some refactoring to keep the code clean. This is trying to catch it before it becomes a mess.
- eithed 19d ago
  > If the change adds a complex function to an empty file, it's likely just a complex single problem. How do you determine that this is correct = adding complexity to a single file vs adding less complexity to multiple files that this one file then orchestrates.
  - sagenschneider 19d ago

altcognito 19d ago

Isn't this just static code analysis? We have tools for this.

sagenschneider 19d ago
Most static code analysis looks for bugs or just too complex functions. The novelty of Change Impact is it looks at the context of the complexity. Adding complexity to something already complex should be flagged for review. I've not come across any static code analysis so far that has provided this. And in my own research of architectures, I came to this formula.

p32929 19d ago

[flagged]

TimByte 18d ago

[flagged]