14 ms·

System design and the cost of architectural complexity (2013)

464 points by damethos 4y ago

deleted 4y ago
[deleted]
neals 4y ago
I've gotta say, this study really hits home for me. As a developer who's worked on a few projects with some gnarly architectural complexity, I can totally see how that would lead to these kinds of costs. Just last year, I was part of a team that had to deal with a super convoluted codebase that felt like a patchwork of different styles and approaches, with no clear hierarchy or modularity. It was an absolute nightmare to navigate and make changes to, and our productivity took a nosedive. Not to mention the bugs that kept creeping in and the insane amount of time we spent debugging. I even saw a few of my colleagues jump ship because of the frustration. I wish management would've realized the potential benefits of investing in some proper refactoring efforts to improve the architecture. It might've saved them a lot of money (and headaches) in the long run!
- tunesmith 4y ago
  Two things that get in the way of "long run" thinking. What's the timeframe that teams should adopt? If it will pay for itself in three years, is that too long? What's a good argument to make here that will appeal to the bean counters that are used to thinking in terms of quarterlies? I personally am comfortable with "eventually/infinite" but that is a tough sell. Capitalization vs Expense. Maintenance/refactoring is not capitalizable, and thus gets discouraged in businesses that care about P&L. New (capitalizable) projects/features are encouraged instead. What are some good ways to encourage maintenance/refactoring in this kind of environment?
- treis 4y ago
  Stuff like this contributes massively to a culture of low expectations too. When it becomes accepted that simple things take a long time to do the normal reward system goes flying out the window. Both internally and organizationally. Internally because the sense of accomplishment from seeing the new bit of functionality is tiny compared to the effort it took to get done. Organizationally it becomes harder to reward productivity because with no sense of how long something should take there's no way of knowing if an engineer is fast or not. It's a productivity death spiral. Engineers slack off because there is no internal or external reward for doing good work. That slacking off slows development and then that velocity gets accepted as normal. Engineers continue to slack off against the new "normal" and establish an even slower normal to slack off against. That's how hours long tasks turn to days and then weeks and then months until the project dies.
kerneltime 4y ago
MIT has an amazing program for System Design and Management. Dan and I are both graduates of this program. Some of the courses I would recommend include System Architecture, System Safety, and System Dynamics. Most of the content is available on OCW. What is taught is not software-specific, but is entirely applicable to software, outside of the world of 'throw everything on the wall and see what sticks' as long as the venture capitalist can be shown growth at all costs. I wish more software developers were mindful of complexity and architecture.
- agomez314 4y ago
  Could you elaborate a bit more on what you got out of the program? I never heard about it before but i'm intrigued. Did you find a particular course memorable?
  - kerneltime 4y ago
    The tag line within SDM was that it is a program for those who want to lead engineering and not leave engineering (MBA) I think the meta framework for thinking and being able to step away from the madness of releasing a v1 product and having tools for thinking about the bigger picture. Also, MIT.. it is a very rewarding ecosystem to be in.
- waitingforset 4y ago
  Sounds super interesting. Can you link to the specific OCW courses? I see a few from different departments.
  - kerneltime 4y ago
    ESD.34 ESD.342 16.863J 15.871 15.783J ESD.33 to name a few Also, 15.965 my favorite and offered by my thesis advisor Michael A. M. Davies: Based on which it is likely that OpenAI won't be walking away with the cake.
    - uptownfunk 4y ago
discreteevent 4y ago
It's a balancing act: "There is no theoretical reason that anything is hard to change about software. If you pick any one aspect of software then you can make it easy to change, but we don’t know how to make everything easy to change. Making something easy to change makes the overall system a little more complex, and making everything easy to change makes the entire system very complex. Complexity is what makes software hard to change." https://martinfowler.com/ieeeSoftware/whoNeedsArchitect.pdf https://martinfowler.com/ieeeSoftware/whoNeedsArchitect.pdf
- eternalban 4y ago
  Worth pointing out that this study does -not- equate "architectural complexity" with abstraction. Many consider use of "hierarchies, modules, abstraction layers" to be 'unnecessary complexity' where as the thesis clearly states they "play an important role in controlling complexity." OP is not a call to get rid of software architects, arguably it says ~'hire competent architects as bad architecture negatively impacts faults, staff turnover, and product success.' "Architecture controls complexity", and under-/poorly- designed architecture while superficially "simple" will give rise to un-intended complexity. Microservices are the du jour poster child here.
  - tunesmith 4y ago
    People regularly forget Rule Zero of patterns: Don't implement a pattern if it doesn't solve a problem. That's the difference between unnecessary complexity and controlling complexity.
    - q7xvh97o2pDhNrh 4y ago
      IMHO people aren't forgetting it — it's pretty commonly the other way around. Anecdotally, the people I see advocating for spaghetti architecture are often utterly convinced they're solving some critical problem. Conversations with stakeholders then turn into fearmongering — often helped by the fact that most stakeholders don't care about architectural nuance — and it becomes easy to wave off any competing simpler architectures by deriding them as "taking on tech debt." In general, it's surprisingly easy to play corporate politics to bring a convoluted architecture into reality.
iancmceachern 4y ago
"we found that differences in architectural complexity could account for 50% drops in productivity, three-fold increases in defect density, and order-of-magnitude increases in staff turnover." I think I can speak for many of us technical professionals when I say, been there, done that.
- pphysch 4y ago
  Yep, working with well-architected software systems is the difference between a low-stress well-paying job and rapid burnout. No surprise that the latter results in massive turnover. It's remarkable how "add feature X" can be a 1 hour task or a 1 month task depending on whether the system was designed to evolve and scale in that direction. But the non-technical management or customer just sees it as a simple request, and the (Nth) software developer is left to pick up the pieces.
- jkubicek 4y ago
  This is exactly why I think the "myth of the 10x engineer" is so obviously false. An average software engineer will create an overly complex system. If a skilled engineer can come in and create something that doubles team productivy, decreases bugs by 60% and improves retention across the org by 10x, that's huge! That's not a 10x engineer, that's a 100x engineer.
  - nerpderp82 4y ago
    Engineers that make simple systems are not rewarded is most organizations. In a ten person company, sure thing! In a large corporate org, no way.
    - dustymcp 4y ago
      Man the dead weight around development in corporations are mind numbing..
      - nerpderp82 4y ago
        The institutionalized brain damage is real, I see bright new hires coming in, within 3 months they are towing all the lines, spouting tautologies, seeking alignment with corporate velocity vectors to gain momentum, generate impulse. Talking in non-specific platitudes that could literally mean anything and thus mean nothing. The biggest problem with a simple architecture and often why someone would complexify it up, is as a protection mechanism so you don't get some performance review motivated yokel doing a drive by to add in a feature and look like a rock star because it was so damn easy. NSS, that was the next step. You see this in open-source as well. Someone builds a beautiful foundation, and Steve rolls up with a submarine pull request to slam that cherry on top. A complex architecture ensures that you can adequately gate keep. The next aspect is you have to relentlessly say no to feature requests that literally destroy the system to add some new feature, the pressure to smash it to bits and make it a no longer simple system is such a perverse incentive that unless you have support from on high, every simple system will decay into a total mess, then the finger-painters will have moved on to some other system to predate on.
alphabetatheta 4y ago
The hardest part here is the tradeoff between architectural complexity as you build systems and speed of shipping product. Earlier stage companies will ship ship ship and ignore good architecture practice. At some point, it will come back to bite you if your company lives to see another day.
- teacpde 4y ago
  I would say it goes beyond early stage companies and extends to later stage product-driven companies, especially those that value time-to-market than anything else.
- duxup 4y ago
  I would argue it's the later stage company who doesn't take the time to fix it / pay off that tech debt who fails. I'm not against picking up some tech debt here or there if you pay it off.
- manv1 4y ago
  This is why you need good people when you start: because good architecture is the difference between fast iteration and slow iteration. A good architecture will allow you to make changes easily. A bad one doesn't. It's actually pretty simple, conceptually speaking. If you believe that "late stage" companies make correct architecture choices you're probably incorrect. It's not about late stage or early stage, it's about knowing how to build software from scratch in a way that you don't hamstring yourself (and others) down the road.
- jasondigitized 4y ago
  Always a tradeoff. You can build a Ferrari but may end up caught in the garage while a competitor has paying customers doing laps around the feature track collecting $200 at every turn.
duxup 4y ago
I'm always thinking about "Can I (or anyone) get back into this easily 6 months from now?" In my situation, I probably will have to do that so there's a selfish reason there for sure. I recently had a whole series of frustrating situations where I dug through rediscovering how old code / systems work to make small changes or to find out the small change was enormous. Really deflating stuff. It's not my fault but it can be so demoralizing. Feels like a weight on you... I was done for the day after both of those horror shows. Then yesterday I had a 3 day project start and in 2 hours I ... did the thing. It was super flexible / powerful, handled errors gracefully, and easy to change / test. All because a year ago someone (well myself and anther person) took the time to simplify the original spaghetti code that originally existed and break it into more digestible functional-esque chunks. Dropping something "in between the chunks" (fancy technical terms here) was easy to do, test and read. Completely the opposite experience, it was energizing and fun.
- latchkey 4y ago
  > I'm always thinking about "Can I (or anyone) get back into this easily 6 months from now?" As I age, my memory is getting worse and worse and I realize that quite clearly. Therefore, I always try to write documentation as I'm writing code, so that I can remember why I did something. It helps a lot so that 6 months later, I can do exactly that... but I also know that anyone else looking at my stuff will also realize why things are the way they are.
  - duxup 4y ago
    I’m the same way, notes, good documentation, etc. Sometimes I think I get some tasks done faster than when I was younger…
- bob1029 4y ago
  > All because a year ago someone (well myself and anther person) took the time I've been saying for half a decade or longer: "Going slower today means we can go faster tomorrow". It took a long time for some of my team members to process this, but I believe they've all taken it to heart by now. The aggressive, rapid nature of a startup can make it very difficult to slow down enough to consider more boring, pedantic paths. Thinking carefully about this stuff can really suck today, but when its 3am on Saturday and production is down, it will all begin to make a lot of sense. Having empathy for your future self/team is a superpower.
germamme 4y ago
Software needs regular refactoring just like cars need regular maintenance. It is very difficult to determine how much time and effort to spend working on making it extendible when you're writing it the first time (even the product owner might not know how it will be used initially) but after a few months in production and a few feature requests, you'll get a better idea of what the pain points are and will be in a much better position to refactor. The problem is convincing the people who cut the check to allow engineers the time to do it
nmstoker 4y ago
Should have (2013) in the title.
- dang 4y ago
  Added. Thanks!
unixhero 4y ago
I cannot believe this is free. This is going straight to my read soon queue
holoduke 4y ago
I still dont know what is better. Create quick throw away code you replace in a few years. Or build a well structured system that lasts longer. I have worked all my carreer in only the first 0-8 years of company startups. Most of the joy and success I got from 'quick hacking together working systems'. A lot of people arround me dont share that opinion and are better suited at structured companies. Maybe there is no better.
- bob1029 4y ago
  > Or build a well structured system that lasts longer. What if you had the ability to build a system that was initially structured well enough such that it could be made to last indefinitely? From a cost/benefit standpoint, is the Ship of Theseus approach not the most ideal for a business owner? Even for a developer, the notion that you have to constantly "trojan horse" your new product iterations into production via the existing paths means you will achieve mastery over clever workarounds and other temporary shims. Once you gain competence in this area, it is possible that you will never want for a new boat again. You start to get attached to the old one and all of the things it takes care of that you absolutely forgot about years ago.
  - WJW 4y ago
    > if you had the ability to build a system that was initially structured well enough such that it could be made to last indefinitely If you could have that, it would obviously be amazing. Do you (or anyone) have that ability though? So far it seems the answer is no. People are just not smart enough to predict the future. As a more nuanced answer, a system that is scalable enough to grow to 1000x the current load is usually way too expensive to build with the resources you have "right now". The best you can do is build a system for 10-100x the current load and hope you haven't forgotten any edge cases, but usually you encounter them way sooner than 10x the load. Building so that you can easily refactor your current system is the way to go, but even then you will sooner or later run into problems your original design did not consider.
    - bob1029 4y ago
r3trohack3r 4y ago
Having worked in and out of FAANG and with ex-FAANG in startups, FAANG engineers have a tendency to treat everything like a FAANG problem that demands FAANG solutions. You'll see these massive systems with event driven architectures and layers of nested microservices deployed to hand rolled K8S clusters with custom plugins and Cloud Native CI/CD systems strapped over the top of a complex repository setup and a complex metrics/tracing/logging stack to make sense of all the inter-dependencies and lifecycles of a request in this system. All of this for a system that is taking <100 RPS and likely won't see more than that for many years. I'm not exaggerating when I say you could host the entire company on a single R620 (+ a cold backup) with PID1 managing a couple of bash scripts and a few Python/Go/Java/JavaScript processes wired up to an SQLite db and a nightly backup. But their architecture choice does create jobs in our industry. These services need teams to maintain them. You have a K8S team, a DevTools team, a CI/CD team, teams for the API services, teams for the backend microservices, SREs to respond to incidents, etc. Those teams all need to be managed, coordinated, staffed, paid, etc. so you hire on the management and administrators to handle the size of the engineering organization responsible for this thing. So instead of two 4-5 figure servers and a few $100 a month in hosting, plus a handful of well compensated engineers who keep this thing running, your engineering department is burning 6-7 figures per month in cloud costs and millions in headcount. To be clear, some companies need these setups. I do platform engineering consulting and help companies build these exact org structures and systems I'm talking about here. But I only take those contracts when the work is justified. There is a spectrum, and very few companies fall on the end of the spectrum that demands these solutions. Today, a majority of the companies in the American market should be on serverless offerings that scale to near-zero - simply to reduce the need to staff an engineering org to maintain the "infrastructure" under the system. You buy that engineering org from the serverless vendor with a support contract. I'm not setting a mom-and-pop shop up with an R620 because they'll be 100% dependent on a high-skill engineer for the rest of their existence, which isn't likely to yeild an ROI for them. Past that you start migrating to VMs or bare metal. That'll get you pretty far. Modern computers are Very Fast, you can serve a lot of traffic from a single 1U slot. Very few companies get to the scale where any of these FAANG architectures start to make sense. If I were to take a guess, I think a lot of the FAANG stuff is salary chasing. To justify a $250k - $500k salary folks think they need to build these fancy architectures to demonstrate their value, and the junior engineers and management are all in because they're chasing the salary ladder and need to get this experience to get into FAANG. But the reality is, with back of napkin math, you can save a company _millions_ per year with conservative architectures. For me at least, negotiating contract rates against that pool of savings is an unsolved problem.
pixel_tracing 4y ago
Looks like we need a better system architecture for the MIT website hosting the material as it’s overloaded with too many requests
austin-cheney 4y ago
Complex means many, not challenging - https://www.dictionary.com/browse/complex https://www.dictionary.com/browse/complex The most practical measure of complexity is duplication. Are there two, or more, pieces of code accomplishing the same or a similar job? That is complex. The solution is to refactor the many parts into fewer parts.
- tjr 4y ago
  composed of many interconnected parts; compound; composite: a complex highway system. characterized by a very complicated or involved arrangement of parts, units, etc.: complex machinery. I would say you could have a software system with many interconnected parts, and/or with a complicated or involved arrangement of parts, without having duplicate code or duplicated functionality.
  - austin-cheney 4y ago
    Duplication is just one of many potential measures. The end goal though is converting many to few.
- ryanklee 4y ago
  No one uses the word complex to mean many. Further, sometimes 1 thing is overloaded in difficult-to-understand ways, and so there should be more things, not fewer. Sometimes there are many things that should be 1 thing. There isn't just one good measurement of complexity, as complexity isn't inherent to things or systems in themselvesas, rather complexity is a feature of perception, which gets confused in all sorts of irreducible ways.
  - austin-cheney 4y ago
    This popular language author invalidates your comment: https://www.infoq.com/presentations/Simple-Made-Easy/ https://www.infoq.com/presentations/Simple-Made-Easy/
    - ryanklee 4y ago
opportune 4y ago
In my experience this problem arises from a lack of consideration of two related things, and not doing a third thing: 1. Not considering how inevitable development issues and bugs will affect the entire system. That is, if a bug gets introduced here, how bad will the systemic effects be, and how do we prevent it? A severe case is if you start writing bad data from one component that then needs to be manually backfilled or reverted (especially if this then generates yet more bad data further down the line). 2. Not considering how entire-system failures will present themselves, and not having a good way to diagnose them. 3. Failure to develop testing (integration or whole-system blackbox) that catches the first case, failure to develop tooling (tracing, logging, synchronizing changes across components) that assists in the second case. Or instead adjusting the system so that the first or second cases are considered. It’s easy to get stuck in a local optimum where a few old hats who understand the entire system are the only ones capable of predicting and diagnosing failures across components. It’s also easy to say that less skilled or new engineers just need to put in the time to get good, but it’s often the case that the old hats have a potentially-automatible procedure for narrowing down the problem or tracking where the issue came from, or that the benefits of separate components don’t justify the increased rate of bugs and time spent tracking them down to fix them.
avi_vallarapu 4y ago
I believe that the great examples always arise from OpenSource projects. The design and modular code always play a great role in increasing the ability to customize or add features and also have an increase in collaboration from more volunteers. Developer life gets more interesting with a massively improved code quality when some tough decisions are taken much earlier.
- yCombLinks 4y ago
  There are great examples in open source, but my main issue is open source projects are usually frameworks. Frameworks solve a lower level problem than end user application code. I often see excess abstraction and unneeded flexibility in business code that just makes the problem domain harder to understand, without ever providing value.
jasmer 4y ago
This is the economic value of refactoring, right on paper. 'Complexity is the mind-killer' - Linus Atreides
uptownfunk 4y ago
A wise engineer who I worked closely with told me "the job of the engineer is to manage complexity. Engineers don't like complexity". There are those hard-learned sutras that just make life so much easier down the line. Down the line always comes (unless you are playing career musical chairs and are willing to gamble that you won't be around to have to deal with the code mess when there's a sev2-, but as they say.. karma's a b..." Avoid premature optimization TDD Avoid complexity Readable code Documented code Probably helps avoid most problems.
beders 4y ago
Architectures need to be judged by their difficulty to make code and data changes. Changes in a stratified architecture are much simpler than in a layered design. (Changes in a layered design almost always mean changes to all the layers)
farrugp 4y ago
Can't wait to read this, really resonates with me as I'm dealing with this right now at a big FAANG company. In my experience the problem is that as with all things it's all about balance. We shouldn't throw away architecture entirely and write stupidly simple, quick solutions because they will be messy. But we also shouldn't over-abstract things so much that only the person/people who built the system can understand it and work with it. Both could have dire consequences for an organization, making building new features and delivering value to users slow, difficult and costly. Once I finish reading this MIT paper, I want to dig further into exactly what makes software 'complex'. In my experience: - too many layers of indirection - overly theoretical concepts not grounded in real world thinking - lack of documentation - bad naming - lack of testability - tight de-coupling - following 'best practices and patterns' without clear justification - trying to solve for problems before they exist We should be building systems that are grounded in concepts that are easily understandable - which is exactly why Object Oriented Programming has been so successful. We write programming languages as a means of communicating with each other about program logic, why not do it in terms that we as users already understand in the real world?
jen729w 4y ago
In the late 90s I travelled the world with a couple of CD-ROMs installing NT 4.0 on individual physical servers. I understood the entire stack. I was the only engineer and I was often very remote. In the mid 2000s we installed Server 2003/8 on a whole bunch of physical servers in a data centre. This was for one of Australia’s ‘big four’ banks. Me and a team of less than ten, most of whom I still know, managed the entire thing. We were the ‘3rd level infrastructure’ team. Most of us understood most of the stack end-to-end, although there were some specialities. In 2023 I work for an IT integrator. These days we use Azure, because Microsoft has fooled us in to thinking that it’s cheaper. My current project is, essentially, a website migration. Not even a big or complex website. The project has been running for 6 months. I only joined recently so I can’t be sure but we’ve probably burned AU$2m. The schedule I’m managing has us doing the cutover in June. That is optimistic. The architects are currently trying to work out how Azure [details] connect to Azure [details] while still [security] and being able to [complex integration]. Every day some new issue appears that the architects have to figure out how to work around. No single person has a goddamned clue how the end-to-end thing fits together. Not one, not a clue. That scares me. ‘Architectural complexity’ has crept down to the level of infrastructure. It’s painful to see. I hope it stops soon but I am not hopeful.
- lifeisstillgood 4y ago
  This is the essence - that a system needs to fit inside one persons head. All of it. There maybe a few people who understand all of a Boeing airplanes systems - there certainly were when they first flew 747s. Maybe a couple now. but once it stops fitting inside one persons head then the thing is literally only possible to design by committee - it cannot fit together and perhaps should stop being called a single system
  - bob1029 4y ago
    > it cannot fit together and perhaps should stop being called a single system The magic is the enterprise message bus. It is meme crap until you actually need one. When it works, it really works. This is the only thing I've ever seen realistically tie together certain industrial environments.
hinkley 4y ago
I've long had an allergy to accidental complexity but recently had some new aspects illuminated for me. I had a block of code that had been added to by others, making it a bit of a recursive mess with a handful of bugs that were hard to reproduce let alone fix. When I finally got tired of it all, I sat about to replacing the recursive code with some iteration, and ended up with DP code, which avoided a bunch of duplicate errors and the need for caching. Without all the DP terminology, it's what other people would call building a (programmatic) plan and then executing it, rather than a depth first search algorithm. I used to use this quite regularly, but have only used it a couple of times on my current gig. Strictly speaking planning an entire action before starting it might result in a bit extra data hanging around because of building the data structures ahead of time, but at the end of the day it allowed me to eliminate a whole lot of steps that seemed necessary, and also remove duplicate warning messages. It wasn't necessarily faster than it could be, but it was way faster than I ever expected it to be. Writing obvious code with clear actors and data flow steps makes it a lot easier to add features, and make performance improvements. Obscure code leads to more obscure code (qualitatively and quantitatively).
- malcolmstill 4y ago
  Can you define your use of DP here? I'm not sure if you mean dynamic programming, or design pattern or something else, and am curious about your insight.
hyperthesis 4y ago
They find architectural complexity accounts for defects (x3), productivity (50%) and staff turnover (order-of-magnitude). The "McCabe cyclomatic complexity metric" doesn't predict the last two. Instead they use the "MacCormack" method (who supervised this PhD...), so I wonder how it differs? funfact: they reference an evaluation of Mozilla'a refactoring (the one Joel said you must never do). It's a PhD thesis, and it's long, detailed and academic (tautology), yet lacks an introduction, so: The Missing Introduction "Architectural complexity" here doesn't mean the complexity of architecture features themselves (e.g. architecture astronauts making too many layers etc), but direct and indirect interactions between parts. Instead of diagrams of arcs and nodes, they believe a matrix representation helps show complexity (vertical axis: using; horizontal axis used - or maybe vice versa - with dots at that location), called a "Design Structure Matrix" (DSM). The key sections seem to be: 3.6-7 [p.34-42] about DSMs 5.1 [p.67-83] about complexity by the "MacCormack approach" [pasted below, from p. 70] [5.1.2.1] Capture a network representation of a software product's source-code using dependency extraction tools [5.1.2.2] Find all the indirect paths between files in the network by computing the graph's transitive closure [5.1.2.3] Assign two visibility scores to each file that represent its reachability from other files or ability to reach other files in the network. [Visibility Fan In, Visibility Fan Out] [5.1.3] Use these two visibility scores to classify each file as one of four types: peripheral, utility, control, or core. They also have "network density" and "propagation cost" measures [p.76] --- Aside: A PhD thesis takes some time to read before commenting. I wish there was a way to facilitate discussion of the submission, in addition to the title-based opinion and experience that we have now. The patient_hackernews experiment (https://old.reddit.com/r/patient_hackernews/ https://old.reddit.com/r/patient_hackernews/) didn't seem to work out. Perhaps just a re-submission (or a bump?) labelled "for readers only" 24 hours later?
berniedurfee 4y ago
This is my jam. It’s really difficult to stop myself from chasing shiny things, but the last year running an app with a dead simple architecture has been very peaceful. Other than some hairy legacy stuff here and there, it all just works and the run rate is exceptionally low. When it occasionally breaks, it’s obvious what’s wrong and quick to fix. We spend most of our time making it better. Long live the 3-tier monolith.
bshanks 4y ago
This thesis provides empirical support that a specific measure of software architectural complexity is costly. Specifically, they look thru source code in an automated manner, construct the graph whose nodes are source code files and whose edges are the following cross-file relationships (page 73, section 5.1.2.1): - The site of function calls to the site of the function's definition - The site of class method calls to the site of that class method's definition - The site of a class method definition to the site of the class definition - The site of a subclass definition to the site of its parent class' definition - The site at which a variable with a complex user-defined type is instantiated or accessed to the site where that type is defined. (User-defined types include structure, union, enum, and class.) Then they compute the transitive closure of this graph. Then they compute two metrics for each node by looking at the transitive closure graph (page 76, section 5.1.2.3): - Visibility Fan In (VFI): how many other nodes have edges that go from the other node to this node? - Visibility Fan Out (VFO): how many other nodes have edges that go from this node to the other node? They observe that by looking at the VFI metric across various files, files tend to sharply cluster into either 'low VFI' or 'high VFI', and similarly for VFO (although some files may be high in one metric and low in the other) (page 79, section 5.1.3). They then classify each file as: - low VFI, low VFO: 'peripheral' - high VFI, low VFO: 'utility' - low VFI, high VFO: 'control' - high VFI, high VFO: 'core' They then find that 'core' files are the most costly, in terms of defect density, developer productivity, and probability of staff turnover.
sktrdie 4y ago
What exactly did decrease the complexity? And if, apparently, we can measure it, shouldn't it be part of any development process, similar to a linter?

464 points by damethos 4y ago

deleted 4y ago

[deleted]

I've gotta say, this study really hits home for me. As a developer who's worked on a few projects with some gnarly architectural complexity, I can totally see how that would lead to these kinds of costs. Just last year, I was part of a team that had to deal with a super convoluted codebase that felt like a patchwork of different styles and approaches, with no clear hierarchy or modularity. It was an absolute nightmare to navigate and make changes to, and our productivity took a nosedive. Not to mention the bugs that kept creeping in and the insane amount of time we spent debugging. I even saw a few of my colleagues jump ship because of the frustration. I wish management would've realized the potential benefits of investing in some proper refactoring efforts to improve the architecture. It might've saved them a lot of money (and headaches) in the long run!

tunesmith 4y ago
Two things that get in the way of "long run" thinking. What's the timeframe that teams should adopt? If it will pay for itself in three years, is that too long? What's a good argument to make here that will appeal to the bean counters that are used to thinking in terms of quarterlies? I personally am comfortable with "eventually/infinite" but that is a tough sell. Capitalization vs Expense. Maintenance/refactoring is not capitalizable, and thus gets discouraged in businesses that care about P&L. New (capitalizable) projects/features are encouraged instead. What are some good ways to encourage maintenance/refactoring in this kind of environment?
treis 4y ago
Stuff like this contributes massively to a culture of low expectations too. When it becomes accepted that simple things take a long time to do the normal reward system goes flying out the window. Both internally and organizationally. Internally because the sense of accomplishment from seeing the new bit of functionality is tiny compared to the effort it took to get done. Organizationally it becomes harder to reward productivity because with no sense of how long something should take there's no way of knowing if an engineer is fast or not. It's a productivity death spiral. Engineers slack off because there is no internal or external reward for doing good work. That slacking off slows development and then that velocity gets accepted as normal. Engineers continue to slack off against the new "normal" and establish an even slower normal to slack off against. That's how hours long tasks turn to days and then weeks and then months until the project dies.

kerneltime 4y ago

MIT has an amazing program for System Design and Management. Dan and I are both graduates of this program. Some of the courses I would recommend include System Architecture, System Safety, and System Dynamics. Most of the content is available on OCW. What is taught is not software-specific, but is entirely applicable to software, outside of the world of 'throw everything on the wall and see what sticks' as long as the venture capitalist can be shown growth at all costs. I wish more software developers were mindful of complexity and architecture.

agomez314 4y ago
Could you elaborate a bit more on what you got out of the program? I never heard about it before but i'm intrigued. Did you find a particular course memorable?
- kerneltime 4y ago
  The tag line within SDM was that it is a program for those who want to lead engineering and not leave engineering (MBA) I think the meta framework for thinking and being able to step away from the madness of releasing a v1 product and having tools for thinking about the bigger picture. Also, MIT.. it is a very rewarding ecosystem to be in.
waitingforset 4y ago
Sounds super interesting. Can you link to the specific OCW courses? I see a few from different departments.
- kerneltime 4y ago
  ESD.34 ESD.342 16.863J 15.871 15.783J ESD.33 to name a few Also, 15.965 my favorite and offered by my thesis advisor Michael A. M. Davies: Based on which it is likely that OpenAI won't be walking away with the cake.
  - uptownfunk 4y ago

discreteevent 4y ago

It's a balancing act: "There is no theoretical reason that anything is hard to change about software. If you pick any one aspect of software then you can make it easy to change, but we don’t know how to make everything easy to change. Making something easy to change makes the overall system a little more complex, and making everything easy to change makes the entire system very complex. Complexity is what makes software hard to change." https://martinfowler.com/ieeeSoftware/whoNeedsArchitect.pdf https://martinfowler.com/ieeeSoftware/whoNeedsArchitect.pdf

eternalban 4y ago
Worth pointing out that this study does -not- equate "architectural complexity" with abstraction. Many consider use of "hierarchies, modules, abstraction layers" to be 'unnecessary complexity' where as the thesis clearly states they "play an important role in controlling complexity." OP is not a call to get rid of software architects, arguably it says ~'hire competent architects as bad architecture negatively impacts faults, staff turnover, and product success.' "Architecture controls complexity", and under-/poorly- designed architecture while superficially "simple" will give rise to un-intended complexity. Microservices are the du jour poster child here.
- tunesmith 4y ago
  People regularly forget Rule Zero of patterns: Don't implement a pattern if it doesn't solve a problem. That's the difference between unnecessary complexity and controlling complexity.
  - q7xvh97o2pDhNrh 4y ago
    IMHO people aren't forgetting it — it's pretty commonly the other way around. Anecdotally, the people I see advocating for spaghetti architecture are often utterly convinced they're solving some critical problem. Conversations with stakeholders then turn into fearmongering — often helped by the fact that most stakeholders don't care about architectural nuance — and it becomes easy to wave off any competing simpler architectures by deriding them as "taking on tech debt." In general, it's surprisingly easy to play corporate politics to bring a convoluted architecture into reality.

iancmceachern 4y ago

"we found that differences in architectural complexity could account for 50% drops in productivity, three-fold increases in defect density, and order-of-magnitude increases in staff turnover." I think I can speak for many of us technical professionals when I say, been there, done that.

pphysch 4y ago
Yep, working with well-architected software systems is the difference between a low-stress well-paying job and rapid burnout. No surprise that the latter results in massive turnover. It's remarkable how "add feature X" can be a 1 hour task or a 1 month task depending on whether the system was designed to evolve and scale in that direction. But the non-technical management or customer just sees it as a simple request, and the (Nth) software developer is left to pick up the pieces.
jkubicek 4y ago
This is exactly why I think the "myth of the 10x engineer" is so obviously false. An average software engineer will create an overly complex system. If a skilled engineer can come in and create something that doubles team productivy, decreases bugs by 60% and improves retention across the org by 10x, that's huge! That's not a 10x engineer, that's a 100x engineer.
- nerpderp82 4y ago
  Engineers that make simple systems are not rewarded is most organizations. In a ten person company, sure thing! In a large corporate org, no way.
  - dustymcp 4y ago
    Man the dead weight around development in corporations are mind numbing..
    - nerpderp82 4y ago
      The institutionalized brain damage is real, I see bright new hires coming in, within 3 months they are towing all the lines, spouting tautologies, seeking alignment with corporate velocity vectors to gain momentum, generate impulse. Talking in non-specific platitudes that could literally mean anything and thus mean nothing. The biggest problem with a simple architecture and often why someone would complexify it up, is as a protection mechanism so you don't get some performance review motivated yokel doing a drive by to add in a feature and look like a rock star because it was so damn easy. NSS, that was the next step. You see this in open-source as well. Someone builds a beautiful foundation, and Steve rolls up with a submarine pull request to slam that cherry on top. A complex architecture ensures that you can adequately gate keep. The next aspect is you have to relentlessly say no to feature requests that literally destroy the system to add some new feature, the pressure to smash it to bits and make it a no longer simple system is such a perverse incentive that unless you have support from on high, every simple system will decay into a total mess, then the finger-painters will have moved on to some other system to predate on.

alphabetatheta 4y ago

The hardest part here is the tradeoff between architectural complexity as you build systems and speed of shipping product. Earlier stage companies will ship ship ship and ignore good architecture practice. At some point, it will come back to bite you if your company lives to see another day.

teacpde 4y ago
I would say it goes beyond early stage companies and extends to later stage product-driven companies, especially those that value time-to-market than anything else.
duxup 4y ago
I would argue it's the later stage company who doesn't take the time to fix it / pay off that tech debt who fails. I'm not against picking up some tech debt here or there if you pay it off.
manv1 4y ago
This is why you need good people when you start: because good architecture is the difference between fast iteration and slow iteration. A good architecture will allow you to make changes easily. A bad one doesn't. It's actually pretty simple, conceptually speaking. If you believe that "late stage" companies make correct architecture choices you're probably incorrect. It's not about late stage or early stage, it's about knowing how to build software from scratch in a way that you don't hamstring yourself (and others) down the road.
jasondigitized 4y ago
Always a tradeoff. You can build a Ferrari but may end up caught in the garage while a competitor has paying customers doing laps around the feature track collecting $200 at every turn.

duxup 4y ago

I'm always thinking about "Can I (or anyone) get back into this easily 6 months from now?" In my situation, I probably will have to do that so there's a selfish reason there for sure. I recently had a whole series of frustrating situations where I dug through rediscovering how old code / systems work to make small changes or to find out the small change was enormous. Really deflating stuff. It's not my fault but it can be so demoralizing. Feels like a weight on you... I was done for the day after both of those horror shows. Then yesterday I had a 3 day project start and in 2 hours I ... did the thing. It was super flexible / powerful, handled errors gracefully, and easy to change / test. All because a year ago someone (well myself and anther person) took the time to simplify the original spaghetti code that originally existed and break it into more digestible functional-esque chunks. Dropping something "in between the chunks" (fancy technical terms here) was easy to do, test and read. Completely the opposite experience, it was energizing and fun.

latchkey 4y ago
> I'm always thinking about "Can I (or anyone) get back into this easily 6 months from now?" As I age, my memory is getting worse and worse and I realize that quite clearly. Therefore, I always try to write documentation as I'm writing code, so that I can remember why I did something. It helps a lot so that 6 months later, I can do exactly that... but I also know that anyone else looking at my stuff will also realize why things are the way they are.
- duxup 4y ago
  I’m the same way, notes, good documentation, etc. Sometimes I think I get some tasks done faster than when I was younger…
bob1029 4y ago
> All because a year ago someone (well myself and anther person) took the time I've been saying for half a decade or longer: "Going slower today means we can go faster tomorrow". It took a long time for some of my team members to process this, but I believe they've all taken it to heart by now. The aggressive, rapid nature of a startup can make it very difficult to slow down enough to consider more boring, pedantic paths. Thinking carefully about this stuff can really suck today, but when its 3am on Saturday and production is down, it will all begin to make a lot of sense. Having empathy for your future self/team is a superpower.

germamme 4y ago

Software needs regular refactoring just like cars need regular maintenance. It is very difficult to determine how much time and effort to spend working on making it extendible when you're writing it the first time (even the product owner might not know how it will be used initially) but after a few months in production and a few feature requests, you'll get a better idea of what the pain points are and will be in a much better position to refactor. The problem is convincing the people who cut the check to allow engineers the time to do it

nmstoker 4y ago

Should have (2013) in the title.

dang 4y ago
Added. Thanks!

unixhero 4y ago

I cannot believe this is free. This is going straight to my read soon queue

holoduke 4y ago

I still dont know what is better. Create quick throw away code you replace in a few years. Or build a well structured system that lasts longer. I have worked all my carreer in only the first 0-8 years of company startups. Most of the joy and success I got from 'quick hacking together working systems'. A lot of people arround me dont share that opinion and are better suited at structured companies. Maybe there is no better.

bob1029 4y ago
> Or build a well structured system that lasts longer. What if you had the ability to build a system that was initially structured well enough such that it could be made to last indefinitely? From a cost/benefit standpoint, is the Ship of Theseus approach not the most ideal for a business owner? Even for a developer, the notion that you have to constantly "trojan horse" your new product iterations into production via the existing paths means you will achieve mastery over clever workarounds and other temporary shims. Once you gain competence in this area, it is possible that you will never want for a new boat again. You start to get attached to the old one and all of the things it takes care of that you absolutely forgot about years ago.
- WJW 4y ago
  > if you had the ability to build a system that was initially structured well enough such that it could be made to last indefinitely If you could have that, it would obviously be amazing. Do you (or anyone) have that ability though? So far it seems the answer is no. People are just not smart enough to predict the future. As a more nuanced answer, a system that is scalable enough to grow to 1000x the current load is usually way too expensive to build with the resources you have "right now". The best you can do is build a system for 10-100x the current load and hope you haven't forgotten any edge cases, but usually you encounter them way sooner than 10x the load. Building so that you can easily refactor your current system is the way to go, but even then you will sooner or later run into problems your original design did not consider.
  - bob1029 4y ago

r3trohack3r 4y ago

Having worked in and out of FAANG and with ex-FAANG in startups, FAANG engineers have a tendency to treat everything like a FAANG problem that demands FAANG solutions. You'll see these massive systems with event driven architectures and layers of nested microservices deployed to hand rolled K8S clusters with custom plugins and Cloud Native CI/CD systems strapped over the top of a complex repository setup and a complex metrics/tracing/logging stack to make sense of all the inter-dependencies and lifecycles of a request in this system. All of this for a system that is taking <100 RPS and likely won't see more than that for many years. I'm not exaggerating when I say you could host the entire company on a single R620 (+ a cold backup) with PID1 managing a couple of bash scripts and a few Python/Go/Java/JavaScript processes wired up to an SQLite db and a nightly backup. But their architecture choice does create jobs in our industry. These services need teams to maintain them. You have a K8S team, a DevTools team, a CI/CD team, teams for the API services, teams for the backend microservices, SREs to respond to incidents, etc. Those teams all need to be managed, coordinated, staffed, paid, etc. so you hire on the management and administrators to handle the size of the engineering organization responsible for this thing. So instead of two 4-5 figure servers and a few $100 a month in hosting, plus a handful of well compensated engineers who keep this thing running, your engineering department is burning 6-7 figures per month in cloud costs and millions in headcount. To be clear, some companies need these setups. I do platform engineering consulting and help companies build these exact org structures and systems I'm talking about here. But I only take those contracts when the work is justified. There is a spectrum, and very few companies fall on the end of the spectrum that demands these solutions. Today, a majority of the companies in the American market should be on serverless offerings that scale to near-zero - simply to reduce the need to staff an engineering org to maintain the "infrastructure" under the system. You buy that engineering org from the serverless vendor with a support contract. I'm not setting a mom-and-pop shop up with an R620 because they'll be 100% dependent on a high-skill engineer for the rest of their existence, which isn't likely to yeild an ROI for them. Past that you start migrating to VMs or bare metal. That'll get you pretty far. Modern computers are Very Fast, you can serve a lot of traffic from a single 1U slot. Very few companies get to the scale where any of these FAANG architectures start to make sense. If I were to take a guess, I think a lot of the FAANG stuff is salary chasing. To justify a $250k - $500k salary folks think they need to build these fancy architectures to demonstrate their value, and the junior engineers and management are all in because they're chasing the salary ladder and need to get this experience to get into FAANG. But the reality is, with back of napkin math, you can save a company _millions_ per year with conservative architectures. For me at least, negotiating contract rates against that pool of savings is an unsolved problem.

pixel_tracing 4y ago

Looks like we need a better system architecture for the MIT website hosting the material as it’s overloaded with too many requests

austin-cheney 4y ago

Complex means many, not challenging - https://www.dictionary.com/browse/complex https://www.dictionary.com/browse/complex The most practical measure of complexity is duplication. Are there two, or more, pieces of code accomplishing the same or a similar job? That is complex. The solution is to refactor the many parts into fewer parts.

tjr 4y ago
composed of many interconnected parts; compound; composite: a complex highway system. characterized by a very complicated or involved arrangement of parts, units, etc.: complex machinery. I would say you could have a software system with many interconnected parts, and/or with a complicated or involved arrangement of parts, without having duplicate code or duplicated functionality.
- austin-cheney 4y ago
  Duplication is just one of many potential measures. The end goal though is converting many to few.
ryanklee 4y ago
No one uses the word complex to mean many. Further, sometimes 1 thing is overloaded in difficult-to-understand ways, and so there should be more things, not fewer. Sometimes there are many things that should be 1 thing. There isn't just one good measurement of complexity, as complexity isn't inherent to things or systems in themselvesas, rather complexity is a feature of perception, which gets confused in all sorts of irreducible ways.
- austin-cheney 4y ago
  This popular language author invalidates your comment: https://www.infoq.com/presentations/Simple-Made-Easy/ https://www.infoq.com/presentations/Simple-Made-Easy/
  - ryanklee 4y ago

opportune 4y ago

In my experience this problem arises from a lack of consideration of two related things, and not doing a third thing: 1. Not considering how inevitable development issues and bugs will affect the entire system. That is, if a bug gets introduced here, how bad will the systemic effects be, and how do we prevent it? A severe case is if you start writing bad data from one component that then needs to be manually backfilled or reverted (especially if this then generates yet more bad data further down the line). 2. Not considering how entire-system failures will present themselves, and not having a good way to diagnose them. 3. Failure to develop testing (integration or whole-system blackbox) that catches the first case, failure to develop tooling (tracing, logging, synchronizing changes across components) that assists in the second case. Or instead adjusting the system so that the first or second cases are considered. It’s easy to get stuck in a local optimum where a few old hats who understand the entire system are the only ones capable of predicting and diagnosing failures across components. It’s also easy to say that less skilled or new engineers just need to put in the time to get good, but it’s often the case that the old hats have a potentially-automatible procedure for narrowing down the problem or tracking where the issue came from, or that the benefits of separate components don’t justify the increased rate of bugs and time spent tracking them down to fix them.

avi_vallarapu 4y ago

I believe that the great examples always arise from OpenSource projects. The design and modular code always play a great role in increasing the ability to customize or add features and also have an increase in collaboration from more volunteers. Developer life gets more interesting with a massively improved code quality when some tough decisions are taken much earlier.

yCombLinks 4y ago
There are great examples in open source, but my main issue is open source projects are usually frameworks. Frameworks solve a lower level problem than end user application code. I often see excess abstraction and unneeded flexibility in business code that just makes the problem domain harder to understand, without ever providing value.

jasmer 4y ago

This is the economic value of refactoring, right on paper. 'Complexity is the mind-killer' - Linus Atreides

uptownfunk 4y ago

A wise engineer who I worked closely with told me "the job of the engineer is to manage complexity. Engineers don't like complexity". There are those hard-learned sutras that just make life so much easier down the line. Down the line always comes (unless you are playing career musical chairs and are willing to gamble that you won't be around to have to deal with the code mess when there's a sev2-, but as they say.. karma's a b..." Avoid premature optimization TDD Avoid complexity Readable code Documented code Probably helps avoid most problems.

beders 4y ago

Architectures need to be judged by their difficulty to make code and data changes. Changes in a stratified architecture are much simpler than in a layered design. (Changes in a layered design almost always mean changes to all the layers)

farrugp 4y ago

Can't wait to read this, really resonates with me as I'm dealing with this right now at a big FAANG company. In my experience the problem is that as with all things it's all about balance. We shouldn't throw away architecture entirely and write stupidly simple, quick solutions because they will be messy. But we also shouldn't over-abstract things so much that only the person/people who built the system can understand it and work with it. Both could have dire consequences for an organization, making building new features and delivering value to users slow, difficult and costly. Once I finish reading this MIT paper, I want to dig further into exactly what makes software 'complex'. In my experience: - too many layers of indirection - overly theoretical concepts not grounded in real world thinking - lack of documentation - bad naming - lack of testability - tight de-coupling - following 'best practices and patterns' without clear justification - trying to solve for problems before they exist We should be building systems that are grounded in concepts that are easily understandable - which is exactly why Object Oriented Programming has been so successful. We write programming languages as a means of communicating with each other about program logic, why not do it in terms that we as users already understand in the real world?

jen729w 4y ago

In the late 90s I travelled the world with a couple of CD-ROMs installing NT 4.0 on individual physical servers. I understood the entire stack. I was the only engineer and I was often very remote. In the mid 2000s we installed Server 2003/8 on a whole bunch of physical servers in a data centre. This was for one of Australia’s ‘big four’ banks. Me and a team of less than ten, most of whom I still know, managed the entire thing. We were the ‘3rd level infrastructure’ team. Most of us understood most of the stack end-to-end, although there were some specialities. In 2023 I work for an IT integrator. These days we use Azure, because Microsoft has fooled us in to thinking that it’s cheaper. My current project is, essentially, a website migration. Not even a big or complex website. The project has been running for 6 months. I only joined recently so I can’t be sure but we’ve probably burned AU$2m. The schedule I’m managing has us doing the cutover in June. That is optimistic. The architects are currently trying to work out how Azure [details] connect to Azure [details] while still [security] and being able to [complex integration]. Every day some new issue appears that the architects have to figure out how to work around. No single person has a goddamned clue how the end-to-end thing fits together. Not one, not a clue. That scares me. ‘Architectural complexity’ has crept down to the level of infrastructure. It’s painful to see. I hope it stops soon but I am not hopeful.

lifeisstillgood 4y ago
This is the essence - that a system needs to fit inside one persons head. All of it. There maybe a few people who understand all of a Boeing airplanes systems - there certainly were when they first flew 747s. Maybe a couple now. but once it stops fitting inside one persons head then the thing is literally only possible to design by committee - it cannot fit together and perhaps should stop being called a single system
- bob1029 4y ago
  > it cannot fit together and perhaps should stop being called a single system The magic is the enterprise message bus. It is meme crap until you actually need one. When it works, it really works. This is the only thing I've ever seen realistically tie together certain industrial environments.

hinkley 4y ago

I've long had an allergy to accidental complexity but recently had some new aspects illuminated for me. I had a block of code that had been added to by others, making it a bit of a recursive mess with a handful of bugs that were hard to reproduce let alone fix. When I finally got tired of it all, I sat about to replacing the recursive code with some iteration, and ended up with DP code, which avoided a bunch of duplicate errors and the need for caching. Without all the DP terminology, it's what other people would call building a (programmatic) plan and then executing it, rather than a depth first search algorithm. I used to use this quite regularly, but have only used it a couple of times on my current gig. Strictly speaking planning an entire action before starting it might result in a bit extra data hanging around because of building the data structures ahead of time, but at the end of the day it allowed me to eliminate a whole lot of steps that seemed necessary, and also remove duplicate warning messages. It wasn't necessarily faster than it could be, but it was way faster than I ever expected it to be. Writing obvious code with clear actors and data flow steps makes it a lot easier to add features, and make performance improvements. Obscure code leads to more obscure code (qualitatively and quantitatively).

malcolmstill 4y ago
Can you define your use of DP here? I'm not sure if you mean dynamic programming, or design pattern or something else, and am curious about your insight.

hyperthesis 4y ago

They find architectural complexity accounts for defects (x3), productivity (50%) and staff turnover (order-of-magnitude). The "McCabe cyclomatic complexity metric" doesn't predict the last two. Instead they use the "MacCormack" method (who supervised this PhD...), so I wonder how it differs? funfact: they reference an evaluation of Mozilla'a refactoring (the one Joel said you must never do). It's a PhD thesis, and it's long, detailed and academic (tautology), yet lacks an introduction, so: The Missing Introduction "Architectural complexity" here doesn't mean the complexity of architecture features themselves (e.g. architecture astronauts making too many layers etc), but direct and indirect interactions between parts. Instead of diagrams of arcs and nodes, they believe a matrix representation helps show complexity (vertical axis: using; horizontal axis used - or maybe vice versa - with dots at that location), called a "Design Structure Matrix" (DSM). The key sections seem to be: 3.6-7 [p.34-42] about DSMs 5.1 [p.67-83] about complexity by the "MacCormack approach" [pasted below, from p. 70] [5.1.2.1] Capture a network representation of a software product's source-code using dependency extraction tools [5.1.2.2] Find all the indirect paths between files in the network by computing the graph's transitive closure [5.1.2.3] Assign two visibility scores to each file that represent its reachability from other files or ability to reach other files in the network. [Visibility Fan In, Visibility Fan Out] [5.1.3] Use these two visibility scores to classify each file as one of four types: peripheral, utility, control, or core. They also have "network density" and "propagation cost" measures [p.76] --- Aside: A PhD thesis takes some time to read before commenting. I wish there was a way to facilitate discussion of the submission, in addition to the title-based opinion and experience that we have now. The patient_hackernews experiment (https://old.reddit.com/r/patient_hackernews/ https://old.reddit.com/r/patient_hackernews/) didn't seem to work out. Perhaps just a re-submission (or a bump?) labelled "for readers only" 24 hours later?

berniedurfee 4y ago

This is my jam. It’s really difficult to stop myself from chasing shiny things, but the last year running an app with a dead simple architecture has been very peaceful. Other than some hairy legacy stuff here and there, it all just works and the run rate is exceptionally low. When it occasionally breaks, it’s obvious what’s wrong and quick to fix. We spend most of our time making it better. Long live the 3-tier monolith.

bshanks 4y ago

This thesis provides empirical support that a specific measure of software architectural complexity is costly. Specifically, they look thru source code in an automated manner, construct the graph whose nodes are source code files and whose edges are the following cross-file relationships (page 73, section 5.1.2.1): - The site of function calls to the site of the function's definition - The site of class method calls to the site of that class method's definition - The site of a class method definition to the site of the class definition - The site of a subclass definition to the site of its parent class' definition - The site at which a variable with a complex user-defined type is instantiated or accessed to the site where that type is defined. (User-defined types include structure, union, enum, and class.) Then they compute the transitive closure of this graph. Then they compute two metrics for each node by looking at the transitive closure graph (page 76, section 5.1.2.3): - Visibility Fan In (VFI): how many other nodes have edges that go from the other node to this node? - Visibility Fan Out (VFO): how many other nodes have edges that go from this node to the other node? They observe that by looking at the VFI metric across various files, files tend to sharply cluster into either 'low VFI' or 'high VFI', and similarly for VFO (although some files may be high in one metric and low in the other) (page 79, section 5.1.3). They then classify each file as: - low VFI, low VFO: 'peripheral' - high VFI, low VFO: 'utility' - low VFI, high VFO: 'control' - high VFI, high VFO: 'core' They then find that 'core' files are the most costly, in terms of defect density, developer productivity, and probability of staff turnover.

sktrdie 4y ago

What exactly did decrease the complexity? And if, apparently, we can measure it, shouldn't it be part of any development process, similar to a linter?