17 ms·

The collapse of complex software

• throwaway23234 4y ago
  This is the truth: We get usefulness and simplicity from complexity. The more useful AND simple you need something to be, the more complex it actually is.
• fbn79 4y ago
  Probably many huge codebases could be reduced to a handfull of good old SQL queries and nothing more... If you trash things like: framework boilerplate, complex graphical UI libs, devops complications, auto-testing ecc....
  • deleted 4y ago
    [deleted]
• parsley27 4y ago
  I like to believe that serverless technologies and cloud services reduce complexity for the org, but obviously that's at the cost of offloading that complexity to the cloud providers (and welding an org's software to that provider for years, if not decades). So different, but not less complex overall. But maybe there is value in having some of that complexity consistent across a small number of cloud providers.
  • hinkley 4y ago
    Every time our operating costs go up (eg, cloud providers), something else in the organization gets more complicated in order to make up the shortfall. Or super simple when we go out of business. No free lunches here.
    • parsley27 4y ago
      That assumes operating costs go up with cloud providers. Some orgs see savings, but obviously not all.
      • hinkley 4y ago
        Some developers are merely bad at math. Most are awful. Others take this flaw to epic proportions. Be afraid of the developer who confidently tells you the math works out. It's true that a lot of organizations had operational teams that have become money pits, or pushed back on all quality of life improvements because they don't have the talent, the budget, or the imagination to pull it off. When we move to the cloud we start aspiring to these things we didn't have because they were expensive. I don't know of anyone who moved to the cloud and didn't move the goalposts. We were just talking in another thread, as we often do here, about how much YAGNI is going on out there. Yes, the price per feature goes down, but the overall price doesn't seem to. And I get stuck taking care of things someone else used to worry about, which is opportunity cost on top of it all. What is also true is that developers can learn a lot from their operational peers and avoid expensive mistakes. With cloud we have none of those peers. We have to learn everything first hand. By someone who is all too happy to let us wrap rope around our necks and then 'rescue' us from themselves. That's a perverse incentive and quite a setup for a fairly fucked up codependent relationship. At least with interdepartmental drama some of the money stays in the company longer before going to vendors.
        • datavirtue 4y ago
          We used to have devs bringing in frameworks and libraries without reading the manual. Now they spin up entire services or subsystems without reading the manual. You should read through the docs for MongoDB Atlas...which plainly illustrate that you now need a DBA unless you're cool with people pressing buttons in a panic during some type of self-induced performance issue.
  • tomrod 4y ago
    Having moved services from on prem servers to EC2 to OCP to EKS, I don't think these decrease complexity.
  • mark242 4y ago
    They absolutely do reduce complexity! For example, think about building, testing, and deployments. In pre-cloud Internet times, you'd have an untold number of extremely brittle bash scripts, cron jobs, rsync ssh key setup, fleets of build + test boxes to manually worry about disk space, pre-provisioned dev/QA database servers with also untold brittle sql startup/teardown scripts, and all of the requisite people whose job it was to solely maintain this infrastructure along with database tuning, build fleet monitoring, the list of menial tasks just goes on and on and on. Today, you have a yaml file in your .github/workflows directory. Now I agree that there are "different" requirements. Understanding the complexity of your workflows etc is no small feat-- but you're replacing such a huge amount of what used to be extremely expensive and brittle architecture with, basically, a text file or two. That's a huge cost savings.
    • beardedetim 4y ago
      Are you counting Chef/Ansible/Puppet/etc as things that came during the cloud era or that those things were not used to solve the problems you raised before the cloud era?
      • mark242 4y ago
        Having used Puppet, the promise of Puppet was very different than the realities of Puppet and imo more accurately reflects the "different complexities" point. The point of those provision-bare-metal-on-the-fly frameworks just evaporated when, for example, you could launch a container on Fargate with, again, a single text file, or similar on Heroku.
    • chasd00 4y ago
      idk, in the enterprise world all those things still exist but they're applied against AWS/other instead of your servers in the datacenter. Instead of "build+test boxes" you have "build+test instances". It takes the same level of redtape/approvals to get an AWS instance as it took to get a server in the old datacenter. All my enterprise clients have the dedicated infrastructure teams they always had, only now they're working in AWS/other and not the datacenter.
• rektide 4y ago
  It's hard to describe just how much of software engineering & it's enjoyability comes from attaining perspective, from reaching a point where you see & understand how a big pile of stuff works. > One thing working in complexity’s favor, though, is that engineers like complexity. Admit it: as much as we complain about other people’s complexity, we love our own. We love sitting around and dreaming up new architectural diagrams that can comfortably sit inside our own heads It's harder to acquire a sense of mastery & possession, of real understanding, when there's been multiple iterations of teams hacking on a piece of software, when it's many layered & has lost cohesion & concerted intentionality. Trying to describe how relishable, how enjoyable it can be to explore, to search & quest for meaning, to push designs that hopefully make sense, that hopefully grow, that hopefully wrangle: it's an under-sung & hard to tell story. It's interesting to me having such a big part of the world running off of a knowledge-working that is so under-described, so hard to communicate personally about.
• _xnmw 4y ago
  I'm waiting for the day that AWS collapses under its own weight. Heard horror stories from backend engineers working on the infrastructure/dashboard code there.
  • danuker 4y ago
    Judging by the price, it already has.
  • WaxProlix 4y ago
    If anything were going to do it, it'd be the labor crunch that we're seeing here. Without a constant influx of fresh-faced college coal lumps to burn in the tech debt engines Amazon would have a harder time of it. They've done a good job of pivoting though, and I think they'll weather the storm here (and be better off for it). See the increased comp to boost hiring rates - especially of non college hires - and an internal acknowledgement that addressing meaningful debt or internal pain is also a potential path to promotion. So, it'd be funny to see, but I don't think it's happening anytime soon.
    • chrisweekly 4y ago
      > "constant influx of fresh-faced college coal lumps to burn in the tech debt engines" made me literally LOL. That turn of phrase is gold.
      • strix_varius 4y ago
        Or even diamond, with enough pressure!
        • stjohnswarts 4y ago
          They're going to need diamond hands to hodl it together.
  • greatpostman 4y ago
    I think there’s basically a zero percent chance of this happening. Aws may internally be full of technical debt, but for ten years now Amazon engineers have been automating the reliability issues of the platform. The capabilities are expanding, but the core components are some of the most rock solid engineering on the planet.
• popcorncowboy 4y ago
  tl;dr premise = society is complex, software is complex; complex societies collapse... therefore complex software collapses, question mark, waggling eyebrows, surely makes you think etc etc. > ...Opinions expressed in this blog are mine *and frequently wrong*. Emphasis added. Equating the complexities of software with the complexities of societies (the premise of this post) is a fun and provactive blog post. Which is all this is. I was really hoping for a serious treatment that meaningfully dug into this, but that's my problem.
  • narag 4y ago
    I don't buy the explanation of societies collapsing because they're too complex, Some software systems do. But not because some esoteric systemic property of complex systems, but just because working projects get deprived of maintainers and eventually there's nobody that understands WTF original programmers intended and how to fix the holes. So incomprehensible portions of the system are circumvented or replaced with something that just works. Actually the same happens in societies, unless some group sabotages the workaround.
  • Forge36 4y ago
    OS/360? https://sudonull.com/post/111636-IBM-System-360-A-Failing-Story https://sudonull.com/post/111636-IBM-System-360-A-Failing-St... I think we've seen this with the rise and fall of hardware vendors. Most recently with the Mobile OS ecosystem (symbian, windows mobile)
    • popcorncowboy 4y ago
      My issue lies with software engineers making associations that society is like software. The causal factors behind why societies fail and why software projects fail are completely different. Complexity is a description of the system and a very bad measure to draw an equivalence between the two. The risk with drawing this equivalence is it grants license to software engineers to think that similar enough models apply to "managing software" as "managing societies". Because when all you have is a hammer... But society is not a nail.
• daenz 4y ago
  This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coprocessor" assist in seeing the solutions. I believe there are well-known strategies (taken from the real world) for managing hierarchies of complexity that ease the cognitive load of exploring any problem. We just need to get these approaches into a programming "langugage"!
  • andrepd 4y ago
    Can you elaborate and/or give examples? I read your comment and I can only see vague pseudo-profound statements.
  • astrange 4y ago
    This is essentially Alan Kay/VPRI's old research program. I'm not sure why, but when people get into this they always think dataflow programming will solve everything - we already have Excel and it does do a lot, but it doesn't do everything.
  • papito 4y ago
    It's a solved problem, if you are solving the right one. Most companies out there are solving problems they don't have. The FAANG disciples descended upon the world to preach the gospel of Complexity, and everyone bought it - because everyone wants to be like Google. What, you think "microservices" is a new invention? We called it distributed systems, and we really knew not to go there unless we wanted to decimate our productivity and sleep.
  • lbriner 4y ago
    Good luck but your motivations will probably not line up with the masters who dictate things that become complexity. For example, I look at our software running on .net MVC and think as an Engineer, it simple and knowable but the Front-End Team are less worried about simplicity and more on flashy front-end stuff since that is their job. We end up bolting on a Front-end JS framework and complexity immediately ramps up by like 300%. Are they wrong for wanting a better and more flexible front-end? Not necessarily, I mean all the other companies have cool stuff and if we don't maybe our company dies. Ditto for lots of other examples...
• groby_b 4y ago
  Our industry's "solution" to complexity is somewhat unique: We sweep it under the carpet. The reason that actually works is that the marginal cost to reproduce an existing pile of complexity is zero. Once we've got a sufficiently complex pile that actually has promise, everybody duplicates the pile. And then puts an additional layer on top. These many pile+layer combos then co-evolve (and co-erode ;) At some point, something so useful has evolved that we start copying that pile... and put new layers on top. Rinse. Repeat. The complexity rarely collapses because those new layers can add support beams, if necessary. Worst case, a layer gets ablated, and we try again from one step below. We don't ever go back to the beginning. (This also explains, to me, why Enterprise software is what it is - the complexity piles can rarely be shared or reused, and so there's no co-evolution, little chance to settle on the best pile) And, to be fair, it's similar in other disciplines, but maybe a bit slower. Mechanical engineering has individual components, larger components built from those, and so on. It just is more expensive to copy the piles^W components we know to work. Total collapse is exceedingly rare. (That even holds for human societies - they, too, often "just" collapse down to simpler societal expressions. And we do co-evolve larger and larger forms of society. It's just that the erode-and-try-again process is uniquely painful for the humans involved)
  • nonrandomstring 4y ago
    "Don't worry. We'll fix it when it goes bad" -- Dolittle (Dark Star)
• chrchang523 4y ago
  Note that Google's "Frequent rewrites" practice -- see section 2.11 of https://arxiv.org/pdf/1702.01715.pdf https://arxiv.org/pdf/1702.01715.pdf -- substantially reduces the severity of this problem (though it requires a good higher-level architecture).
  • undershirt 4y ago
    > 2.11. Frequent rewrites > Most software at Google gets rewritten every few years. > This may seem incredibly costly. Indeed, it does consume a large fraction of Google’s resources. However, it also has some crucial benefits that are key to Google’s agility and long-term success. In a period of a few years, it is typical for the requirements for a product to change significantly, as the software environment and other technology around it change, and as changes in technology or in the marketplace affect user needs, desires, and expectations. Software that is a few years old was designed around an older set of requirements and is typically not designed in a way that is optimal for current requirements. Furthermore, it has typically accumulated a lot of complexity. Rewriting code cuts away all the unnecessary accumulated complexity that was addressing requirements which are no longer so important. In addition, rewriting code is a way of transferring knowledge and a sense of ownership to newer team members. This sense of ownership is crucial for productivity: engineers naturally put more effort into developing features and fixing problems in code that they feel is “theirs”. Frequent rewrites also encourage mobility of engineers between different projects which helps to encourage cross-pollination of ideas. Frequent rewrites also help to ensure that code is written using modern technology and methodology.
• bitwize 4y ago
  > One thing working in complexity’s favor, though, is that engineers like complexity. Admit it: as much as we complain about other people’s complexity, we love our own. We love sitting around and dreaming up new architectural diagrams that can comfortably sit inside our own heads Where does this yutz work? Because he doesn't speak for me. Generally, I write a complex program because I don't have time to write a simpler one; and when I have to deal with that complexity later I'm like arrrggh. Rube Goldberg machines are fun to watch, but no one wants to maintain, change, or build on one.
  • chasd00 4y ago
    didn't Mark Twain say something like "i'm sorry this letter is so long, i didn't have time to write a shorter one". I feel like that with some of my code at times.
    • bitwize 4y ago
      I believe it was Blaise Pascal, to whose missive I was making a deliberate reference.
• eldude 4y ago
  Complexity in software is a lot like complexity in cuisine; most chefs want to stretch their abilities and create something memorable, transcendent even, but at the end of the day unless they’re cooking for themselves or friends and family there’s a business to be run. So complexity has a diminishing return for economical reasons.
  • chasd00 4y ago
    i kind of like that analogy. And many times, a ham sandwich and a beer really hits the spot.
• 0xdeadbeefbabe 4y ago
  > It takes a lot of discipline to resist complexity, to say “no” to new boxes and arrows. To say, “No, we won’t solve that problem, because that will just introduce 10 new problems that we haven’t imagined yet.” Or to say, “Let’s go with a much simpler design, even if it seems amateurish, because at least we can understand it.” Or to just say, “Let’s do less instead of more.” Discipline and complexity are best friends. Laziness and procrastinating seem more valuable for avoiding complexity.
• lbriner 4y ago
  An interesting problem is whether the complexity is unavoidable and if it is, we need to teach migration and proxying in Software Engineering 101. We have an old app: dotnet webforms, it's clearly hanging on for its life and needs replacing but the mythical rewrite is impossible if you intend to rebuild and switch, it just involves too much time away from your core product. Instead, we had to setup a proxy which could send all traffic between one of the two apps so we can move things a few at a time to the new system and all new work just gets built in the new system. It still takes a long time so I am now wondering whether as soon as we finish, we should immediately setup another new app alongside the current new one and repeat indefinitely. It certainly wasn't easy to do because old and new frameworks are rarely able to share cookie formats/sessions etc so some hand-cranking was required.
• loudmax 4y ago
  I highly recommend Rich Hickey's "Simple Made Easy" talk as a way to think about software complexity. We all know that complex systems are hard to work with, but I've found his presentation very useful in understanding the nature of complexity. Understanding complexity helps being able to avoid or minimize it. For example, easy is not the same as simple, and simple changes to one part of a system can introduce great complexity at the macro level. https://www.youtube.com/watch?v=LKtk3HCgTa8 https://www.youtube.com/watch?v=LKtk3HCgTa8
  • appleflaxen 4y ago
    I've watched that video at least three times, but I still can't articulate the exact distinction he is making between simple and easy :/
    • aidenn0 4y ago
      Add two 10000 digit numbers by hand. That's quite simple to do, but it's certainly not easy.
      • honkler 4y ago
        it's easy but mundane. If you pay me, I'll do it.
    • kortex 4y ago
      A for loop is easy: everyone can conceptualize doing a process N times. A map is not as easy of a concept in some ways (it's more abstract, you are taking N elements of X to N elements of Y). But a map is simpler than a for loop, because there is no state between steps: it's completely symmetrical. The for-loop "complects" each iteration together, even if it's as little as incrementing an integer. It's a side effect, however trivial. To kick it up a notch, now imagine a 2-D loop. Now we have 2 counters. If it's a 2-D array, each addressing operation interacts with 2 counters. But map just needs a second map chained on, and each map is completely "unaware" of the other map. You could map over N dimensions and each map is symmetrical, while the for loop needs an ever increasing number of counters. This slight difference in complexity shows how complexity compounds with other complexity.
• olah_1 4y ago
  Complexity is something that is underrated especially in the crypto-currency world. They claim to be building for the next hundreds of years, but you wouldn’t know it.
• roansh 4y ago
  This is a good talk by Jonathan Blow on/against software complexity https://www.youtube.com/watch?v=pW-SOdj4Kkk https://www.youtube.com/watch?v=pW-SOdj4Kkk Also talks about the risk of losing knowledge, and a way to avoid that (make multiple copies).
  • neoneye2 4y ago
    Great talk indeed. Just watched it. Thanks.
• rglover 4y ago
  The more abstract the mental model is for building the thing—at the tool/code level—the more likely it is that the thing being built will be a difficult to maintain mess, if not an absolute "burn it all down and start from scratch" failure. Fix the tools, and you start to fix the code. Unnecessary terminology, confusing APIs, superfluous patterns...it's like watching someone blindfolded in a dark room trying to justify the high price tag on their computer science degree. After years of running my mouth, I proved my rants about the unnecessary complexity by building my own JS framework [1] and I'll never look back. The emperor is bare-ass naked. [1] https://github.com/cheatcode/joystick https://github.com/cheatcode/joystick
• jrochkind1 4y ago
  I appreciated this essay. Among other things, it suggests that there are some incentives for complexity: That engineers just enjoy complexity, and that making simple designs might not be good for the respect of your peers/bosses/career: "A simple design might make it seem like you’re not really doing your job." While that's true, I think the essay isn't clear enough about the fact that simplicity is just plain hard. Complexity is sort of the "natural" outcome of "adding things". It takes time, effort, and skill to create software that can do what is asked for it while maintaining an internal simplicity. Even if everyone involved really wants it. I do not think software complexity comes only, or mainlly, from people not wanting it enough (explicitly or implicity), from incentives to complexity. It's just plain hard. It's a mistake to think just wanting it harder (or having the right incentives) is sufficient. It may be the key challenge in software engineering -- how do we make software that does everything that's asked for it (powerful, complex things), while maintaining an internal simplicity? You can not accomplish it by cargo-culting any "design patterns" or other approaches. It's still a skilled craft. That comes from years of experience with the intent of developing this skill (years of experience alone aren't enough), looking at other people's code who are also trying to do it, and just some aptitude. And domain knowledge, if you understand the nature of the problems you are working on better, you can design simple solutions for them better--contrary to some current practices treating developers as interchangeable. Yes, you need your organization to allow you to do it (including taking the more time it will take), but that's necessary but not sufficient for the outcome.
  • mananaysiempre 4y ago
    There is the well-worn Pascal quote about having no time to write a shorter letter[1]. There is also, I think, an important point hidden in the “domain knowledge” part: the freedom to change the spec; the effusive texts on Charles Moore’s approach to Forth programming[2,3], however much scepticism they deserve otherwise, are very direct about that freedom being essential. But that means simplicity is a cross-cutting concern, like security, and does not particularly fit into any environment that would attempt to separate design and implementation. It needs “redraw all the module boundaries” to be a plausible (if improbable) response to “this one 300-line part of this one module is awkward and three times longer than it really needs to be”. Aside from the obvious administrative problems, suggests that the complete edifice needs to fit into the mind of a couple of people at most or the iteration simply will not converge most of the time. I don’t see a good solution to this. [1] https://quoteinvestigator.com/2012/04/28/shorter-letter/ https://quoteinvestigator.com/2012/04/28/shorter-letter/ [2] http://www.ultratechnology.com/forth.htm http://www.ultratechnology.com/forth.htm [3] There was an example about changing a negative-feedback control loop, written into the spec by hardware engineers emulating analog circuitry, into a couple of fixed-point operations that behaved differently but did the actual job just as well; I can’t find the reference right now but this a minor domain-specific change among the kind I have in mind.
• henning 4y ago
  > engineers like complexity. Admit it: as much as we complain about other people’s complexity, we love our own No. Speak for yourself. I am happiest when my solution to a problem is simplest. I am least happy when a solution requires introducing new infrastructure or major new ideas.
  • commandlinefan 4y ago
    I feel the same way, but we both have to accept that we're tilting at windmills. Complexity can't be measured, so there's no way to demonstrate, much less prove, that the time you spent reducing complexity was time well spent. You can demonstrate a run time or memory performance improvement. You can demonstrate a new feature. Unless you can reduce complexity in zero time and do so without introducing any unintentional side effects/bugs, you're just hurting yourself when you try.
  • a_e_k 4y ago
    I came here to say this. I have no idea who first said it, but one of my favorite sayings is: "Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands." I strongly believe that this applies to computers as well. I love it when a system neatly solves its requirements in a way that looks effortless. When an algorithm looks like it just barely works for all the potential edge cases without any special handling because those edge cases just miraculously happen to work out as they fall through the happy path just like any other case. That's when a solution "feels" right. If I can present my solution/code/explanation to a junior engineer, a C.S. student, or even a non-technical person and they get it, then that's when I feel like I've really done my job. It's music to my ears if I can make a problem look so easy that someone asks "Is that it all it takes?" (until they've tried to solve it on their own :-)
• nonrandomstring 4y ago
  Very nicely written. A lot of gems in this short essay. What stood out for me was: > It takes a lot of discipline to resist complexity That can be taken in many ways. Self-discipline, to resist convenience, the omniscient and omnipotent desire to control everything and always have the right answer immediately. That takes discipline to let go of. Also to discipline others, to say no, push back, become the master and set firm boundaries against crushing 'demands' for complexity. And intellectual discipline, to see clearly, elegantly, eliminating what is unnecessary. Over-complexity is a failure of the ordering principle of an organism, a step on the road to chaos and entropy.
• waynesonfire 4y ago
  one word... path of least resistance. this is why we can't have nice things. it takes WORK to reduce complexity.
  • CRConrad 4y ago
    That's four words.
• elisharobinson 4y ago
  it is a knapsack problem for sure , minimize ( tooling cost and complexity ) and maximize ( feature complexity aka "richness" or "usefulness" )
• danielovichdk 4y ago
  I will tell you this. It's based on my 20+ years, 50 different teams and many hundreds, perhaps thousands of encounters with software developers. It's almost never the domain complexity that is high. Sending a rocket to Mars is complex. And perhaps complex every time. Dealing with people that has to work together can only be simplified when agreements are made and agreements are kept. Otherwise you cannot trust and have faith in people. It comes down to far and near organization leadership, mandate and empowerment in small teams with few communication links outwards, clear expectations from whomever is asking for something and process management. Writing code is rarely hard if you know the expectations and the boundaries of the task. Empowerment gives the team the possibility to act on what they discover and encounter. Don't look into the future. Because you cannot. And you are both naive and arrogant if you believe you can. Deal with what is known now! One thing is complexity in code. Another is complexity in business. Some things are not made for software to solve. Less experienced people are generally guided by their beliefs around their own ideas. If the mind is locked onto something it's hard not to thing it's a good idea. Usually in software there is simpler approach. Look for it I can go on because I help companies and teams for a living. But I will stop here
  • _gabe_ 4y ago
    > Don't look into the future. Because you cannot. And you are both naive and arrogant if you believe you can. Deal with what is known now! Yes! 100% this. We need less fortune tellers and more pragmatists.
  • 0xFACEFEED 4y ago
    > Don't look into the future. Because you cannot. And you are both naive and arrogant if you believe you can. Deal with what is known now! NO! This mentality is short sighted by definition and leads to stagnation. It may work within large financially healthy companies that aren't trying to rock to the boat. Making assumptions about the future is incredibly important for us to move forward. But with these caveats: 1) Acknowledge that you are taking a risk and that you may be wrong. 2) Validate your assumptions as quickly and as often as you can. 3) Be prepared to change course whenever your assumptions are invalidated. > Less experienced people are generally guided by their beliefs around their own ideas. If the mind is locked onto something it's hard not to thing it's a good idea. Usually in software there is simpler approach. Less experienced people are frequently the driving force behind new ideas precisely because of their belief in it. As experienced people we should be encouraging them to carry forward responsibly instead of projecting our aged and jaded perspectives. > I can go on because I help companies and teams for a living. Honestly, that's a weird way to phrase what you do.
• an9n 4y ago
  I seem to be in drastic opposition in that I hate complexity, and will do anything to avoid it. I shudder with horror when I look at the teetering stacks all over the place these days.
• angarg12 4y ago
  Preventing the Collapse of Civilization / Jonathan Blow [1] https://www.youtube.com/watch?v=ZSRHeXYDLko https://www.youtube.com/watch?v=ZSRHeXYDLko
• jv22222 4y ago
  > It takes a lot of discipline to resist complexity, to say “no” to new boxes and arrows. If we refactor a very large function into smaller sub functions we are essentially adding new boxes... but are we creating complexity, or simplicity?
• skadamat 4y ago
  This was my favorite deep dive post on this exact topic from a few years back in the Atlantic: https://www.theatlantic.com/technology/archive/2017/09/saving-the-world-from-code/540393/ https://www.theatlantic.com/technology/archive/2017/09/savin...
  • pugworthy 4y ago
    The 911 overflow is an interesting bug. Kind of asks for a philosophy of, "Limit your build, but don't build in limits" Once I was on a research ship, and the data acquisition system crashed on the 256th day of the year. Care to guess the issue? :)
• krm01 4y ago
  For the last 12 years, I’ve worked with 40+ different software teams (helping with UI/UX design [1]). From small scrappy startups to fortune 500. The teams that were great to work with and ended up having the biggest wins, were always small teams with a great cultural balance. They always strived for the simplest solutions. You could trace this balance of engineers in all of the cases back to GREAT hiring by the CEO. The teams that were the least productive and just burnt through VC money were the ones that had a handful of toxic engineers at their core. Finding complex and seemingly clever (over engineered) ‘solutions’ seemed to be their goal in every meeting. A way to find out if your team belongs to the second group, measure the amount of jargon that’s being used in meetings. [1] www.fairpixels.pro
• js8 4y ago
  I feel like it's a bit backwards. The software complexity increases because people lack the patience to tackle the existing complexity, or learn why things are complex, and prefer to believe that it can be simple, or get job done somehow, and in order to do that, they (somewhat egoistically) go their own way, which actually - barring occasional good insight - mostly increases the total complexity. The same goes for societies. If people have more autonomy (like, not have external pressure to be productive), they might be more willing to contribute to the society as a whole, to fix it, and also take a more long-term view of the future. IME, in established open source software, there is less churn and accidental complexity. Complexity is also often the result of in-fighting, which again, is the symptom of a society where the trust is low. Though I am not sure, honestly, if there is a reliable way to increase the trust inside society rather than the actual collapse. That paradoxically often leads to increased cooperation.
  • atoav 4y ago
    I believe many software developers often tend to write software which has precisely the segree of complexity they can just barely manage. There is not many people who have the guts to build a simple straightforward step for step script. Software developers that fall into love with the beauty of their own abstractions can be a real problem.
    • commandlinefan 4y ago
      > precisely the segree of complexity they can just barely manage "Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?" -- Brian Kernighan
    • aylmao 4y ago
      +1, plus I'll add that I think there's organizational factors too. Overall, organizations incentivize complexity, and I'd like to point out three ways in which they do: - One is that new systems tend to be rewarded, because people and organizations tend to associate "new" with "better". Related, people and organizations tend to associate high complexity systems with "hard work", and the transactional view of work tends to err on the side of rewarding "hard work". - Next is that complexity digs a moat of job security. I don't think this is a bad thing, job security is good. Again, unfortunately the transactional view of work can easily devolve into "we can pay someone less to do what you do" and building a complex system is a way to counter that. - Lastly, a story. At some point I worked on a team that quadrupled in size to tackle huge project, only to then fall into a sort of "identity crisis" because the project was done and there wasn't enough meaningful work afterwards for everyone. This insane growth, in my opinion, was also fueled by egos. The overarching team manager seemed pretty adamant in finding more people to hire, but I don't necessarily remember us feeling understaffed. They clearly expressed they wanted to use all the headcount the company had authorized, and the primary reason they always mentioned was that we'd "lose it if we didn't use it by the end of the year". This led to an "induced demand"-like situation. By the end of it, we had a huge team, the bigger meta-project was done, and everyone needed to find the next thing to work on and build. I saw plenty of proposals for solutions looking for a problem, and worked on some myself too before leaving.
• agentultra 4y ago
  A legacy system exists: it’s big, it’s complex, and no one fully understands how it works. Architects are brought in to “fix” the system. They might wheel out a big whiteboard showing a lot of boxes and arrows pointing at other boxes, and inevitably, their solution is… to add more boxes and arrows. Nobody can subtract from the system; everyone just adds. In my experience this is the only safe thing to do when you don't fully understand the system and cannot reason about it in a precise way. You create well defined abstractions over the messy parts, verify the abstractions, and write lots of tests. Eventually you can start deleting code. Although nobody ever ends up deleting code because they skip the "well defined" part and end up adding more indirection, afraid to delete code. The only real way to tame complexity is to reason about it with precise, mathematically-defined abstractions. You can't invent Paxos using boxes and arrows on a white-board and prove the minimum quorum required without showing your work. Fortunately we have languages and systems that can automate a lot of the effort in checking our work as long as we can express it in formal language. The other problem with this is that learning how to use these tools and techniques requires a lot of effort that isn't highly valued or rewarded in industry. Most businesses are fine with the trade off of errors and mistakes down the line if they're first to market. Ship it now, fix it later. We do a reasonably good job of preventing enough obvious errors with our tooling and processes. We can have software that obviously has no errors or software that has no obvious errors. Time and again we chose the latter because the former is too difficult and doesn't reward us with money now.
  • commandlinefan 4y ago
    > the only safe thing to do when you don't fully understand the system and cannot reason about it Well, the other option (the most sane one, in fact) would be to not mess with it until you understand it and can reason about it - but in every corporate "how long is that going to take"/"what can we do to speed this up"/"why aren't you finished yet" quick-and-dirty-ocracy, that's not an option unless you can do that in about a day.
• alexfromapex 4y ago
  I have been thinking about this exact issue, with regards to the way clinical research is conducted.
• deleted 4y ago
  [deleted]
• commandlinefan 4y ago
  > engineers like complexity Is that really true? If it's descriptive, then I'm not an engineer (and I may well not be). I don't like complexity at all, in any form. I want the least complex solution to any problem that will work. However, I've mostly given up on decomplexifying, because it's mostly a waste of time. Software becomes complex because getting something done fast, right now, "this time", because, "we're in a hurry this sprint" by duplicating code will always win over trying to modify something to be reusable. If you try to spend time fighting complexity, you won't just be fighting the complexity inherent in the existing system, you'll be fighting your coworkers who've given up on fighting complexity and just get their tickets into a done state.
• bell-cot 4y ago
  Back on the "Collapse of Complex Societies" side, my impression is that a fair number of such collapses are now blamed on climate change, and/or resource depletion. The Anasazi thrived from ~700 to ~1130 due to an unusually rainy spell in SW North America. When the period of heavier rains ended, so did the agricultural productivity that supported their large population and complex society. Similar reasons may hold for the fall of the Khmer Empire ( https://en.wikipedia.org/wiki/Khmer_Empire https://en.wikipedia.org/wiki/Khmer_Empire ). A number of sharp, grim setbacks for European civilizations occur suspiciously close to major volcanic eruptions, which could have hit their agricultural foundations hard. ...and "yes", I read second ~half of the article. My point here is that the author's analogy works at least as well where he did not bother to apply it as where he did.
• jldugger 4y ago
  > One thing working in complexity’s favor, though, is that engineers like complexity. Honestly, I hate complex solutions. Even my own. What I like, and what impresses me, is the simple solution that takes a lot of work or math to prove it works. Like, emergent behavior type shit. Control theory, for example, runs systems with very simple runtime behavior. A few open loops with some parameters. The analysis to chose those parameters is where the engineering happens. Microservice practices are not yet that simple, but some day I hope they can be.
• jacobsenscott 4y ago
  This "software grows too complex" is certainly a culture issue, and not a software problem. It mostly comes from businesses throwing a bunch of under-skilled labor at the problem, and demanding more and more too quickly. Of course it will outgrow the ability of that team to work with, reason about, keep clean, and extend. They will get frustrated and move on, and a new group comes in, re-writes it from scratch, and so it goes. Look at software that has been around for decades, continues to improve, never "needs" a re-write, and will never be re-written. It exists and is not uncommon. Things like the linux kernel, all the GNU utils, emacs, vim, apache, nginx, postgres. The list goes on. We know how to write software that lasts for decades, continuously improves, and doesn't need constant re-writes. You give it to skilled people who care about it, and don't put artificial corporate constraints or impose fad architectures on them. I'm not saying the corporate model is "wrong" - it works. "Works" means it builds companies and generates profit. That is the goal. But it will not generate stable long lived well designed software. That's okay though because that isn't the goal. Sure, the poor quality cuts into the margins a bit, but not enough to matter.
  • sethetter 4y ago
    I'm curious how big of a factor a consistent core team is to having long-term sustainable and stable software. When companies have people rotating on projects constantly and the original vision and design principles of the product are lost or changed, I can only imagine that contributes to the problem.
    • phendrenad2 4y ago
      I think that's a big factor. This is what RSUs and bonuses are supposed to solve: You entice employees to stay rather than job hop. However, companies have gotten too greedy and RSUs amount to a pittance, a vestigial perfunctory play act mimicking a bygone era.
      • amenghra 4y ago
        It's common for tech companies to give RSUs which vest over 4 years with annual refreshers being comparatively quite small. The financial incentives can dwindle after just 4-6 years at a given job.
• kitanata 4y ago
  This is crazy. I just gave a nearly identical presentation for a series of interviews. One at AKASA, one at Scribd, and one at Panorama. Like I gave this presentation, with the exact same title 2 weeks ago to a bunch of junior engineers at these various companies. It is super timely. I’m consistently surprised how multiple people can arrive to the same point, via the same inspirational sources at the same time and reach completely different conclusions. I’ll have to write up a blog response to this article.
  • LeonenTheDK 4y ago
    Please do! I'd love to see more takes and discussion on the matter.
• jeffreygoesto 4y ago
  Hey. Come to the automotive industry, where your code will live for whooping twenty years! =;-D
• psychoslave 4y ago
  >A software company with more modest aims, that has a stable customer base and doesn’t change much over time (does such a thing exist?) Here is the choice, stagnate and disappear through incapacity or adapt and perish in burnout.
• ozten 4y ago
  Reminds me of the simplicity potential STEPS Toward the Reinvention of Programming (Oct 2012 Alan Kay and folks) https://news.ycombinator.com/item?id=11686325 https://news.ycombinator.com/item?id=11686325
• eranation 4y ago
  > Simplicity of design sounds great in theory, but it might not win you many plaudits from your peers. A complex design means more teams to manage more parts of the system, more for the engineers to do, more meetings and planning sessions, maybe some more patents to file. A simple design might make it seem like you’re not really doing your job. “That’s it? We’re done? We can clock out?” And when promotion season comes around, it might be easier to make a case for yourself with a dazzling new design than a boring, well-understood solution I couldn’t disagree more. The vast majority of companies I worked with recognized and rewarded simpler designs, the vast majority of software engineers I worked with would rather delete code than write code, rather throw away a complex design to a simpler one, and with some NIH here and there, YAGNI and KISS seems to be triumphant and what would get you promoted, not adding more arrows and boxes.
  • mike00632 4y ago
    Haha, this makes me want to work at your company.
  • a1369209993 4y ago
    > The vast majority of companies I worked with recognized and rewarded simpler designs, the vast majority of software engineers I worked with would rather delete code than write code What made up fantasy world do you live in, and are they accepting immigrants?
  • wtetzner 4y ago
    Yeah, the thing that seems to be missed here is that designing simple things is hard. Things are complicated because the first version of pretty much everything ends up being more complicated than it needs to, and most engineers don't get the chance to revise/rewrite the first version into something simpler.
• Willish42 4y ago
  > A software company with more modest aims, that has a stable customer base and doesn’t change much over time (does such a thing exist?) will be more like the humble tribe that follows the yearly migration of the antelope and focuses on sustainable, tried-and-true techniques. (Whether such companies will end up like the hapless Gauls, overrun by Caesar and his armies, is another question.) That last part is the key: with the strong preference for short term gains in an economic system where finances are considered and reconsidered quarterly, sustainability is an afterthought. "Growth at all costs" aligns with the dominant ideology of imperialism, and it's just as prevalent in corporations and organizations as it is in nation states. Literal millennia of evidence for that approach out-competing others is hard to beat, and I think if we knew how to survive a different way even at the micro level we'd probably live in a much better society.
• DenisM 4y ago
  Business assets depreciate over time, nearly all of them, it’s only the amalgam that can steadily appreciate, as assets are cycled in and out. Business is fine with that. The problem with software is that software deprecation cannot be readily modeled in Excel. We’re back to the problem of quantifying Engineering Debt. The best I can come up with number of WTF moments per unit of cyclomatic complexity. The second best is aggregate cortisol level of developers divided by the business value produced during a release.
• imbnwa 4y ago
  The word 'simple' is hella overloaded in software circles
• tasuki 4y ago
  I like that the author is working for Salesforce <3
• protomyth 4y ago
  I always thought it was because we really couldn't break the applications down without massive dependencies and the complexity of human procedures brought to the computer's logical world. It seems like folks who've built some form of common bus (shared data types and communications channels) do better scaling applications. If I can effectively split off a part of the application and plug a new part in its place, I can scale. OOP seems to love dependencies that accumulate until objects cannot be replaced because of all the connections. Clean separation is a hard problem. Computers are logical machines. Human laws and regulation are not. Modeling physics is an easier problem (programming not computing need wise) than calculating taxes. At one point in the US, every gallon of bio-diesel had to have its own serial number[1], this adds a level of complexity that goes beyond simple programs. My own thoughts (or my longtime hobby project) is that Agent oriented programming hasn't really been given a workout, and that decoupling data structures might lead to some improvements. I honestly think being able to replace parts of a running program in an orderly manner might be a indicator of a good direction. 1) https://mnbiofuels.org/media-mba/blog/item/1495-rins-101 https://mnbiofuels.org/media-mba/blog/item/1495-rins-101 - as a programming exercise, write a program to add, subtract gallons from a storage unit - it gets fun.
• p0nce 4y ago
  I think Software education is in its infancy, instead of teaching people how to make debt decisions to keep complexity and costs low, we actively teach them to write disposable code. That is the content of most courses: code that will be thrown away, and that has to be written under tight time constraints. And when it comes to keeping complexity low, there is zero theory to back up anything, so it's not taught. Our trade is somewhat similar to reading/writing, and if you were to teach someone to read/write, you would probably insist on good writing, not just "finish your novel at this date, I won't read it anyway". And you would have good material for teaching to read, material that is currently missing.
• c-smile 4y ago
  Nothing new in principle, "transition from quantity to quality" (quantitative differences beyond a certain point pass into qualitative changes) was discovered by Hegel around 1800. So nice, working application after adding next 5,000 more lines here and there suddenly becomes too fragile to perform its main function. Usual solution for such problems is composition/modularization. Software shall be made of simple but composable modules. Like text editor (relatively simple) shall be composable with syntax analyzers to make IDE possible. The very same syntax analyzers should be usable as web server modules for the Web. That dream requires design of (again) simple but powerful architectures of composition/modularization. Seems like each generation of software philosophers approaching this task anew. Linux with its stdin/stdout modularization. MS Windows with OLE. COM and CORBA, and so on. Probably it's time to evolve something like COM and CORBA into something new?
• BenoitEssiambre 4y ago
  In some sense this is the eternal question of life, the universe and everything, life being complex self reproducing software itself which adapts to environments and sometimes collapses when it changes. It's a fascinating subject, especially from the perspective of trying to push towards the limits of information and entropy. I'm not sure the root of the problem is solvable however.
• tbwriting 4y ago
  For those curious about what this looks like in the trenches: https://tylerberbert.substack.com/p/complexity https://tylerberbert.substack.com/p/complexity
• collaborative 4y ago
  Some systems, such as mainframes, resist being crossed with a big X for a long time. Their simplicity (raw tcp bytes that no REST API or HTTP website can outperform) and the big name (IBM) make it extremely resilient. Users' AHTs can't afford an extra 500ms between screens/pages. Users simply will refuse using any other system
• z9znz 4y ago
  > One thing working in complexity’s favor, though, is that engineers like complexity. No, absolutely not for me. I want simplicity. The problem is that our expectations for software systems is so vastly more complex than it ever was in the past. It always grows more complex. The needs and wishes only increase. Yes there are some developers who like building complex structures. Admittedly it can be fun. But eventually we reach a point where it's too much for one person to keep up with. Then the challenge shifts from complex software systems to complex knowledge and management systems. It is theoretically possible to have an infinitely complex system which is managed by a perfect, infinitely complex organization. In practice it doesn't work, for numerous reasons. One reason is that business drives software development, and business is seeking profit. So business doesn't want documentation or complexity management; it wants features. That means a lot of important information gets stored in a few heads of architects and senior devs. And inevitably, people leave organizations after a while. A benefit of being a broad generalist is that I can do a lot of things in a lot of languages on a lot of platforms with a lot of frameworks. But I can't remember details anymore. So I have to make things pretty obvious. Functional programming patterns help a lot, because I can usually isolate things into well-named functions which do what they say and nothing more. Then with composition, systems of moderate complexity can be fairly well understood and navigated by readers. The rest of the post doesn't offer any solutions other than to say, "oh well, that's how it goes". It doesn't have to be a hands-up situation. We as developers must try to balance the "demands" of sprints, product managers, and executives to push stuff out. We should repeatedly explain how technical debt will slow us down in reaching future goals, and how we should refactor and rethink our code regularly. It can be done, and I've successfully changed a few companies. It takes persistence and passion, but eventually you have a whole team onboard with an approach to managing complexity. It is very much worth the struggle.
• ngcc_hk 4y ago
  Not complexity guy, competition and innovation. So far the internet is still in its early stage, yes, it is not your “system” but the whole (internet, user, those tidy systems like hbo, Facebook, …). Complexity of the individual system just like any eco-system comes and goes. But what worry is whether the fundamental change (like before and after internet). Just like you observed individuals like us die is not a (perhaps but our individual) system failure. You do not much use basic to do computing or pascal. Even though in 1970s … Even internet itself has a under threat. That is what we should observe. Otherwise all is fine. Sone cones some goes, systems moved on. Even mainframe based system moved on.
  • CRConrad 4y ago
    > You do not much use basic to do computing or pascal. Not you, perhaps, but some do: https://www.lazarus-ide.org/ https://www.lazarus-ide.org/
• Archelaos 4y ago
  If we want a useful, testable theory that "complexity" causes "software collapse", we need to define both independently from each other and find a way to meassure both. Not that I doubt that there is something to be learned here (see for instance the famous Mythical Man-Month). But without a better theory, we are still poking around in the dark. (C64 pun intended.)
• Ericson2314 4y ago
  While there is much good work to be done on making simple shorter code more expressive there is also tones of low hanging fruit. These https://en.wikipedia.org/wiki/Law_of_Demeter https://en.wikipedia.org/wiki/Law_of_Demeter https://en.wikipedia.org/wiki/Principle_of_least_privilege https://en.wikipedia.org/wiki/Principle_of_least_privilege We just need to do this again and again and again. For example, we all know there should be / and everything should be done via file descriptors by now. Great, but who is making it happen?!?
• galaxyLogic 4y ago
  There is no solution. Software is isomorphic to theorem proving (Curry-Howard). Trying to get rid of complexity in software would be like trying to get rid of complexity in math in general. Yes you can often find a simpler proof for some theorem, but that depends on the theorem, on the problem. Just because you can discover a simpler proof for something doesn't mean you can discover a simpler, less complex proof to anything else. And didn't Godel prove that some things can't be proven at all. That would mean some programs can't be written. Still, mathematicians don't give up, they do their best. And so do programmers.
  • winternett 4y ago
    Think of software like car making. Cars once ran off steam, very simple and mechanical... Now cars run on combustion or electronics, but relatively none run off steam now, because the industry forgot to make steam engines evolve... The same goes for development... It's not impossible to rethink things in a more simple manner, it's just that we've stopped listening to the right size shoe for each foot that needs one because of shiny marketing. Technology doesn't become obsolete these days, it just becomes out-marketed and under-supported.
    • galaxyLogic 4y ago
      That is a good point. In software development momentum is everything. Why because you got to have many people working on it to allow it to become better. And "better" is a holistic thing: You got to be COMPATIBLE with other popular development tools, to be "better".
    • CRConrad 4y ago
      Your analogy certainly works, but probably not quite in the way you intended: 1) Cars are transitioning to electric propulsion, not electronic. Electronics have long been present in engine management of internal combustion engines too. 2) This is actually not new, but just a resurgence of something very old -- at the turn of the century (19th/20th, not 20th/21st), electric propulsion was quite common[1], the land speed record was held by an electric car[2], and the first "series-hybrid" system[3] was introduced[4]. 3) Steam engines -- discounting steam turbines -- are also just combustion-driven piston engines; the only difference is that the combustion is external, not internal. OK, so it's not the combustion gases themselves that are expanding within the chamber, but steam heated (created) by them. But that's a minor detail, IMO.[5] What does this mean for software engineering (or "engineering", in scare quotes)? Not sure... Probably something like: A) Old tech doesn't go away. It just falls out of use and gets forgotten. B) "New tech" often isn't. It's just old tech being uncovered or independently rediscovered. C) What we think is a huge innovation is often just a return to older tech. ("The Cloud" ~= the mainframe?) A + B + C = "COBOL, here we come!"? :-) Idunno. . EDIT -- oops, footnotes: ___ [1]: https://en.wikipedia.org/wiki/Electric_car#Early_developments https://en.wikipedia.org/wiki/Electric_car#Early_development... [2]: https://en.wikipedia.org/wiki/La_Jamais_Contente https://en.wikipedia.org/wiki/La_Jamais_Contente [3]: Where an internal combustion engine drives a generator that drives the wheels via electric motors and charges the main battery. https://en.wikipedia.org/wiki/Hybrid_vehicle_drivetrain#Series_hybrid https://en.wikipedia.org/wiki/Hybrid_vehicle_drivetrain#Seri... [4]: https://en.wikipedia.org/wiki/Lohner%E2%80%93Porsche https://en.wikipedia.org/wiki/Lohner%E2%80%93Porsche [5]: In fact, AFAICS a four-stroke (preferably diesel?) internal combustion engine could fairly easily be converted into another kind of hybrid, an "external combustion - internal evaporisation" engine. I've been thinking about whether the (alleged?) greater efficiency of continuous combustion in comparison to intermittent would make such a conversion worthwhile. (Just one more of the many half-baked inventions bopping around in my head. :-)
• ramoz 4y ago
  Software is never done - it creates new paradigms we adapt to. "Complex Software" is a fallacy. The problem is organizational structure and IT culture around software. People get invested in software, their code, their favorite dev community... they get comfortable... they become biased and influenced as they learn (not pragmatic)... they enter cycles of doom largely centered around conways law & organizational complexity. Scaling any organization is a people/business/cultural problem. E.g. look at React... how tf did it develop into the hook mess. Early versions we so amazing and simple. A react app/oss component that is 2y old now is garbage. In contrast, look at how Angular has developed, or look at Golang, or Kubernetes (yes really) as a modern examples. People are struggling with AI today because python sucks at scale - but they suck at engineering real solutions because they aren't culturally adaptive. Im no AI/data scientist, but our models running in production on Golang, parallelized across K8s, and the data processing our "yesterday's devops engineers" are now doing is working out just fine. "Software Engineering" was a mistake. Industry never adapted from the physical personality of the word... building bridges and building software are not the same thing. Edit: grammar/phrasing fix, end of P2.
• d--b 4y ago
  Where I work, we kind of organically setup two different teams: the ones who move ahead, go fast and make new stuff, and the ones who stay behind slowly cleaning the mess the first ones did years ago. Ideally you rotate people between the two teams. And you adjust the number of people when the ones ahead create too much to clean up. The problem with complexity is that it doesn’t come out of nowhere, it mostly comes from changing requirements. So it inevitably is always cheaper to hack a system to do something it was almost meant to do than to figure out where in the original design things need to change for the new stuff to be supported.
• psychoslave 4y ago
  "Can we do without?" is to my mind the single most precious question from an engineering point of view. “It seems that perfection is attained not when there is nothing more to add, but when there is nothing more to remove.” Antoine de Saint Exupéry
• 1vuio0pswjnm7 4y ago
  Suggested follow-up blog entry: "The survival of simple software."
• waterproof 4y ago
  One great tool for fighting complexity is to Always Consider Option Zero: whenever comparing possible solutions to a problem, consider the option (Option Zero) where you simply don’t solve the problem. It’s amazing how much it changes the conversation when you stack up the pros and cons of the status quo, against the pros and cons of each solution. Quite often you realize the best path is much simpler than you thought. It’s not a panacea, but it really does help slow the growth of your complexity.
  • m25n 4y ago
    I’m definitely gonna try this out. Thanks for bringing it up!
  • stuven 4y ago
    It always feels like a huge win to successfully make the case _not_ to build something we don't need. Unfortunately it's hard to make the case for such work in performance reviews.
• m25n 4y ago
  I’ve reached an interesting point in my career. I started out truly in love with “sitting around and dreaming up new architectural diagrams that can comfortably sit inside [my head].” However, I don’t derive any pleasure from that anymore. Possibly because I’ve had to face the hard reality of what happens when the architecture “outgrow[s] the size of any one person’s head.” I’ve come to be very interested in two things: figuring out how to delay that as long as possible and what to do when you can’t delay anymore. The first seems to be bringing in as little software (in any form) as possible. Dig and dig to find the actual problem and then solve that using as few tools and abstractions that the team can possibly tolerate. The second seems to be about process and Conway’s law. Figure out communication structures to distribute the management of complexity.
• menotyou 4y ago
  The article refers to this kind of complexity: [0] <https://twitter.com/internetofshit/status/986006653605687296/photo/1 https://twitter.com/internetofshit/status/986006653605687296...>
• pketh 4y ago
  A counter example from societies and software that seems to grow and scale well are biological systems like tissues. Maybe we could design better software and better teams using patterns from biology instead of the romans of old.
  • infinite8s 4y ago
    Also the idea of architecture (and discardable scaffolding) being important. To build a roman arch you need scaffolding in place until you can drop in the keystone. To build a (mammalian) body you need scaffolding in the form of a womb until the body is self-sufficient.' There's also the matter of different types of architecture at different levels of scale. For cellular biology the foundational properties are statistical mechanics and physical chemistry. What are the foundational properties of computation?
• leonroy 4y ago
  I think as software is growing increasingly complex (and it only grows more complex) it's reaching the level where it will become difficult for a human brain to fully grasp the system. We can only keep going back to the drawing board so much before the cost of rebuilding the system becomes prohibitive. What do I see instead happening? I think tools like Github Copilot which assist user's writing code will become ever more sophisticated and broad in their application. That they will not only perform simple bug fixes or write blocks of repetitive code, but even automatically build integrations with third party systems with just an API specification, refactor code to make it cleaner, less repetitive and more decoupled and be able to break apart complex monolithic code bases into smaller libraries or services. Of course building such a system is a complex undertaking in its own right, but much like how a compiler eventually eaches a point where it can compile its own binary so too such a system might reach a point where it can extend and improve itself taking over from its human creators.
• mbrodersen 4y ago
  Software complexity is 95% a software developer problem. That’s why 10x developers exist. All they have to do is to not add complexity while adding features and they are already 10x more productive long term than the average developer who is busy adding more dependencies to “fix” problems with the dependencies they added last week.