11 ms·
Blah bullshit blah. The reason software is so complex now is that our expectations as users is orders of magnitude higher than it was "in the good old days".
by blunte 9y ago
Blah bullshit blah.
The reason software is so complex now is that our expectations as users is orders of magnitude higher than it was "in the good old days". I'm old enough to remember the good old days, so I can speak with a little authority here.
The Pareto principle may not be exactly accurate, but it describes many situations quite well. And in software, it fits very well. 80% of the requirements take about 20% of the code (and complexity). It's the edge and special cases that add the bulk of the work.
When a program just did one useful thing, we humans would do our human thing and integrate several different programs with our manual effort. If an edge case appeared, we didn't even identify it as an edge case - we're built for handling edge cases! We just made the minor _human_ judgement and fixed the data and pushed it into the next program.
Software is complex because we're trying to make it replace more human activity. And beyond the core functionality, human activity is all about applying human judgement. If anyone is familiar with people training people on a particular task, particularly the quaint concept of apprenticeship, it involves teaching and showing and mentoring until a student has enough knowledge AND wisdom to get the job done.
So to boil it down, modern software is primarily complex because it attempts to replicate human judgement and wisdom.
That software complexity is not killing us. It may be a pain in the ass, but if anyone can remember what it was like before we had these tools, I would dare to say that it's still a whole lot better than before computers.
- dismissed 9y agoSure, I agree that things have changed. Despite this, outright dismissal of the premise that complexity is hurting us is unjustified and actually moves the conversation in an unproductive direction.
- emodendroket 9y agoI guess my problem with that claim is that, like, there are process issues, but software is mostly complex because the problem it's solving is complex, and "simplifying" past a certain point means actually not addressing the problem at hand.
- cookiecaper 9y agoThe author's terms are not really well-defined. From the text of the article (which, admittedly, I only skimmed), it appears that he is talking about complexity for its own sake, in the form of trying to force a project to use microservices or all sorts of mumbo-jumbo that is not actually needed to solve the problem at hand. In essence, the argument is against over-engineering, and that is a problem that causes us a lot of grief.
- emodendroket 9y agoWell, alright, but without a way to measure one man's overengineering is another's appropriate design.
- cookiecaper 9y agoI mean, yeah, there is ultimately a judgment call involved. The problem is that many people make judgment calls that are clearly biased toward complexity, with vague non-technical justifications like "This will be useful when we get 10x bigger", "<Last-years-buzzword> is outdated technology", and "This is based on a whitepaper from Google." Unless the project is explicitly intended for experimentation, these are not good justifications! If you're using something like Docker or Kubernetes, good justifications are like "Docker replaces a bunch of scripts we used to have to use to deploy our compiled binaries over a fleet of servers" and "Kubernetes really helped us measure and manage the shared hosting environment that runs our lightweight services". How often do you hear these justifications, or others that refer to actual improvements in the technical processes? Usually it's just "containers are really cool d00d" or "We spend way too much on AWS, and since the concept of not using AWS is unfathomable to us, we are collapsing everything down to 8 instances that run Kubernetes and spending tens of thousands of dollars on the engineering overhaul to support that instead of just provisioning cheaper servers."
- emodendroket 9y agoI see. Yes, that's a fair point that I think maybe I've gotten a little numb to because the place I work at kind of has the opposite bias (i.e., "we got burned in the past so now you have to fight tooth and nail to be allowed to introduce tools that would make work on the project significantly more productive").
- jetheredge 9y agoAuthor here... we talk a lot at our company about essential complexity and incidental/accidental complexity. Software is complex, absolutely. It is complex because business and reality are complex. I think that is inescapable. What we can try to escape though is the incidental complexity that we add to our systems while we are following trends or chasing down shiny new tools.
- blunte 9y agoYou get an outcome appropriate for the risk assumed. The capitalist business world values speed of development over reduction of risk. That's why at many companies, whatever tools and languages allow you to get an MVP out the door before a competitor are not merely accepted but are often celebrated and lauded. When risk really matters, such as the commercial flight industry, the pace moves much slower. It takes comparatively forever for new features to reach the market because the cost of mistakes is so significant (human life). If business behaved the way regulated commercial flight behaved, we wouldn't be having this conversation. Pointing to open source or expressive languages as a source of complexity is, obviously to me, a red herring. The failure is our lack of rigorous engineering practices - those practices we gave up in the interest of beating our competitors to market. You don't even have to look very far back in time to recall when software was expected to be correct upon delivery. Nowadays, particularly with online patching, we are totally accustomed to bugs and frequent fixes. Only a decade ago this was unheard of. The cost of reprinting manuals, burning new CDs (or gasp stacks of floppies) and shipping those out to all customers was so prohibitive that it was worth delaying release until all important features were in place and a full series of tests had been performed. It's our behavior and expectations that got us where we are, not the tools. And probably there's no going back. This is the age of instant, so we'll do our best to adapt. Ironically, the human who used to have to BE the integration between his/her software tools is now the monitor making sure that the automated integrations are managed properly when an unexpected situation arises. And now we've built new layers of logging tools to help us spot these situations. But we're still stuck managing the processes...
- majormajor 9y ago> That's why at many companies, whatever tools and languages allow you to get an MVP out the door before a competitor are not merely accepted but are often celebrated and lauded. The linked article has some points I agree strongly with, but I disagree in its approach to tools. An MVP that's out the door as soon as possible is unlikely to have the sort of incidental complexity tech debt that kills projects. It can't, because the focus was on shipping. A quick prototype for an MVP is great, get stuff out the door as soon as possible. If your app needs some javascript, write it however lets you get it out the door the quickest - if that means importing someone else's big framework, fine. I think the secret is more akin to keep _your own code_ as minimal as possible. Your infrastructure and stack should respect the M part of MVP just as much as the product should. Cause I rarely see true MVPs. I see "ok in order to scale to a million users in the first year, we're going to need x, y, and z on top of...". And then you're not just importing someone else's JS code and writing your business logic on top. Now you're building several systems that have to work together correctly even as they get modified independently in the future, and so in the process you're encoding a bunch of assumptions about your business that the MVP hasn't actually proved out yet! And that's tech debt of the deadly sort. We're far away from having tools that will prevent a dev from building something overly complicated if they really really want to. Instead, we have to prevent it by always asking the hard questions "do we really need this right now" and not being afraid to look like the fun police.
- ravenstine 9y agoI disagree. Users, in reality, don't care about all the crap that stakeholders and some developers care about. They don't care about Material design, fancy animations, beautiful buttons, "elegant" code,a ton of niche features, mobile apps(no I'm not kidding), your annoying push notifications, your fancy menu bars that you also fix to the top of the screen for whatever reason, your autoplay videos, your little "delightful" asides that interrupt the content, etc. They want something that works without much effort or worry that the software will fail, and that doesn't necessarily have to do with how fancy or feature rich an app is. Craigslist and the Quartz app are just a few examples of what I'm talking about. Nobody except maybe some stakeholders ever asked for those things to be more fancy than they are. They are simple and they work. The end. Unfortunately, we are in an era where everyone wants to be like Google, or at least thinks they need to. I do not believe that the rest of the world would be worse off if they stopped spending time making "production ready" software that visually competes with the Big 5. Software becomes complex because we allow it to become complex. Just because you can add another feature doesn't mean that you now have more value for a finite amount of pay for X developer time. Companies seem often blind to continual costs for support that go up every time each time the solution is to add a new feature or software package for a small segment of the customer base or the company itself. Whomever is in charge should acquiesce less often. But everyone wants to be a wizard.
- blunte 9y agoI think either you are not understanding me, or I'm completely misunderstanding you. What you're saying seems orthogonal to my arguments...
- ravenstine 9y agoI'm disagreeing with your point that users are expecting more of software. Their expectations have changed in some ways, but not really in any that makes software significantly more complex. To shorten my last post, it's poor development practices and a lack of foresight from upper management that makes software too complicated. You could make an interface that looks like it's from the mid 2000's(garbage by today's standards) and people would still use it if it works well and is reliable. Look at HN. Do we need more from it than it already provides? If it was up to the average upper management, the backend would need constant retooling for every feature request from different departments, 12 forms of analytics, newsletter signup forms in between posts, job alerts, a mobile app with a nag message on mobile pages, single sign on from Facebook and Google, share buttons, a waffle iron, etc. None of which the actual users and even the developers ever wanted. EDIT: Some people might point to Slack as an exception to my argument. Indeed, Slack looked very nice compared to the competition when it first came out. But what made it a killer app was that it took the best of the IRC experience and made it a universal experience between devices and browsers. Plus it can use a company's SSO. When it first came out, it was also not very complicated. I remember when it didn't have reactions or threads, but it was a breath of fresh air after having come from using things like HipChat and Gchat, both of which were pretty dreadful.
- avinium 9y ago> The reason software is so complex now is that our expectations as users is orders of magnitude higher than it was "in the good old days". I'm old enough to remember the good old days, so I can speak with a little authority here. Eh, maybe I've just turned into an old curmudgeon, but I very rarely fire up a "new" application and think "wow, this is so much better than the old days". At least in terms of design, I find that most applications held up as some "gold standard" - iOS, Slack, Discord - are incredibly intrusive and obnoxious (not to mention unstable). Applications used to be a lot simpler and less opinionated. They did one thing, and they did it very well. I suspect it's a case of too many cooks in the kitchen, too much money and teams that are too large.
- digi_owl 9y agohear hear.
- rrobukef 9y agoYet I don't have driver issues, codec issues, bluescreens anymore. I don't need to fail compiling a kernel. I can watch a movie without downloading it, I can store a high quality picture on a floppy disc, maybe two. Computers entertainment has changed: the internet works (no more unsupported browsers), gaming works (rarely hard system requirements like specific video cards). Business works too (no more emailing spreadsheets, accounting software works, shared calendars) Applications did one thing, they didn't always do it well.
- peoplewindow 9y agoYes, operating systems got a lot better. Arguably apps got worse. Things that are now routine for apps that used to be considered unacceptable: - Don't start if you're offline - Limited or crap drag/drop. - No app icon or start menu integration. - File type associations, forget about it. - Generic object linking/embedding, gone. - Scriptability, gone. A well written Windows 95 app would work offline, integrate with the Explorer, have an icon, might be able to embed itself into Word documents, and probably exported an API via DCOM. Modern apps? Uh, well.... they do similar things but at the same speed at best, quite possibly slower. They routinely barf if they can't reach the internet (e.g. any web app). Every resource dimension is 10x larger. Quite likely, I can't actually save stuff to a file. They crash less, but experience other kinds of flakyness more. And they probably find scrolling through any kind of non-trivial dataset a monumental challenge. Slack in particular is terribly flaky.
- computerex 9y agoI don't understand what you are saying. Can you possibly give a concrete example of software trying to replicate human judgement and wisdom and how that's unnecessary complexity?
- zaphoyd 9y agoThe parent is not saying that replicating human judgement is unnecessary complexity, quite the opposite, that our expectations for software replicating human judgement are making it complicated and as a result of that complexity we have "better" software. Concrete example: Photo organizing program Old style simple, non-complicated: Photos exist as files on a single computer that are manually placed into albums by people, manually shared in curated batches by printing or emailing. New style complicated, replicates human judgement: Photos exist in the cloud shared across devices. AI software takes date, location, face & object detection to create automated albums, reject the photos that people aren't looking, etc.
- hcarvalhoalves 9y agoCouldn't agree more. Not only we have old businesses being entirely automated, automation itself is also opening up completely new markets. Software complexity is exploding probably at the same rate it's absorbing the complexity that used to be scattered around companies in a less formalized, less cost-efficient way, across people and processes, w/ all the challenges that this implies (communication, correctness, monitoring, maintenance, etc).
- deleted 9y ago[deleted]
- jdmichal 9y agoI heard this interesting tidbit a while ago that fits your sentiments perfectly. Can't remember who to credit it to, unfortunately. Software is really about codifying process -- pun completely intended. What happens is some slick new software comes along that implements that happy 80% path on the process, and does it quickly and efficiently. Then business complains about the 20% falling into the gap, and starts adding features to reduce that, thereby slowing down the processing. This keeps going, until someone has an idea and implements slick new software that implements that happy 80% path on the process, and does it quickly and efficiently...
- rrobukef 9y agoOr people learn and see that they can make a happy path of 90%. Or the business removes an implemented unhappy path of 5% that fails 50% of the time. Or the business introduces a new unhappy path to go with the times or times just change and the happy path is now only 50% of the cases. You're completely correct. Software needs are easy and hard, and times change.
- deleted 9y ago[deleted]
- wonderwonder 9y agoI think you nailed it. Software today is expected to handle all cases and there is not supposed to be any built in manual work arounds. Eliminating the need for manual work arounds takes 80% of the work. Simple example, company I work for sells both physical and digital products. They used to have separate systems to handle each case and everything worked great. We are now building a new system to handle both at the same time. Issue is that due to US tax codes, there is a different tax rate used for a physically shipped product VS a digitally delivered one. The system has to account for this and no one ever considered this in the design or planning because they never had to deal with this concept before because each system just had to deal with one tax rate per order before. If in the rare case a customer purchases both types of products in a single order, both tax rates have to be accounted for. Simple work around to save a bunch of work is just have the system limited to only digital or physical per order. This is not a optimal solution so complexity increases by a good deal, especially when refunds, reporting etc. is factored in. This is not necessarily a bad thing but it is the reason we have such large code bases now, because we need to handle these edge cases without manual human intervention. Our systems have to be both a jack and master of all trades.
- lr4444lr 9y agoI agree completely, and would add further than more lay people than ever are using computers to do even more cotidian but critical tasks than ever, involving commerce, personal information, their digital memories, love lives, etc. Not only do individual programs do more, but more of life itself is done through computers and by people who are not specially trained to operate each system. This is also why fears of devs writing themselves out of a job is a laughable. The easier to use and more powerful we make our software for public consumption, the more opprtunities to optimize our lives the public demands. It's hard work, but the future is bright.
- tboyd47 9y agoI have to this day never heard the words "single page app" uttered from anyone but developers. I'm pretty sure we are doing it to ourselves at this point.
- eropple 9y agoI regularly hear "faster" or "snappier" or "more responsive." Which was my original impetus to learn React: that full page loads and page blanks weren't cutting it anymore. If you have a better answer than single-page application tooling for this that is not "well, print a huge document, people will totally scroll to find things and wait for postbacks instead of expecting a navigable drill-down interface that responds to changes as they are made," I would genuinely be interested in hearing about it.
- tboyd47 9y agoI've worked on more than one SPA and it's always slower than a traditional site except in very narrowly defined ways. YMMV, and some people love it.
- eropple 9y agoI don't intend to jump on you, but this claim strikes me as super suspect. There are very few situations outside of the pathological case where the process of querying for data, rendering server-side HTML, and performing a full page reload is going to be faster than local-side processing of a (much smaller) web response based on that same data query followed by a partial DOM rebuild. And that's not even taking into account perceptive effects; if you sit in a UX lab, having a page flash during rendering makes people think it feels slower. (I have done this test--adding a hook to the end of a web request to hide spinners and draw a white "flash" over a SPA--and the subjective "snappiness" score dropped from an average of 8 to an average of 5.) Nor is it taking into account slow connections and degraded functionality during interruptions, but I'll cut you some slack on that one just because so few people pay attention to it in the first place. I'd love for you to explain more, but I'm gonna register ahead of time that I kind of expect a "doctor, it hurts when I do this" explanation.
- alkonaut 9y agoExactly this. Software is complex because users have complex needs, or more specifically: users have varying needs and the union of those needs is complex. Also: quality and dev cost are always factored in and complexity wins. As a developer I always argue for the simple, clean, maintainable, performant solution - but the business need of the customer is such that they’d nearly always rather have the slow, expensive, buggy and bloated solution (that is - given the choice at fixed cost). Even after pressuring customers on this - the answer in my experience is always “yes we accept poor performance, poor reliability so long as the features are there. Make it complex, we’ll pay for it with money and we’ll pay for it with accepting poor quality. Go for 10 features working 80% rather than 3 features working perfectly”. And the reason they want this? It’s because their existing apps (from the ”good old days”) might be fast and so simple it’s superficially bug free but it just doesn’t solve their problem. They have to do manual work instead and it’s costing them. So in the end we are making apps with complexity so high that making them properly (I.e with the kind of thought and care that goes into e.g a compiler) would take at least ten times as long. But customers would reject that product either on cost or lack of features or both.