9 ms·
Long Term Software Development
- cjblomqvist 2y agoWould be awesome if we had some actual proof (science) backing this up. Anyone?
- cratermoon 2y agoLehman, M. M. “Laws of Software Evolution Revisited.” In Software Process Technology, edited by Carlo Montangero, 108–24. Berlin, Heidelberg: Springer Berlin Heidelberg, 1996. Lehman, M.M. “On Understanding Laws, Evolution, and Conservation in the Large-Program Life Cycle.” Journal of Systems and Software 1 (1979): 213–21. https://doi.org/10.1016/0164-1212(79)90022-0 https://doi.org/10.1016/0164-1212(79)90022-0. ———. “Programs, Life Cycles, and Laws of Software Evolution.” Proceedings of the IEEE 68, no. 9 (September 1980): 1060–76. https://doi.org/10.1109/PROC.1980.11805 https://doi.org/10.1109/PROC.1980.11805. Lehman, M.M., J.F. Ramil, P.D. Wernick, D.E. Perry, and W.M. Turski. “Metrics and Laws of Software Evolution-the Nineties View.” In Proceedings Fourth International Software Metrics Symposium, 20–32, 1997. https://doi.org/10.1109/METRIC.1997.637156 https://doi.org/10.1109/METRIC.1997.637156.
- ahubert 2y agoAdded these references to https://berthub.eu/articles/posts/on-long-term-software-development/#some-books https://berthub.eu/articles/posts/on-long-term-software-deve..., thanks!
- elric 2y agoIf you're interested in some scientific background to Software Engineering, I can recommend the book "Making Software" (O'reilly) by Andy Oram & Greg Wilson. It's a bit old now, but addresses and challenges many common beliefs about Software Engineering. https://www.oreilly.com/library/view/making-software/9780596808310/ https://www.oreilly.com/library/view/making-software/9780596...
- MH15 2y agoGood vibe-check on my day to day work in web tech. Sometimes I wonder how web deployments would work if we still deployed quarterly, yearly, etc.
- withinboredom 2y agoWhere I work now basically does quarterly deployments. We service enterprises that generally outsource their development work. So we deploy pretty regularly to a staging environment and deliver documentation weeks/months before we actually deploy it. The exception to that rule is for hotfixes, which is also extremely rare. Like 2-5 times per year.
- ChrisMarshallNY 2y ago> I’ve personally been burned on Python by the last bullet point where one of the dependencies required version 3.14 or less of module such and such, and another dependency needed 3.15 or higher. This is what people used to call “DLL Hell,” in Microsoft Windows. COM was supposed to fix that, but I don’t think it worked especially well.
- NoZZz 2y agoIt required the re-implementation of stable interfaces as the components updated in version number; not everyone did that. I guess. A case of good plan, lazy execution.
- a1o 2y agoWhat I found mostly curious is I didn't even knew phyton 3.15 had already released
- lambda 2y agoThe example was talking about versions of another package that both of the given packages depend on. Not Python itself. The version numbers were made up.
- db48x 2y agoRust fixes the problem at the language level by allowing your dependencies to use different versions of their dependencies. It can be a little rough on the size of your binaries, but you can end up with as many different versions of the common dependencies as you need.
- ChrisMarshallNY 2y agoThe issue is probably the same as with COM. COM introduced versioned “packages,” inside of the library, but that meant that library providers needed to package up multiple versions, as part of their CD system. In practice, a lot of orgs just stopped including deprecated versions.
- jcelerier 2y agoWe went with Qt, CMake and modern C++ for https://ossia.io https://ossia.io in 2013 knowing that it would be a long term effort for an extensively extensible linux/mac/windows desktop software aiming to do real-time audio, visuals and networking and so far this "classic" stack keeps on giving and allowing me to ship regular features and improvements, here's to the next ten years :) in the meantime I can't count how many techs and frameworks I've seen come and go but these are here to stay.
- withinboredom 2y agoI worked at a place that required vendoring dependencies and had to be code reviewed just like it was your own. In fact, we were responsible for fixing any bugs in them ourselves. Sometimes this was just a matter of opening an issue, sometimes it required backporting the fix to our version, and sometimes it meant we took over a fork because the dependency was no longer maintained. There were too many cases where I looked at the PR preview to do my own code review and decided to just write it myself. There is a ton of shitty code out there. More than you'd expect.
- db48x 2y agoThat is the only sensible way, in the end. Ultimately you have to fix the bug in your product, even if the bug comes from a library. Users won’t generally like to hear you make excuses of that kind.
- BobbyTables2 2y agoI’m amazed at the “big name” open source projects that effectively accept PR requests without review nor testing. Open-source feels like a cheap buffet at times…
- dartos 2y agoI mean… you still pay at a cheap buffet.
- bigfatkitten 2y agoYou're either paying at the counter or you're paying in the bathroom.
- kidneystereotyp 2y ago[dead]
- dartos 2y agoAt a cheap buffet, probably both
- rramadass 2y agoRelevant: Lehman's Laws of Software Evolution - https://en.wikipedia.org/wiki/Lehman%27s_laws_of_software_evolution https://en.wikipedia.org/wiki/Lehman%27s_laws_of_software_ev... Studying the laws of software evolution in a long-lived FLOSS project - https://pmc.ncbi.nlm.nih.gov/articles/PMC4375964/ https://pmc.ncbi.nlm.nih.gov/articles/PMC4375964/
- rramadass 2y agoSoftware Evolution - https://en.wikipedia.org/wiki/Software_evolution https://en.wikipedia.org/wiki/Software_evolution
- ahubert 2y agoAdded this to https://berthub.eu/articles/posts/on-long-term-software-development/#some-books https://berthub.eu/articles/posts/on-long-term-software-deve..., thanks!
- dijksterhuis 2y ago> Keep it simple. Simpler than that. Yes, even simpler. You can always add the complexity later if needed! This. A thousand times this. Be grug brained, not big brained.
- BobbyTables2 2y agoBut if we don’t require an entire Kubernetes cluster with dozens of beefy nodes to deploy and manage a trivial app, how can we have a scalable enterprise application? /s
- n_ary 2y agoAnd of course, if you didn’t touch all of cloud stuff, how will you explain to the junior randomly promoted to tech lead(sometimes CTO!) that you are worth hiring when your current corp decides to lay you off? /s
- maccard 2y agoI know you’re kidding but a container is one of the simplest easiest most portable ways to deploy a web app.
- sjamaan 2y agoOnly if you consider web apps have become so complex that it requires a container to wrap up the complexity.
- maccard 2y agoDisagree. If I have a container with pho, I can run it in any hosting platform in any underlying architecture without needing to worry if my host has the right version underneath. That’s a huge win. I can run it on AWS, on digital ocean for $5/mo , on lambda, or GCR. Completely portable. And I can run the exact same setup at home.
- 2y ago
- jandrewrogers 2y agoSomething that has also worked well for me historically is aggressively updating tool chains as an integral part of the development process. I've seen far too many companies de-prioritize upgrading their tool chains until it got to the point where they were so out-of-date that things broke through decay and obsolescence, and often had security vulnerabilities with no easy path toward remedying them. At this point, it usually requires heroic efforts to bring everything up to a vaguely modern standard. I've adopted a policy that for every significant compiler or build system release, we create a branch that verifies a good build. If it doesn't build without warnings or errors, or fails in testing, that is treated the same as a bug in the code to be dealt with ASAP. Prioritizing this also tends to enforce good build configuration hygiene and automation. Not being able to do this is a bit of an architecture/code smell anyway. This also makes it much easier to incrementally modernize and refactor the code base with new language features -- the tool chain supports the latest and greatest even if the code base doesn't yet. I also find it difficult to rationalize third-party dependencies these days, they almost all become disappointments over the long-term. The benefits rarely justify the risks. Anecdotally, this policy has not materially reduced development velocity; the scope of useful functionality can usually be replaced with a small amount of code that has good mechanical sympathy with the rest of the code base because it was designed with the code base in mind. I've found in many cases you'll write just as much code trying to interface a third-party library. That said, for new projects it is often useful to use a few third-party dependencies to plug holes with a clear plan to replace them in the near- to mid-term. Stopgap dependencies have been a reasonable practice in my experience.
- wutwutwat 2y ago> I also find it difficult to rationalize third-party dependencies these days Your home grown auth code doesn’t have the minds and the scrutiny of security experts like the go-to third party auth lib everyone uses does, nor does it have the free pen testing, security fixes, maintenance, optimizations, refactors, ever collaborative maintainers, or corporate sponsorships. Auth is a solved problem by minds much smarter than my own. I’m glad to outsource things like that to third party deps.
- 2y ago
- kmoser 2y agoI agree 100% with the section on documenting your system and code. It's sobering to consider how many HN posts about software development contain comments to the contrary, indicating that documentation (and heavily commenting your code) is useless. It seems the longer you've been developing software, and/or the bigger the project, the more you become a fan of documentation.
- OtomotO 2y agoDocumentation can be utterly useless, if taken to the extreme. Like e.g. in Java it was common to have a comment on a simple (!) setter, telling you "this sets X" No shit, captain obvious! Never would have guessed that. I like a combination of "literate programming" (in a light form at least), that leads to readable, self explanatory code that's still fast and well (not prematurely) optimized. But reading docs about the most trivial things: Brrr!
- kmoser 2y agoI'll gladly deal with a few (or even many) useless "this sets X" comments if they come along with helpful comments for other non-trivial methods.
- medo-bear 2y ago> Like e.g. in Java it was common to have a comment on a simple (!) setter, telling you "this sets X" Given the prevelance of mutating functions in Java this is welcome
- OtomotO 2y agoNo, it's not. Because it doesn't add anything. Just document if something is mutating and what it mutates. Or simply name your functions accordingly. Like of your setter is doing a HTTP call, maybe name it so that it's obvious.
- elric 2y ago> Like e.g. in Java it was common to have a comment on a simple (!) setter, telling you "this sets X" This has never been common in any sensible environment. Only on projects with project leads who had zero clue about anything, and with silly catch-all rules such as "EVERY METHOD MUST BE DOCUMENTED!". This predictably leads to shitty documentation. /** * Sets foo to <code>foo</code> * @param foo new foo value */ public void setFoo(int foo) { this.foo = foo; } No one sensible does this. Some IDEs might auto generate this garbage. In which case that should be disabled.
- sroerick 2y agoComing out of python land and spending a bit of time working in Emacs Lisp was kind of a breath of fresh air. Working with the org roam library, I’d see a lot of complaints about how it hasn’t been updated in two years. This was of course, a feature and not a bug. I really did feel a prevailing sense of calm that libraries would not drop out from under me, that I was just running basically all my own code or borrowed code, and if it works, it would continue to work. This contrasted wonderfully with my experiments with Gatsby and Node, which I foolishly deployed to client websites not knowing that as a result I would be doomed to years of deployments breaking and nightmare library updates. Of course, the trade off here is that with the exception of a few wonderful libraries like org-roam, basically everything, my own code included, shipped basically broken out of the box
- cpeterso 2y agoOn this subject, I recommend Marianne Bellotti‘s “Kill It with Fire: Manage Aging Computer Systems (and Future Proof Modern Ones)”. She has managed the maintenance and modernization of large legacy systems at the United States Digital Service. https://nostarch.com/kill-it-fire https://nostarch.com/kill-it-fire
- ahubert 2y agoAdded this book to https://berthub.eu/articles/posts/on-long-term-software-development/#some-books https://berthub.eu/articles/posts/on-long-term-software-deve..., thanks!
- mrbluecoat 2y ago> Write boring simple code. Even more simple than that. Even more boring. Write super boring code. Write naive but obvious code. That's the rub, isn't it? Boring code tends to bore developers so we stop doing it, regret it, return to boredom, only to find ourselves listless once again..
- sam_lowry_ 2y agoRemember the Google 20% rule? There are ways to deal with boredom without jeopardizing business.
- int_19h 2y agoGoogle effectively killed it off though as part of becoming a "proper enterprise".
- MantisShrimp90 2y agoI think as you go on it becomes a out being sure that you are solving problems in the simplest way possible. Chances are even with the simplest solution possible you will still need to do some creative stuff. However, too many programmers in my experience (including myself) can frett about using tools and features that are cool little puzzles, but don't really contribute to the core solution and can in the worst case even make less manageable code. As in all things, it's about finding the balance for the context. Let your personal projects be where you play with new tools while you keep solutions for your job dead simple due to all the job-related reasons to do that.
- bruce511 2y ago>> There are many databases around, but most of them work more or less the same. If you built on MySQL and you come to your senses and shift to something else Im spending a bit of time on databases lately, mostly SQLite, PostgreSQL and MS Sql Server. With a bit of Firebird. I never went down the MySql path, but statements like the above intrigue me (because I like to learn from others experience. ) So my question is - why not MySql? (And does the same argument apply to MariaDB?
- brabel 2y agoMySQL is like the JS of databases, it does some truly bizarre shit in order to try to "help" you. https://sql-info.de/mysql/gotchas.html https://sql-info.de/mysql/gotchas.html
- bruce511 2y agothanks
- sjamaan 2y agoMySQL doesn't care about your data. There are many places where it silently truncates, "helpfully" tries to "do the right thing" and so on. It used to have very impoverished data types. At least that has been mostly amended, but AFAIK you still can't make an index on a TEXT column without a limit. The "utf8" text type, despite the name, can't hold all of unicode. You need to use utf8mb4. This is merely one example of the general attitude. Another fine example: you can't use mysql_escape_string() in the C API to safely escape strings. Instead, use mysql_real_escape_string(). Instead of hiding the unsafe version, PHP exposes both these functions as-is. PHP and MySQL are truly a match made in hell. The MySQL issue queue is also littered with unaddressed corruption issues which have been open for many years. MariaDB is just a fork, most criticisms to MySQL apply to it as well.
- exac 2y ago> Ever tighter security standards. JavaScript over http is dying, for example. Bert, I can assure you that this is not the case.
- ahubert 2y agoThere are already some practical restrictions on non-127.0.0.1 http cookies. I fully expect more of this in the future. And a good thing too.
- vaylian 2y ago> Bert, I can assure you that this is not the case. Can you give some examples? There are still legacy websites that don't support HTTPS. But these are by definition a dying breed.
- suzzer99 2y ago> Write super boring code. Write naive but obvious code. “Premature optimization is the root of all evil”. If it was too simple, you can always make it more complex later. And that moment might never arrive. Don’t write clever code until you simply have to. You will not ever regret writing code that was simple. Hang this in the Louvre.
- globular-toast 2y agoBut be very careful not to mistake easy for simple. Too many people think that simple means easy when in fact easy often leads to the most complex code. I don't like that clever is thrown in here. Your software architecture should be clever. It should be clever in that it allows you to write simple code. Three simple functions in three different layers of an architecture is better than one function that couples various layers. See Rich Hickey for the difference between simple and easy: https://m.youtube.com/watch?v=SxdOUGdseq4 https://m.youtube.com/watch?v=SxdOUGdseq4
- suzzer99 2y agoYeah for sure there's an art to it that only comes with experience. But you better be 100% sure you need those three layers. My first big project as a developer was on an over-designed reporting/query-builder application that had like 7 layers of of abstraction between the browser and the database - stuff like just in case we ever want to switch from Oracle to Sybase, unnecessary connection factories, Java controller classes for every type of html form field, etc (still have no idea why we needed a back-end CheckboxController). I ended up maintaining that thing and it was a great lesson in premature optimization. The bottom 3 or 4 layers remained a grey box that I did everything to avoid touching. Just changing the front-end query-builder flow from a series of forms to a flat panel that could do everything in one step involved ripping out half the back-end code. All those layers of abstraction and none of them considered a form-flow change, which is exactly why I never assume anything now. It's so much easier to add a layer than remove an unneeded one, which can be practically impossible on a complex app.
- globular-toast 2y ago
- kidneystereotyp 2y ago[dead]
- cess11 2y agoTen years is not a long period, unless you're a kid or adolescent. Aim for a century.
- int_19h 2y agoSoftware engineering hasn't even been around for a century, so we don't know how to write software that lasts that long. We can only make educated guesses. But, in any case, given the current state of things, 10 years is already a massive improvement.
- cess11 2y agoWe don't? There are computer systems running since the seventies in areas like banking and aerospace, why wouldn't they last another fifty years? Most of my professional work over the last fifteen years has been on applications that were ten years or older when I was recruited. Currently I do work in public sector digital archiving, where it's common to retire systems that are still being updated and have been going since the nineties. This kind of long term storage aims for century or more.
- revskill 2y agoTo me it is testable and debuggable.
- ChrisMarshallNY 2y agoI’ve found the best way to write software that lasts, is to be “boring.” Avoid the buzzwords, stick to the language basics. Do things by hand. Defintely avoid dependencies. I’ve written libraries in C, that were still in use, 25 years later.
- mentalgear 2y agoKISS
- dtrav 2y agoThis post starts with " Software that controls (nuclear) power plants, elections, pacemakers, airplanes, bridges, heavy machinery " and proceeds to list pracitises you should apply to, say, shopify or facebook. Those practises are no where near robust enough for industrial software. Such is the degree of risk adversity that the use cases above can run on out of date hardware and ancient software, and change is deplored in favour of workarounds. Why is that ? Well engineers see risk differently to us - they will attempt to remediate all risk, whereas we will mitigate that risk. That mitigation is the substance of the post, and I have no quibble with that, but to assume that is best practise for my ICD is naive. The author proposes that doing what we do well is sufficient, whereas we need to shift our thinking to a world in which we are in complete control.
- rramadass 2y agoVery good point particularly w.r.t. "remediate vs. mitigate". Do you have any articles/resources/books that you can point to on how it is done for "industrial software"?
- devjab 2y agoI think that it is interesting that testing plays such a part in these plans. I guess it shouldn’t be a surprise after so many years of TDD, Clean Code and whatever nonsense the pseudo industry of “best practices” has been successfully selling. Odd on its own considering software seems to be just as broken as it was 20 years ago despite all these efforts. Anyway, if it was me I would look closer at how NASA build things. Which includes testing, but the key tool for finding actual programming errors was assertions. I’ve worked in medial software for a short while. The key features of development when lives are at stake are, assertions, avoid interpretation and no dynamic memory allocation. You should test things, but your assertions should catch any programming errors without a test/debug suite. You should never use an interpreted language, because that is a head ache you don’t want, but you also shouldn’t parse data like JSON. If you need to send data you send bytecode. You do this because you really don’t want dynamic memory allocation in software that will kill people if it fails. Now a little anecdote. In Denmark we have digital elections, not the voting but the voter registration and the system where each municipality reports the results. These systems run on COBOL and are ancient. There has been multiple attempts to replace them with “modern” software, because the old system can only be run by one private company and that is a monopoly. This is because we privatised the sector 20ish years ago. Anyway, every attempt at replacing it with modern long term software has failed, and a big part of the reason is because people have forgotten how to write code which isn’t infected with all sorts of OOP bullshit. So even the best suppliers and large companies like IBM have failed to make something with the same resilience. It’ll be interesting to see if we’re also going to infect embedded software with these things as the public sector decides to enter and hand out “best practices”.
- rramadass 2y agoGreat points! Regarding assertions what techniques/practices did you guys follow? What did you do when an assertion failed? I myself am a fan of DbC and hence am interested in knowing more about how others use/modify/extend the technique. > If you need to send data you send bytecode. I presume you mean some encoding of binary-to-text; eg. base64 etc. ?
- devjab 2y ago
- liampulles 2y agoAn important learning for me was to not be so concerned with different styles and ways of doing things in the same codebase. Fighting against this is ultimately a big waste of time and results in team fragmentation and "grammar Nazism" IMO. Better to learn to read and understand regardless of the style, and micromanage each other less. Save the sweat for the big stuff. This is something I constantly need to remind myself.
- liampulles 2y ago> One of the easiest hacks for successful software longevity is keeping people around for a decade. That doesn't sound like an easy hack at all, but I can appreciate the value of achieving it. Are there any good books or studies on this subject?
- perlgeek 2y agoI don't think it's too complicated, especially if you start in a culture where people don't job-hop all the time. But, you have to make it your priority, which means: * pay them more than competitors, pay them more than if they changed jobs * actually treat them well, which includes listening to and acting on their feedback * celebrate people with long tenure * give them options to grow inside the company * don't just hire fresh graduates, a 40yo might be tired of job-hopping and still has 20+ years of employment in them * give them long-term incentives, like stock options
- inSenCite 2y agoI sincerely hope nobody is taking advice from linkedIn posts. Software development or otherwise...
- tpoacher 2y ago> Keeping it simple requires periodic refactoring / code deletion The equivalent in medicine is "Always question the established diagnoses / management plan during a new admission". And just like the periodic refactoring mentioned here, it's a lofty idea, but without incentives, investment, and/or infrastructure to support or enforce such reviews, it almost never actually happens in practice while one is busy with the daily grind of things. Which is how you end up with people taking drug A to counteract drug B which was given years ago to alleviate the symptoms of drug C which it turns out you didn't even really need in the first place.
- TZubiri 2y agoI understand the strategic advantage for offline software for elections and nuclear power plants. For security reasons you can't depend on outsiders, and you've got long term budget to support in house teams. However in terms of longevity, in the consumer space and corporate space, third party SaaS is much more stable and reliable than offline software. How much of the data from offline software do you keep from the 2000s or 2010s? You changed computers and the data is gone. Yes, many SaaS providers pull the rug, but usually it's fine and you get a chance to migrate the data? If I had to place a bet on whether the files on my desktop will outlive the files on my Google Drive, I wouldn't take it.
- atoav 2y agoYou are sure about that? The oldest machine I have seen at my University IT was a server with an uptime of 16 years. Judging be the look it was probably 30 years old. I could name you numerous services that came and went within three decades, granted that old machine did something simple (can't remember, probably a NTP server), but I am pretty sure an external service would have been more unreliable throughout that timeframe.
- cesarb 2y ago> If I had to place a bet on whether the files on my desktop will outlive the files on my Google Drive, I wouldn't take it. I have files on my desktop which have lived on it (and its predecessors) for longer than Google Drive existed. Some might be older than Google itself. That's probably not an uncommon case. You say "you changed computers and the data is gone", but it's far too easy to copy your documents folder (or equivalent) from one computer to the next, either directly or through an intermediate burned CD or external disk (which ends up also becoming an informal backup of that old data). Meanwhile, I have already lost access to at least one Google account (the one which was originally my Orkut account); if I had any files on it (instead of just using it to log into Orkut), I would have lost them.
- int_19h 2y agoI would totally take that bet. As far as personal files that I care about go, there are some that predate this whole "cloud" thing even before it was called that. And these days, given how dirt cheap storage is and how fast transfers are, it's even easier to copy data from system to system and to maintain backups. Heck, we have 8-terabyte USB drives these days.
- crabbone 2y agoHere's my controversial idea on one of the aspects mentioned in OP. Here's my approach to dealing with dependencies in Python: * When setting up the project, in the initial stage, install dependencies with pip or conda, whichever was used for the project. * Examine what was installed and figure out why it was needed. * Remove dependencies that are only touched by code paths that aren't executed by the project I'm working on. * Write a script that downloads the thinned out list of dependencies, skipping pip or conda. Unpacks and installs it w/o the help of either. Then, possibly post-process the installation (a bunch of Python packages come with a lot of junk, like unit tests, scripts to call library functions etc.) * Finally, use this script, and on a side install the same combination of packages with pip or conda once every so many months (maybe about half a year) to see if there are any changes. If there are worthwhile changes: incorporate them back into my script. On the face of it, this is more work than writing some configuration file that pip or conda will understand. On the other hand, for so many times I was burned by Python's dependency problems, this approach proved to work better for me in terms of how many times CI was broken, and the time I had to spend investigating CI errors went noticeably down. So, to restate it: I see package management software, at least in Python world, as one of, if not the most important contributors to the failure of ensuring project's longevity. This software prioritizes the ability to fetch and to adjust for the new stuff to the ability to support the old stuff. This software prioritizes information and wishes of the third-party developers over the wishes of the user installing the software. I don't think it does this out of malice. It's probably natural for most people to want new things than to try to preserve the old.
- guytv 2y agoThis is why doing long-term development on Android feels like trying to build a sandcastle during high tide. Google forces you to use their ever-changing, drama-prone OS-level components through their “official” libraries. These libraries evolve faster than a teenager’s TikTok algorithm, so good luck keeping up. Oh, and if you don’t update your app to comply with their shiny new toolchains and runtime? The Play Store will ghost you faster than a bad Tinder match.
- tasuki 2y ago> What might not be a great idea is to have 1600 dependencies in 2024, dependencies which already change at such a rapid clip your code base is effectively a moving target. Has the author not heard about reproducible builds? Why would you care that your dependencies change? > Dependencies... > Drift away, leading to adjustments in your code or, worse, silent changes in behaviour > Shift to new major versions with semantic changes, requiring rewrites on your part > Get abandoned or simply disappear, or start to decay Really, these aren't things that can affect you at all if you have reproducible builds! > Tests are always a good idea, especially if you have many dependencies which shift and drift all the time. Not a thing! If your dependencies "shift" or "drift", you're doing it wrong!
- perlgeek 2y agoEven when you have reproducible builds and local copies of all recursive dependencies, you have to be able to react when e.g. a security vulnerability is found in one of them. If you have 1600 of them, you probably don't know them all very well, so you might already be in trouble in the assessment stage. If there's a fix upstream, you might need to update to the newest version, and that in turn might force you to upgrade more dependencies, or introduce new ones etc. Reproducible builds are great, but they aren't a panacea for dependency hell.
- tasuki 2y agoYes, security vulnerabilities are about the only case when you have to react. The kinds of libraries that can introduce security vulnerabilities (crypto, networking, etc) are hard to write by yourself. If you do happen to write that code by yourself -- to minimize dependencies -- chances are it's full of security holes anyway. Security is hard!
- valyala 2y agoVery good post! We at VictoriaMetrics try following similar practices: - Constantly think on how to simplify the code. - Keep external dependencies to the minimum - https://valyala.medium.com/stripping-dependency-bloat-in-victoriametrics-docker-image-983fb5912b0d https://valyala.medium.com/stripping-dependency-bloat-in-vic... . We have public development goals, which outline these practices - https://docs.victoriametrics.com/goals/ https://docs.victoriametrics.com/goals/ Sometimes we have to complicate the code: - when this simplifies the usage of VictoriaMetrics products - when this significantly reduces resource usage (RAM, CPU, disk IO, disk space, network) for typical production workloads