16 ms·

Designing Low Upkeep Software

396 points by mhb 5y ago

mathgladiator 5y ago
Using a filesystem is underrated these days with web this or that. I'm debating relying only on the disk for my Adama platform (http://www.adama-lang.org/ http://www.adama-lang.org/) where part of my thesis is that by combining mem, cpu, storage into a single node that ill get decent utilization once I figure out a durability story sorted out.
- antoinealb 5y ago
  I personally find myself reaching more and more often to SQLite as an in-between. I like having access to basic SQL and not needing to care about filesystem details for reliability is nice. And it is part of Python standard library so I need literally nothing added.
z3ugma 5y ago
Slight alternative: let smart people who design rock-solid software handle serialization for you, use SQLite to get the benefits of files-on-disk, portability, and make it easier if you use different datatypes in your scripts.
- nemetroid 5y ago
  I think SQLite fits the bill described in the post. It's included in Python's standard library, stores data on the file system, and doesn't require an external process.
  - phkahler 5y ago
    But it IS an additional dependency.
    - BiteCode_dev 5y ago
      If you have cPython, you have sqlite. How is that an additional dependency?
      - WillDaSilva 5y ago
        Not all distributions of CPython 3 come with sqlite, as it's an optional feature that is only included when the sqlite library is found at compile time.
        BiteCode_dev 5y ago
        The official installers for windows, linux, the app stores and the red hat and debian repo all include it. At this point, this is nitpicking. Tkinter I would understand, but not sqlite. Even django doesn't work without it.
        WillDaSilva 5y ago
        The most common scenario in which you may be without the sqlite package is when you build Python from source, which is fairly common nowadays thanks to tools like pyenv. Also the FreeBSD Python package doesn't come with sqlite by default.
        BiteCode_dev 5y ago
        Freebsd and compiling is everything but common. Probably 1 coder for 100000. If you do that, you are assumed to have removed the safety belt and know what you are doing. I know we are on HN, but you have to remember half the python coders are not even dev. They are geographers, mathematicians, biologists, students, teachers, data analysts. And of course they are on mac or windows. Even I, and I started to code in python 2.4, never have to compile python. I do it for fun from time to time, but I don't use that professionnally. If you use a platform where sqlite is not there, you are an exeception in a niche of minority. I drive my car assuming there will be breaks on it. It can be missing, but it's not a great assumption.
- KronisLV 5y ago
  This really reminds me of an article that i read a while ago, called "SQLite as an application file format", which seemed to express some ideas like that: https://www.sqlite.org/draft/appfileformat.html https://www.sqlite.org/draft/appfileformat.html
ellimilial 5y ago
For https://biokeanos.com https://biokeanos.com (biomedical data catalog and discovery tool), keeping things on disk and versioning by pushing to repo saved me months already on dubugging and maintenance only.
- warpech 5y ago
  I have been thinking of such setup lately for another piece of software. Is your sistem distributed in any way? If so, how do you resolve conflicts in version control?
drivers99 5y ago
I've been thinking about a bare metal Forth system and one of my goals for it would be that anyone could recreate it without having any specific dependencies to build it. Something fully explained like jonesforth but without a linux and assembler dependency. Part of my motivation for that is when I try to learn bare metal programming, the tutorials are rapidly outdated because of changing dependencies/tools, broken links, outdated information, etc. Maybe just some hex codes of the machine language which you could use xxd to convert to a binary, or if you didn't have that you could use any programming language that can write a binary file to create it, or a hex editor. The ultimate step would be that the system would be able to compile the next version of itself. (Self-hosted forth implementations are really common; you can find a bunch by googling that.)
- senkora 5y ago
  Look into CollapseOS if you haven’t already! https://collapseos.org/ https://collapseos.org/
  - drivers99 5y ago
    Wow that's cool. I'll definitely be checking it out. Ironically though, it illustrates one of my points: the documentation link is already broken (although it also exists in the download). It says it runs on a POSIX environment and uses cc so it still has more dependencies, but definitely worth digging into and a step in the right direction. Thanks.
- Karrot_Kream 5y ago
  Bare metal programming in C99 hasn't changed in... Ever really, so that's another option if you want something less esoteric than Forth. (Though Forth is a lot of fun and will give you a dynamic experience that few other environments at this level of the stack can offer.)
- akkartik 5y ago
  I was just telling someone[1] that I explored Forth for this exact purpose a few years ago, but ended up with a more imperative, statement-oriented approach: https://github.com/akkartik/mu https://github.com/akkartik/mu [1] https://merveilles.town/@akkartik/106978783913724906 https://merveilles.town/@akkartik/106978783913724906
mtlynch 5y ago
I definitely agree with OP on this and do my best to minimize maintenance burden for most of my projects. This has worked out pretty well in that almost everything I've published[0] can chug along in the background for months or years without me touching them at all (some of them earn a few hundred dollars per month[1]) >On the server I run Ubuntu LTS until it's near the end of its supported lifetime. Every two years I run do-release-upgrade and move to the next one. This cost is shared over many jefftk.com projects, so it isn't too bad. I was surprised by this part because running your own server seems like a big maintenance burden. I'd expect that you have to regularly upgrade your packages to protect yourself from security vulnerabilities. I've avoided this by using PaaS solutions like AppEngine or static hosting + cloud functions. The closest thing I have to a VPS is publishing Docker containers, but that's basically my last resort. [0] https://mtlynch.io/projects/ https://mtlynch.io/projects/ [1] https://mtlynch.io/retrospectives/2021/09/#legacy-projects https://mtlynch.io/retrospectives/2021/09/#legacy-projects
- christophilus 5y ago
  Application of security patches can be automated, and depending on your application’s dependencies, can be done without worry. I run about 10 bare metal transcoding servers, and it’s zero hassle.
  - mtlynch 5y ago
    Can you share more about your setup? Is it cronjobs to just apt-get update && apt-get upgrade -y? How do you handle packages that require reboots?
    - christophilus 5y ago
      I haven’t had a reboot required in over a year, I think. The service is a Go program, and the only OS-managed dependency is ffmpeg. If a reboot interrupts the transcoding jobs, they just retry, so it’s really zero maintenance.
    - stevekemp 5y ago
      If you're running Debian/Ubuntu then you should look at the "unattended-upgrades" package.
pnt12 5y ago
I wrote a static website for a family business using vanilla html - it has been up and running for several years. I also have an (abandoned) website using a markdown to html converter. I've switched converters two times because they eventually broke over the years, and finally moved to plain pandoc + script the last time. Even then, the CSS somehow broke and I haven't fixed it yet. Maybe my next project will be to convert it to html-only and actually add updates instead of just dreading to pick it up and find another broken tool.
iveqy 5y ago
Very nice writing except the python part. Just 2to3 really have been a maintenance pain for me. The hardest part with script languages to actually know if somethings is wrong or not without extensive testing. Writing something maintenance free should be done in a compiled language.
- bArray 5y ago
  > Very nice writing except the python part. Just 2to3 really have been a maintenance pain for me. I agree. I could have been talked into getting onboard with Python for long-term low-maintenance projects if only they didn't burn everybody with the backwards compatibility. I understand of course why the changes were made, but it doesn't make it any easier. It's not impossible that 3 -> 4 will do the same. I was wondering the other day whether something that is already dead might make a better scripting language. For example, you haven't got to worry about any random changes to the COBOL language! I suspect something like JS is pretty much at maturity now with WebASM poised to replace it.
  - iamevn 5y ago
    I doubt 3->4 will be too much of a shock, iirc the only planned breakage will be switching annotations to have delayed evaluation so annotations in things like classes can reference the thing that's not yet defined
- joshuaissac 5y ago
  I think it is not necessarily about being compiled versus interpreted. It is more about static typing, and the type-related issues being revealed and fixed right at the beginning instead of only when the user invokes the problematic branch of code at runtime.
  - jhgb 5y ago
    On the other hand, one of most long-lived and apparently low maintenance pieces of software I've ever used so far has been GNU Maxima, which is not written in a statically typed language.
vonadz 5y ago
I don't know about waiting so long on updating Ubuntu, or essential server software. As an Arch Linux user, I've come to loathe outdated packages that break, which is ironic because Arch is notorious for breaking on new updates. From personal experience, the latter happens less frequently than the former.
hsn915 5y ago
People think if a project on github hasn't been updated since 3 months then it has been abandoned!! Like, why don't we just let projects be "done"? Things don't need to be maintained and updated for eternity. In my mind, the best software engineering is where you solve a problem once and your solution just works and needs no configuration or maintenance or updates. This of course has a very low chance of happening if your system has to exist as a part of "ecosystem" where you expect/assume the presence of some external service that can change its API on a whim (or just disappear). Yes, to a certain degree it's impossible to design software that does not exist as a part of an ecosystem, which is why I put it in quotes. Some APIs are stable and are guaranteed to continue to exist for a very long time: CPU instruction sets, networking protocols (IP/UDP/TCP), operating systems (AFAICT: Linux (the kernel) works hard to not break user programs, and Windows is kind of known for bending over backwards to maintain compatibility with old programs), file systems, etc. What I'm advocating for here requires that your program be compiled into a native executable binary file, and it must embed all its library dependencies (aka static linking).
- fiddlerwoaroof 5y ago
  This is one reason I really like Common Lisp and Clojure: both communities have developed a mindset that allows libraries to be done and, as a result, I notice a lot less churn than in other languages.
  - eigenhombre 5y ago
    I also like them for the same reason. The languages themselves evolve slowly (Clojure) or not at all (Common Lisp). A library that emphasizes simplicity and does One Thing Well can be stable and continue to be useful indefinitely. There are network effects too -- if my library uses stable libraries and is built on a stable language, it need not change at all. On the other hand, there are the big libraries that, for convenience, pull one or two small things from several others -- their surface area becomes so large they are practically guaranteed to generate more work in terms of security updates, dependency conflicts, and the like. I experience this at times developing in Clojure, whose hosted nature means issues can emerge from the Java ecosystem, which is obviously enormous and dynamic. We recently had a defect at work relating to a conflict between transitive (Java) dependencies from a third party library used to serialize exceptions prior to transmission to their service, and another tool used only for generating "literate"-style code documentation. While not a production defect, it cost us some time and could have been avoided (both by us and the libraries' authors) by more hard-headed evaluation of the tradeoffs involved in introducing certain dependencies in the first place.
pirate787 5y ago
Related to this -- many popular packages churn through changes for change-sake. For example LTS on Symfony is just 3 years and on Doctrine, a single contributor is renaming a bunch of standard PHP/DBAL functions for no serious reason other than feels. It would be nice if LTS was more like a 10 year commitment but I know that's not fun for developers and it is a serious drawback to a lot of open source projects. The frameworks have to change because PHP is changing and then you get a stack that's broken and needs multiple upgrades and major refactoring.
- jagged-chisel 5y ago
  No one wants to commit to 10yrs because of income concerns. Software developers tend to not be business folks. I'd give a 10yr commitment on a project that was paying me and required minimal maintenance.
- jbjbjbjb 5y ago
  This is a pet peeve of mine too. It helps to have a small public api and encapsulate the parts we don’t need and to just wait until the api is stable before going to v1.
- userbinator 5y ago
  It would be nice if LTS was more like a 10 year commitment but I know that's not fun for developers I'm surprised that some developers think constant breaking changes is "fun".
  - Symmetry 5y ago
    Maybe from a "If it hurts do it more often" sense.
brundolf 5y ago
Re: dependencies, for "low-upkeep" projects I also like pinning specific patch numbers instead of doing any kind of semver pattern Also +1 to using files as a DB for small-scale projects. Something I've done a few times is keep a single in-memory JS object as the source of truth, and just write it to disk as JSON on a certain cadence/read it from disk at startup. It's wonderful having your little one-off server not need to be a distributed system from day 1.
- y4mi 5y ago
  SemVer was specifically for libraries. It's really pointless to adhere to it if your code won't be imported into software projects of other people.
  - brundolf 5y ago
    I meant for my dependencies, so they don't automatically change underneath me, even in ways that are harmless according to semver
    - y4mi 5y ago
      My only point was that SemVer is unnecessary if you're the only consumer of that code. A simple integer or a date would work just as well and would be much easier to automate.
      - brundolf 5y ago
        Maybe, but that's not at all the situation I'm talking about
mst 5y ago
The "essentially no dependencies" thing always fascinates me. Certainly, complex C code is a risk across OS versions but ... perl5 version 8.1 was released just over 18 years ago and after multiple years of annual major releases of perl we're now on version 34. Many of my chosen cpan dependencies produce the exact same output when on a 5.8.1 install as they do on a 5.34.0 install, and I can move them to a new machine/OS via rsync or tar of the local::lib structure (virtualenv like thing configured by setting a couple of environment variables) and it Just Works. Am I weird here? Is the author? Is python unusually bad? Is perl unusually good? This all seems quite strange to me.
- oakfr 5y ago
  I am with you here. That being said, I believe the author refers to « weaker » libraries (higher-level, more recent etc. )
- liveoneggs 5y ago
  perl is unusually good in this regard, as I'm sure you already knew.
  - mst 5y ago
    I do suspect that, but my fingerprints are on enough of cpan these days that I would worry about bias if I simply asserted it.
- TedDoesntTalk 5y ago
  I guess it’s the language maintainers who decide if they want to keep backwards compatibility. And you can’t even say that’s due to language age, necessarily. PHP 8 (Nov 2020) had many breaking changes over past PHP versions and that language is from the mid 1990s. On the other hand, Java also dates to the mid 1990s and code written for Java2 still compiles and executes in Java 17 (2021).
  - mst 5y ago
    Java is not my cup of anything but I do absolutely respect that about it. I have a long standing semi-shitpost opinion that if faced with a choice of technologies, it's often best to pick the one that's been "declared dead/dying" because that means it's stable and in wide use.
samsquire 5y ago
I find that my old JavaScript projects and tools such as Jsbin break whenever dependencies are upgraded or I update to the next version. My python code without dependencies just works. My ruby code breaks all the time. My Java code never breaks but my Java code has very few dependencies. I wish there was a way to abstract the interface of mechanism in a promise that other people can implement and evolve and outsource decision making to someone else who has more skill or nuance in a way that is looser and more reliable than a particular API. APIs want to evolve over time but a promise to fulfil a high level goal is different. An API is a mechanism like a particular bottle whereas a bottle mould captures the essence of a bottle but is not a bottle itself. Interfaces (such as in Java) are just APIs themselves so they don't really solve the problem. For example, I want to specify routes that correspond to the URL of my React app or I want to encrypt data with the most secure algorithm. These two example mechanisms evolve all the time. I kind of want to abstract the intent and input. An API doesn't have to be a function call, it can be a data structure. The mechanism doesn't matter to me, but I want the mechanism to adapt with evolution. How do you decouple the method/mechanism with the goal? Evolution in nature tries to find a variable to mutate that can produces better survivability. Like higher body temperature in cats which must have an evolutionary advantage for the mutation to have survived.
amadeuspagel 5y ago
I don't like the solution for comments. I think there's real value in having a discussion that is directed at and moderated by the author. For example, if an author writes a post that people on the forums from which he aggregates comments generally disagree with, I don't want to see one comment after another denouncing him. I want to see comments from people who at least respect him enough to write a comment on his blog, directed at him. Would a wordpress blog really be harder to maintain then this? If so, I think a better solution would be a mailto link, so that the email contains some basic text like this: > My comment on blog.com/article3: > ... Then maybe you could automate adding the email as a comment to the article in some way.
- marcosdumay 5y ago
  If you accept comments, you will have to deal with spam. That is a very-high maintenance problem.
- jefftk 5y ago
  I've considered building my own comment system, and I think I could do it in a low-maintenance way. The main reason I haven't is that I actually like the pattern where discussion happens on facebook/lesswrong/hn/etc. I get wider and more interesting feedback that way.
haolez 5y ago
A few years ago, I made an extremely simple chat bot for Telegram that gave artists access to Spotify For Artists' API. I've used a simple Lambda and that's it. I've never touched it again and I've found out recently that I now have a user base there and everything still works :) my longest living software ever was made in a couple hours. A little frustrating, but awesome nonetheless.
Havoc 5y ago
One of the reasons the whole server less philosophy appeals. Stuff can still go very wrong but at least there isn’t a server that is your problem when things go sideways.
015a 5y ago
I've been at my current company for maybe 6 years now, in an engineering role. In that time, the service-oriented code which has significantly outlasted anything else, with nearly no maintenance, has maybe surprisingly or unsurprisingly been: AWS Lambda functions. We have quite a few Lambda functions (written in Node) that I believe pre-date even my tenure at the company; the only modifications in the interim, maybe a few lines of obscure bug fixes, updating the lambda node version, we switched from JS to TS everywhere a few years ago, and keeping the CI updated on those repos (which is a rather ironic and wasteful investment of resources given they've only changed a half dozen times in something like six years). Checking in on those dozens of functions handling millions of invocations every month, and seeing them just chug along with no maintenance, has convinced me, personally, that if a problem domain can be solved in a FaaS-like way, its the way to do it. Not every problem can, and I'm not specifically advocating for Lambda vs Azure FaaS vs an open source kubernetes-based solution. More-so that, every deployment artifact has to draw an "operations" line, to the left of that is the rote, daily operations banality, and to the right is the ever-changing business problem space. The further right you can draw that line, to push as much as possible into operations, the better; it gives you optionality to outsource that left part, maybe to automated software, maybe to AWS, maybe to an in-house operations team. FaaS draws that line roughly as far right as I think a general purpose programming system can; further right would approach things like Excel, Retool, etc.
- MathMonkeyMan 5y ago
  I think the answer lies in your last paragraph, but your comment made me think: Perhaps Lambda functions are resilient because they're: - scope constrained - surrounded by APIs only (e.g. no cheating by having a whole POSIX system) - infrastructure is maintained solely by a giant company that you pay to not have to think about it I haven't used any "serverless" stuff, so I don't really know. Per the article's recommendation, I have noticed at work that resilient software is often a 400 line python script with no other dependencies; or, a medium sized piece of C++ infrastructure that depends only on the OS, the compiler, and a hand-written build system.
  - Hendrikto 5y ago
deleted 5y ago
[deleted]
brian_herman 5y ago
I noticed on one of the files he uses lxml, that is not part of the standard library is this one of those do as I say not as I do? https://github.com/jeffkaufman/webscripts/blob/b6a004790b46402f94e4172775b7dd9676775bd4/makerss.py#L38 https://github.com/jeffkaufman/webscripts/blob/b6a004790b464...
- jefftk 5y ago
  When I started this script ~12y ago there wasn't any good html parsing option in the standard library, and that was worth taking a dependency for. I don't see a conflict there with "minimize your dependencies and limit them to ones that value backwards compatibility"?
ChrisMarshallNY 5y ago
I've written SDKs that have lasted for decades. Number of reasons why. Most don't stay static, though. People mod them. They need to be something that can be handed off to a new team. I find documentation is key. I write about that here: https://littlegreenviper.com/miscellany/leaving-a-legacy/ https://littlegreenviper.com/miscellany/leaving-a-legacy/
bogdanoff_2 5y ago
I wonder if there's a way to do server-side javascript programming with this philosophy. For client-side I can just throw together some html and javascript to make a simple app. But if I want to use javascript on the server it seems like I need to download a bunch of npm dependencies to do anything useful.
- WA 5y ago
  Sure, you can do it on the server with Node. Node has `http` as part of its standard library and you can write a simple webapp without third party dependencies. Check this tutorial for example [0]. The problem is that Node's http lib isn't super productive and you gotta write some "solved" stuff like cookie handlers. Also, there is no database standard library. If you want to persist data, you could use files. If you need a database (even SQLite), you gotta use third-party stuff. In comparison, PHP has a lot of stuff build-in and you can be more productive with zero dependencies in PHP. But then, maybe it doesn't really matter if you include a well-maintained npm package or if you have a more powerful standard library that has some features deprecated after a while. It's still "other people's code", although npm packages require some kind of installation, whereas a standard library usually doesn't. [0]: https://www.section.io/engineering-education/pure-node-js-no-frameworks-or-packages/ https://www.section.io/engineering-education/pure-node-js-no...
  - tored 5y ago
    There was a trend a few years ago (maybe still going) that languages with a large standard library, like PHP, is bloated. That we should strive against minimalism and then just share dependencies when needed. But then the nodejs left-pad debacle happened. I used to be more on the minimalist side, but over time I have switched to the opposite, today I can really appreciate the bloated nature of PHP, everything from large projects to small scripts just becomes easier to handle.
Karrot_Kream 5y ago
I follow a lot of these practices myself for hobby code, but if I had to offer one change, it would be to use SQLite instead of the filesystem for data. It does add some complexity in languages like Go that don't "easily" link with C libraries, and you do need to remember the table schemas that you're using, but for me the benefits are worth it. I never worry about locking files, naming some key badly, or any of the other issues that come with flat file dbs.
- bob1029 5y ago
  SQLite is an absolute powerhouse. Filesystem interop is the best way for it to start proving itself in any codebase. Loading assets for a AAA game straight from disk? You would have a really hard time doing better than SQLite - especially if you have millions+ of unique items to contend with. In many cases, SQLite is faster than the actual file system it resides upon [0]. We use SQLite for nearly 100% of business object persistence in our product today. The performance & scalability arguments fall on some very deaf ears after years of experience optimizing and succeeding with this stack in production. [0] https://www.sqlite.org/fasterthanfs.html
phendrenad2 5y ago
Good advice for personal projects. Sadly you can't use a lot of free libraries out there, because more often than not you'll be forced to upgrade (possibly due to a security vulnerability that the author refuses to backport to "old" major revisions), and encounter a backward-incompatibility when upgrading.
TedDoesntTalk 5y ago
> This is not how I approach design at work, or necessarily how I would recommend doing so. Having a budget where initial creation is essentially free (fun!) while maintenance is extremely expensive (drugery!)
titzer 5y ago
The first backend that I achieved self-hosting of Virgil was the JVM backend. I targeted version 5 classfiles. These classfiles are still accepted by JVMs but there have since been several significant additions to the format. In particular, classfiles are required to have stackmaps, which are pretty much a major distraction for me to add to my backend (so I haven't!). The second backend that I bootstrapped on was x86-darwin. That worked great for about 7 years, until Apple decided to deprecate first 32-bit, and now, I guess, x86 altogether. Thankfully I ported to x86-linux soon after, as that syscall ABI is solid as a rock. But you know, things keep changing. To keep up, I finally had to bite the bullet and write an x86-64-linux backend. And I guess at somepoint now arm64 if I ever want to work natively on a mac again. It's an endless cycle. Thankfully now, emulators are pretty good, but I am still struggling with getting QEMU to run fast on MacOS on the M1, so it's just more futzing with enormous emulation stacks just to run old stuff.
Datenstrom 5y ago
Beautiful code denies its existence so perfectly you forget you wrote it.
gfxgirl 5y ago
> The biggest piece is minimizing your dependencies, and limiting them to ones that value backwards compatibility Three.js, one of the most popular JS libraries, has the explicit policy of ignoring backward compatibility. https://github.com/mrdoob/three.js/wiki/Migration-Guide https://github.com/mrdoob/three.js/wiki/Migration-Guide I find it not only irresponsible but arguably mean to other devs time and lives but apparently I'm in the minority. Every time I have to spend an afternoon or evening updating stuff instead of doing something new or visiting friends etc I silently curse the devs
- jefftk 5y ago
  You curse the devs who build excellent libraries and give them away for free, because they do not accept the additional burden of never making changes that would require you to adjust your usage? You could choose not to use such libraries, but as long as they're up front about it I don't see how you can hold it against them.
  - gfxgirl 5y ago
    it upsets me because they are the only library of 100s I use that do this. They consciously break things every month. they don't give a fuck how many man hours it burns of other people's time. No other project I know of does this. It's not just time using the library. Every volunteer that wrote docs or a tutorial their work is now out of date. every answer on S.O. is more than 3-4 months old is wrong. Every new user now has the burden of trying to learn where any content they find is practically guaranteed not to work correctlt with the current version. it's mean in the extreme to piss on so many people
- fulafel 5y ago
  I'm sure you know this but for the record, they don't actually ignore backward compatibility, it features often as a concern in the dev discussions and they're conscious of when they break it, and there can be deprecations[1].. it's a conscious tradeoff allowing development to go faster. I'd argue Three.js beats native to make low upkeep & long lived end user (=portable) 3D apps, given all the churn, constant driver breakages necessitating new workarounds, proprietary-ness and fragmentation in 3D APIs. With Three you update to newer versions on your own time (if ever) and stuff doesn't break from under you. And in the decades long view WebGL 1 / WebGL 2 / WebGPU is used behind the scenes. [1] example: https://github.com/mrdoob/three.js/pull/21777 https://github.com/mrdoob/three.js/pull/21777
AtlasBarfed 5y ago
Well, it probably starts with a stable secure cross-platform language and ecosystem. Things like this are best explained by a very grey neckbeard who's been through a lot. Author is not particularly old, author programs in python (something that has broken compatibility once)... "stick to the standard library" is about at ground level an opinion on this subject that a typical HN probably thought of as they clicked the link. 1) need a stable, mature, secure language Stable eliminates Rust, Python, Perl. Go hasn't been around long enough. Secure eliminates C, C++, probably D and most compiled languages No dependencies: yikes there goes Java because of the JVM dependency. Assembly isn't portable. ...Seriously, are we left with Ada? Secure, strict, mature, reasonably portable. And virtually unknown outside of government/military. 2) UIs age, but CLIs really don't So make it a CLI, and use unix philosophy 3) OS churn Hopefully with the practical cessation of gigahertz scaling, the industry will settle down. Then again, the ARM hordes are approaching to wipe out all the x86 stability in desktops and servers. Perhaps the best thing for this in the industry has been Windows subsystem for Linux. If we could get an OSX subsystem for Linux, that would be a real boon. Even Linux on it's umpteenth release is still changing quite a lot, and NIH reinvention like Wayland and Systemd and toss-it-all-out Gnome 3. Well, it kind of keeps us employed. I'd adhere to assuming a unix-ish os, like maybe POSIX. So: Ada, CLI/text, POSIX compatibility. That's not limiting at all, is it? Sigh.
- jhgb 5y ago
  > 1) need a stable, mature, secure language > Stable eliminates Rust, Python, Perl. Go hasn't been around long enough. > Secure eliminates C, C++, probably D and most compiled languages > No dependencies: yikes there goes Java because of the JVM dependency. Common Lisp, then?
  - AtlasBarfed 5y ago
    Almost, but Java has a VM. Yes you can technically compile Java to an executable but it's not done much. CL has a VM too, does it not?
    - jhgb 5y ago
anyfactor 5y ago
I think the idea of we need continuous improvements to maintain a steady userbase is a myth. And it is disturbing to me that we talk about browser support and the global reach internet services yet I struggle to access many popular websites on my galaxy S2.
karmakaze 5y ago
The most important things I've learned is that the system has to be (1) self-maintaining (e.g. security updates, rotate logs, alerting monitors so safe to ignore), (2) super easy to make and deploy a change. This one consists of a README that word-for-word works so you don't have to think. Minimizing steps with push-to-github auto-deploy is great, as is an easy setup for a hot-reloading local dev. Following those two points, I've been able to use whatever languages/tools tickle my interest with little impact on maintenance effort. Dependencies are few, but not minimal, pinned to exact versions. FYI, these have run the range of Vue/TypeScript, Node, Go, Java/Kotlin, Clojure, F#, and Php, with typically MySQL (sometimes multi-master).
- shoo 5y ago
  > (2) super easy to make and deploy a change. This one consists of a README that word-for-word works so you don't have to think. Minimizing steps with push-to-github auto-deploy is great Great advice. Minimise the energy barrier to being able to ship a patch or a new feature after neglecting the service for months/years and forgetting how you set everything up. I always have a bit of a laugh after patching my hobby project and doing a handful of iterative automated deploys while making small changes, then switching back to the day job where aspiring to do a single deploy will typically require hours of meetings and general bureaucratic overhead. Another possible tip is: understand your deployment environment and learn how to take advantage of what it offers out of the box without adding additional layers or dependencies. E.g. I deploy a hobby webapp to my own debian server. Debian comes with apt for package management and systemd for managing services, so I use both of those (yes, even apt for python package dependencies).
  - karmakaze 5y ago
    Totally agree. Systemd .service unit files are super simple to understand and setup. Well worth the small time investment.
unknownOrigin 5y ago
Designing for minimum maintenance is nice when you're working on something for "yourself"... not so much if you're creating something for someone else. Since designing for minimum maintenance takes additional time (and money) and most stakeholding entities (contractee, employer) will want the job to be done as soon as possible (and only THEN worry about upkeep), this can easily prevent you from "doing the right thing" simply by the nature of competition. Not to mention the elephant in the room - that minimum maintenance often means less money for the person doing the maintenance, i.e. "you". Vendor lockin is an issue only if you're not the vendor.
cblconfederate 5y ago
php+mysql has been low upkeep for at least 15 years is that enough
milofeynman 5y ago
An interesting take-away from this is designing low upkeep business services. I have been dealing with trying to onboard to a service at work, and there are so many dependencies! Several are pinned to versions that require python2, but I can't install python2 because another service I use calls "python" and assumes it's linked to python 3! And sometimes someone will add a new feature, and include an entire library for that single method call, instead of trying to write a little JavaScript. I almost think there should be a non-trivial process at most businesses for including new libraries so that web devs would try to do it without adding a new library, so they didn't have to deal with red tape!
forgotmypw17 5y ago
Lindy Effect says that software will continue working for as long as it's already been working so far. I use this to my advantage by writing software AS IF it has already been working for 20 years, meaning I use only languages and technologies which have not introduced breaking changes in 20 years or so. It's worked out quite well so far!
- est31 5y ago
  Which language or technology has not introduced breaking changes in the last two decades? Windows has good backward compatibility, but even Windows can't run really old programs. C++ has evolved dramatically in the last 20 years. Even C had changes. Do people still use tables to create websites, or jquery?
  - forgotmypw17 5y ago
    Now, to be fair, I'm talking about carefully writing code with the knowledge of the previous 20 years, which is a little bit like "cheating", whatever that concept means here. I use ASCII/textfiles, HTTP, CGI, SSI, access.log, Perl, Bash, and small subsets of PHP, HTML, JS, CSS, and almost no third-party libraries. Certainly no JQuery or anything similar. I'm not hardcore enough for C or C++, but I know that a program written 20 years ago in these languages would most likely compile today.
    - aahortwwy 5y ago
      > a program written 20 years ago in these languages would most likely compile today Not without continuous attention over those 20 years. I'm currently struggling to get a C program last touched 10 years ago to compile.
      - forgotmypw17 5y ago
        Like I said, I'm cheating a little bit, and using the knowledge I have today to "retro-edit" this hypothetical 20-year-old program.
tomerbd 5y ago
Sounds good for personal projects.
oftenwrong 5y ago
On this topic, what happened with https://www.jefftk.com/nextbus/ https://www.jefftk.com/nextbus/ ? Seems to be a 404 now. Was there a breaking change or disruption in the nextbus API? I loved its bare-bones UI. It was definitely one of the most usable webapps. I assume its a11y was nearly perfect as well. If anyone would like to see it, here's an archived snapshot: https://web.archive.org/web/20170622033104/https://www.jefftk.com/nextbus/ https://web.archive.org/web/20170622033104/https://www.jefft... I also admire https://diskprices.com/ https://diskprices.com/ for the same reasons.
tored 5y ago
Few weeks ago I wrote a lengthy post here on HN about how to write low upkeep software in PHP https://news.ycombinator.com/item?id=28232146 https://news.ycombinator.com/item?id=28232146
6510 5y ago
For fun personal projects I like using php to manipulate static html documents. I have quite a few projects with broken admin areas written in ancient php but (besides from user contributions) the html/css/js/xml/json just continues to work as if nothing happened. Sometimes I just delete the admin tools.
XCSme 5y ago
I am not sure writing your own build tools is the way to go, if there already exists something that does that thing even better.(as the author mentioned, he wrote a templating engine that updates all static files when a shared component is edited)

396 points by mhb 5y ago

Using a filesystem is underrated these days with web this or that. I'm debating relying only on the disk for my Adama platform (http://www.adama-lang.org/ http://www.adama-lang.org/) where part of my thesis is that by combining mem, cpu, storage into a single node that ill get decent utilization once I figure out a durability story sorted out.

antoinealb 5y ago
I personally find myself reaching more and more often to SQLite as an in-between. I like having access to basic SQL and not needing to care about filesystem details for reliability is nice. And it is part of Python standard library so I need literally nothing added.

z3ugma 5y ago

Slight alternative: let smart people who design rock-solid software handle serialization for you, use SQLite to get the benefits of files-on-disk, portability, and make it easier if you use different datatypes in your scripts.

nemetroid 5y ago
I think SQLite fits the bill described in the post. It's included in Python's standard library, stores data on the file system, and doesn't require an external process.
- phkahler 5y ago
  But it IS an additional dependency.
  - BiteCode_dev 5y ago
    If you have cPython, you have sqlite. How is that an additional dependency?
    - WillDaSilva 5y ago
      Not all distributions of CPython 3 come with sqlite, as it's an optional feature that is only included when the sqlite library is found at compile time.
      - BiteCode_dev 5y ago
        The official installers for windows, linux, the app stores and the red hat and debian repo all include it. At this point, this is nitpicking. Tkinter I would understand, but not sqlite. Even django doesn't work without it.
        WillDaSilva 5y ago
        The most common scenario in which you may be without the sqlite package is when you build Python from source, which is fairly common nowadays thanks to tools like pyenv. Also the FreeBSD Python package doesn't come with sqlite by default.
        BiteCode_dev 5y ago
        Freebsd and compiling is everything but common. Probably 1 coder for 100000. If you do that, you are assumed to have removed the safety belt and know what you are doing. I know we are on HN, but you have to remember half the python coders are not even dev. They are geographers, mathematicians, biologists, students, teachers, data analysts. And of course they are on mac or windows. Even I, and I started to code in python 2.4, never have to compile python. I do it for fun from time to time, but I don't use that professionnally. If you use a platform where sqlite is not there, you are an exeception in a niche of minority. I drive my car assuming there will be breaks on it. It can be missing, but it's not a great assumption.
KronisLV 5y ago
This really reminds me of an article that i read a while ago, called "SQLite as an application file format", which seemed to express some ideas like that: https://www.sqlite.org/draft/appfileformat.html https://www.sqlite.org/draft/appfileformat.html

ellimilial 5y ago

For https://biokeanos.com https://biokeanos.com (biomedical data catalog and discovery tool), keeping things on disk and versioning by pushing to repo saved me months already on dubugging and maintenance only.

warpech 5y ago
I have been thinking of such setup lately for another piece of software. Is your sistem distributed in any way? If so, how do you resolve conflicts in version control?

drivers99 5y ago

I've been thinking about a bare metal Forth system and one of my goals for it would be that anyone could recreate it without having any specific dependencies to build it. Something fully explained like jonesforth but without a linux and assembler dependency. Part of my motivation for that is when I try to learn bare metal programming, the tutorials are rapidly outdated because of changing dependencies/tools, broken links, outdated information, etc. Maybe just some hex codes of the machine language which you could use xxd to convert to a binary, or if you didn't have that you could use any programming language that can write a binary file to create it, or a hex editor. The ultimate step would be that the system would be able to compile the next version of itself. (Self-hosted forth implementations are really common; you can find a bunch by googling that.)

senkora 5y ago
Look into CollapseOS if you haven’t already! https://collapseos.org/ https://collapseos.org/
- drivers99 5y ago
  Wow that's cool. I'll definitely be checking it out. Ironically though, it illustrates one of my points: the documentation link is already broken (although it also exists in the download). It says it runs on a POSIX environment and uses cc so it still has more dependencies, but definitely worth digging into and a step in the right direction. Thanks.
Karrot_Kream 5y ago
Bare metal programming in C99 hasn't changed in... Ever really, so that's another option if you want something less esoteric than Forth. (Though Forth is a lot of fun and will give you a dynamic experience that few other environments at this level of the stack can offer.)
akkartik 5y ago
I was just telling someone[1] that I explored Forth for this exact purpose a few years ago, but ended up with a more imperative, statement-oriented approach: https://github.com/akkartik/mu https://github.com/akkartik/mu [1] https://merveilles.town/@akkartik/106978783913724906 https://merveilles.town/@akkartik/106978783913724906

mtlynch 5y ago

I definitely agree with OP on this and do my best to minimize maintenance burden for most of my projects. This has worked out pretty well in that almost everything I've published[0] can chug along in the background for months or years without me touching them at all (some of them earn a few hundred dollars per month[1]) >On the server I run Ubuntu LTS until it's near the end of its supported lifetime. Every two years I run do-release-upgrade and move to the next one. This cost is shared over many jefftk.com projects, so it isn't too bad. I was surprised by this part because running your own server seems like a big maintenance burden. I'd expect that you have to regularly upgrade your packages to protect yourself from security vulnerabilities. I've avoided this by using PaaS solutions like AppEngine or static hosting + cloud functions. The closest thing I have to a VPS is publishing Docker containers, but that's basically my last resort. [0] https://mtlynch.io/projects/ https://mtlynch.io/projects/ [1] https://mtlynch.io/retrospectives/2021/09/#legacy-projects https://mtlynch.io/retrospectives/2021/09/#legacy-projects

christophilus 5y ago
Application of security patches can be automated, and depending on your application’s dependencies, can be done without worry. I run about 10 bare metal transcoding servers, and it’s zero hassle.
- mtlynch 5y ago
  Can you share more about your setup? Is it cronjobs to just apt-get update && apt-get upgrade -y? How do you handle packages that require reboots?
  - christophilus 5y ago
    I haven’t had a reboot required in over a year, I think. The service is a Go program, and the only OS-managed dependency is ffmpeg. If a reboot interrupts the transcoding jobs, they just retry, so it’s really zero maintenance.
  - stevekemp 5y ago
    If you're running Debian/Ubuntu then you should look at the "unattended-upgrades" package.

pnt12 5y ago

I wrote a static website for a family business using vanilla html - it has been up and running for several years. I also have an (abandoned) website using a markdown to html converter. I've switched converters two times because they eventually broke over the years, and finally moved to plain pandoc + script the last time. Even then, the CSS somehow broke and I haven't fixed it yet. Maybe my next project will be to convert it to html-only and actually add updates instead of just dreading to pick it up and find another broken tool.

iveqy 5y ago

Very nice writing except the python part. Just 2to3 really have been a maintenance pain for me. The hardest part with script languages to actually know if somethings is wrong or not without extensive testing. Writing something maintenance free should be done in a compiled language.

bArray 5y ago
> Very nice writing except the python part. Just 2to3 really have been a maintenance pain for me. I agree. I could have been talked into getting onboard with Python for long-term low-maintenance projects if only they didn't burn everybody with the backwards compatibility. I understand of course why the changes were made, but it doesn't make it any easier. It's not impossible that 3 -> 4 will do the same. I was wondering the other day whether something that is already dead might make a better scripting language. For example, you haven't got to worry about any random changes to the COBOL language! I suspect something like JS is pretty much at maturity now with WebASM poised to replace it.
- iamevn 5y ago
  I doubt 3->4 will be too much of a shock, iirc the only planned breakage will be switching annotations to have delayed evaluation so annotations in things like classes can reference the thing that's not yet defined
joshuaissac 5y ago
I think it is not necessarily about being compiled versus interpreted. It is more about static typing, and the type-related issues being revealed and fixed right at the beginning instead of only when the user invokes the problematic branch of code at runtime.
- jhgb 5y ago
  On the other hand, one of most long-lived and apparently low maintenance pieces of software I've ever used so far has been GNU Maxima, which is not written in a statically typed language.

vonadz 5y ago

I don't know about waiting so long on updating Ubuntu, or essential server software. As an Arch Linux user, I've come to loathe outdated packages that break, which is ironic because Arch is notorious for breaking on new updates. From personal experience, the latter happens less frequently than the former.

hsn915 5y ago

People think if a project on github hasn't been updated since 3 months then it has been abandoned!! Like, why don't we just let projects be "done"? Things don't need to be maintained and updated for eternity. In my mind, the best software engineering is where you solve a problem once and your solution just works and needs no configuration or maintenance or updates. This of course has a very low chance of happening if your system has to exist as a part of "ecosystem" where you expect/assume the presence of some external service that can change its API on a whim (or just disappear). Yes, to a certain degree it's impossible to design software that does not exist as a part of an ecosystem, which is why I put it in quotes. Some APIs are stable and are guaranteed to continue to exist for a very long time: CPU instruction sets, networking protocols (IP/UDP/TCP), operating systems (AFAICT: Linux (the kernel) works hard to not break user programs, and Windows is kind of known for bending over backwards to maintain compatibility with old programs), file systems, etc. What I'm advocating for here requires that your program be compiled into a native executable binary file, and it must embed all its library dependencies (aka static linking).

fiddlerwoaroof 5y ago
This is one reason I really like Common Lisp and Clojure: both communities have developed a mindset that allows libraries to be done and, as a result, I notice a lot less churn than in other languages.
- eigenhombre 5y ago
  I also like them for the same reason. The languages themselves evolve slowly (Clojure) or not at all (Common Lisp). A library that emphasizes simplicity and does One Thing Well can be stable and continue to be useful indefinitely. There are network effects too -- if my library uses stable libraries and is built on a stable language, it need not change at all. On the other hand, there are the big libraries that, for convenience, pull one or two small things from several others -- their surface area becomes so large they are practically guaranteed to generate more work in terms of security updates, dependency conflicts, and the like. I experience this at times developing in Clojure, whose hosted nature means issues can emerge from the Java ecosystem, which is obviously enormous and dynamic. We recently had a defect at work relating to a conflict between transitive (Java) dependencies from a third party library used to serialize exceptions prior to transmission to their service, and another tool used only for generating "literate"-style code documentation. While not a production defect, it cost us some time and could have been avoided (both by us and the libraries' authors) by more hard-headed evaluation of the tradeoffs involved in introducing certain dependencies in the first place.

pirate787 5y ago

Related to this -- many popular packages churn through changes for change-sake. For example LTS on Symfony is just 3 years and on Doctrine, a single contributor is renaming a bunch of standard PHP/DBAL functions for no serious reason other than feels. It would be nice if LTS was more like a 10 year commitment but I know that's not fun for developers and it is a serious drawback to a lot of open source projects. The frameworks have to change because PHP is changing and then you get a stack that's broken and needs multiple upgrades and major refactoring.

jagged-chisel 5y ago
No one wants to commit to 10yrs because of income concerns. Software developers tend to not be business folks. I'd give a 10yr commitment on a project that was paying me and required minimal maintenance.
jbjbjbjb 5y ago
This is a pet peeve of mine too. It helps to have a small public api and encapsulate the parts we don’t need and to just wait until the api is stable before going to v1.
userbinator 5y ago
It would be nice if LTS was more like a 10 year commitment but I know that's not fun for developers I'm surprised that some developers think constant breaking changes is "fun".
- Symmetry 5y ago
  Maybe from a "If it hurts do it more often" sense.

brundolf 5y ago

Re: dependencies, for "low-upkeep" projects I also like pinning specific patch numbers instead of doing any kind of semver pattern Also +1 to using files as a DB for small-scale projects. Something I've done a few times is keep a single in-memory JS object as the source of truth, and just write it to disk as JSON on a certain cadence/read it from disk at startup. It's wonderful having your little one-off server not need to be a distributed system from day 1.

y4mi 5y ago
SemVer was specifically for libraries. It's really pointless to adhere to it if your code won't be imported into software projects of other people.
- brundolf 5y ago
  I meant for my dependencies, so they don't automatically change underneath me, even in ways that are harmless according to semver
  - y4mi 5y ago
    My only point was that SemVer is unnecessary if you're the only consumer of that code. A simple integer or a date would work just as well and would be much easier to automate.
    - brundolf 5y ago
      Maybe, but that's not at all the situation I'm talking about

mst 5y ago

The "essentially no dependencies" thing always fascinates me. Certainly, complex C code is a risk across OS versions but ... perl5 version 8.1 was released just over 18 years ago and after multiple years of annual major releases of perl we're now on version 34. Many of my chosen cpan dependencies produce the exact same output when on a 5.8.1 install as they do on a 5.34.0 install, and I can move them to a new machine/OS via rsync or tar of the local::lib structure (virtualenv like thing configured by setting a couple of environment variables) and it Just Works. Am I weird here? Is the author? Is python unusually bad? Is perl unusually good? This all seems quite strange to me.

oakfr 5y ago
I am with you here. That being said, I believe the author refers to « weaker » libraries (higher-level, more recent etc. )
liveoneggs 5y ago
perl is unusually good in this regard, as I'm sure you already knew.
- mst 5y ago
  I do suspect that, but my fingerprints are on enough of cpan these days that I would worry about bias if I simply asserted it.
TedDoesntTalk 5y ago
I guess it’s the language maintainers who decide if they want to keep backwards compatibility. And you can’t even say that’s due to language age, necessarily. PHP 8 (Nov 2020) had many breaking changes over past PHP versions and that language is from the mid 1990s. On the other hand, Java also dates to the mid 1990s and code written for Java2 still compiles and executes in Java 17 (2021).
- mst 5y ago
  Java is not my cup of anything but I do absolutely respect that about it. I have a long standing semi-shitpost opinion that if faced with a choice of technologies, it's often best to pick the one that's been "declared dead/dying" because that means it's stable and in wide use.

samsquire 5y ago

I find that my old JavaScript projects and tools such as Jsbin break whenever dependencies are upgraded or I update to the next version. My python code without dependencies just works. My ruby code breaks all the time. My Java code never breaks but my Java code has very few dependencies. I wish there was a way to abstract the interface of mechanism in a promise that other people can implement and evolve and outsource decision making to someone else who has more skill or nuance in a way that is looser and more reliable than a particular API. APIs want to evolve over time but a promise to fulfil a high level goal is different. An API is a mechanism like a particular bottle whereas a bottle mould captures the essence of a bottle but is not a bottle itself. Interfaces (such as in Java) are just APIs themselves so they don't really solve the problem. For example, I want to specify routes that correspond to the URL of my React app or I want to encrypt data with the most secure algorithm. These two example mechanisms evolve all the time. I kind of want to abstract the intent and input. An API doesn't have to be a function call, it can be a data structure. The mechanism doesn't matter to me, but I want the mechanism to adapt with evolution. How do you decouple the method/mechanism with the goal? Evolution in nature tries to find a variable to mutate that can produces better survivability. Like higher body temperature in cats which must have an evolutionary advantage for the mutation to have survived.

amadeuspagel 5y ago

I don't like the solution for comments. I think there's real value in having a discussion that is directed at and moderated by the author. For example, if an author writes a post that people on the forums from which he aggregates comments generally disagree with, I don't want to see one comment after another denouncing him. I want to see comments from people who at least respect him enough to write a comment on his blog, directed at him. Would a wordpress blog really be harder to maintain then this? If so, I think a better solution would be a mailto link, so that the email contains some basic text like this: > My comment on blog.com/article3: > ... Then maybe you could automate adding the email as a comment to the article in some way.

marcosdumay 5y ago
If you accept comments, you will have to deal with spam. That is a very-high maintenance problem.
jefftk 5y ago
I've considered building my own comment system, and I think I could do it in a low-maintenance way. The main reason I haven't is that I actually like the pattern where discussion happens on facebook/lesswrong/hn/etc. I get wider and more interesting feedback that way.

haolez 5y ago

A few years ago, I made an extremely simple chat bot for Telegram that gave artists access to Spotify For Artists' API. I've used a simple Lambda and that's it. I've never touched it again and I've found out recently that I now have a user base there and everything still works :) my longest living software ever was made in a couple hours. A little frustrating, but awesome nonetheless.

Havoc 5y ago

One of the reasons the whole server less philosophy appeals. Stuff can still go very wrong but at least there isn’t a server that is your problem when things go sideways.

015a 5y ago

I've been at my current company for maybe 6 years now, in an engineering role. In that time, the service-oriented code which has significantly outlasted anything else, with nearly no maintenance, has maybe surprisingly or unsurprisingly been: AWS Lambda functions. We have quite a few Lambda functions (written in Node) that I believe pre-date even my tenure at the company; the only modifications in the interim, maybe a few lines of obscure bug fixes, updating the lambda node version, we switched from JS to TS everywhere a few years ago, and keeping the CI updated on those repos (which is a rather ironic and wasteful investment of resources given they've only changed a half dozen times in something like six years). Checking in on those dozens of functions handling millions of invocations every month, and seeing them just chug along with no maintenance, has convinced me, personally, that if a problem domain can be solved in a FaaS-like way, its the way to do it. Not every problem can, and I'm not specifically advocating for Lambda vs Azure FaaS vs an open source kubernetes-based solution. More-so that, every deployment artifact has to draw an "operations" line, to the left of that is the rote, daily operations banality, and to the right is the ever-changing business problem space. The further right you can draw that line, to push as much as possible into operations, the better; it gives you optionality to outsource that left part, maybe to automated software, maybe to AWS, maybe to an in-house operations team. FaaS draws that line roughly as far right as I think a general purpose programming system can; further right would approach things like Excel, Retool, etc.

MathMonkeyMan 5y ago
I think the answer lies in your last paragraph, but your comment made me think: Perhaps Lambda functions are resilient because they're: - scope constrained - surrounded by APIs only (e.g. no cheating by having a whole POSIX system) - infrastructure is maintained solely by a giant company that you pay to not have to think about it I haven't used any "serverless" stuff, so I don't really know. Per the article's recommendation, I have noticed at work that resilient software is often a 400 line python script with no other dependencies; or, a medium sized piece of C++ infrastructure that depends only on the OS, the compiler, and a hand-written build system.
- Hendrikto 5y ago

deleted 5y ago

[deleted]

brian_herman 5y ago

I noticed on one of the files he uses lxml, that is not part of the standard library is this one of those do as I say not as I do? https://github.com/jeffkaufman/webscripts/blob/b6a004790b46402f94e4172775b7dd9676775bd4/makerss.py#L38 https://github.com/jeffkaufman/webscripts/blob/b6a004790b464...

jefftk 5y ago
When I started this script ~12y ago there wasn't any good html parsing option in the standard library, and that was worth taking a dependency for. I don't see a conflict there with "minimize your dependencies and limit them to ones that value backwards compatibility"?

ChrisMarshallNY 5y ago

I've written SDKs that have lasted for decades. Number of reasons why. Most don't stay static, though. People mod them. They need to be something that can be handed off to a new team. I find documentation is key. I write about that here: https://littlegreenviper.com/miscellany/leaving-a-legacy/ https://littlegreenviper.com/miscellany/leaving-a-legacy/

bogdanoff_2 5y ago

I wonder if there's a way to do server-side javascript programming with this philosophy. For client-side I can just throw together some html and javascript to make a simple app. But if I want to use javascript on the server it seems like I need to download a bunch of npm dependencies to do anything useful.

WA 5y ago
Sure, you can do it on the server with Node. Node has `http` as part of its standard library and you can write a simple webapp without third party dependencies. Check this tutorial for example [0]. The problem is that Node's http lib isn't super productive and you gotta write some "solved" stuff like cookie handlers. Also, there is no database standard library. If you want to persist data, you could use files. If you need a database (even SQLite), you gotta use third-party stuff. In comparison, PHP has a lot of stuff build-in and you can be more productive with zero dependencies in PHP. But then, maybe it doesn't really matter if you include a well-maintained npm package or if you have a more powerful standard library that has some features deprecated after a while. It's still "other people's code", although npm packages require some kind of installation, whereas a standard library usually doesn't. [0]: https://www.section.io/engineering-education/pure-node-js-no-frameworks-or-packages/ https://www.section.io/engineering-education/pure-node-js-no...
- tored 5y ago
  There was a trend a few years ago (maybe still going) that languages with a large standard library, like PHP, is bloated. That we should strive against minimalism and then just share dependencies when needed. But then the nodejs left-pad debacle happened. I used to be more on the minimalist side, but over time I have switched to the opposite, today I can really appreciate the bloated nature of PHP, everything from large projects to small scripts just becomes easier to handle.

Karrot_Kream 5y ago

I follow a lot of these practices myself for hobby code, but if I had to offer one change, it would be to use SQLite instead of the filesystem for data. It does add some complexity in languages like Go that don't "easily" link with C libraries, and you do need to remember the table schemas that you're using, but for me the benefits are worth it. I never worry about locking files, naming some key badly, or any of the other issues that come with flat file dbs.

bob1029 5y ago
SQLite is an absolute powerhouse. Filesystem interop is the best way for it to start proving itself in any codebase. Loading assets for a AAA game straight from disk? You would have a really hard time doing better than SQLite - especially if you have millions+ of unique items to contend with. In many cases, SQLite is faster than the actual file system it resides upon [0]. We use SQLite for nearly 100% of business object persistence in our product today. The performance & scalability arguments fall on some very deaf ears after years of experience optimizing and succeeding with this stack in production. [0] https://www.sqlite.org/fasterthanfs.html

phendrenad2 5y ago

Good advice for personal projects. Sadly you can't use a lot of free libraries out there, because more often than not you'll be forced to upgrade (possibly due to a security vulnerability that the author refuses to backport to "old" major revisions), and encounter a backward-incompatibility when upgrading.

TedDoesntTalk 5y ago

> This is not how I approach design at work, or necessarily how I would recommend doing so. Having a budget where initial creation is essentially free (fun!) while maintenance is extremely expensive (drugery!)

titzer 5y ago

The first backend that I achieved self-hosting of Virgil was the JVM backend. I targeted version 5 classfiles. These classfiles are still accepted by JVMs but there have since been several significant additions to the format. In particular, classfiles are required to have stackmaps, which are pretty much a major distraction for me to add to my backend (so I haven't!). The second backend that I bootstrapped on was x86-darwin. That worked great for about 7 years, until Apple decided to deprecate first 32-bit, and now, I guess, x86 altogether. Thankfully I ported to x86-linux soon after, as that syscall ABI is solid as a rock. But you know, things keep changing. To keep up, I finally had to bite the bullet and write an x86-64-linux backend. And I guess at somepoint now arm64 if I ever want to work natively on a mac again. It's an endless cycle. Thankfully now, emulators are pretty good, but I am still struggling with getting QEMU to run fast on MacOS on the M1, so it's just more futzing with enormous emulation stacks just to run old stuff.

Datenstrom 5y ago

Beautiful code denies its existence so perfectly you forget you wrote it.

gfxgirl 5y ago

> The biggest piece is minimizing your dependencies, and limiting them to ones that value backwards compatibility Three.js, one of the most popular JS libraries, has the explicit policy of ignoring backward compatibility. https://github.com/mrdoob/three.js/wiki/Migration-Guide https://github.com/mrdoob/three.js/wiki/Migration-Guide I find it not only irresponsible but arguably mean to other devs time and lives but apparently I'm in the minority. Every time I have to spend an afternoon or evening updating stuff instead of doing something new or visiting friends etc I silently curse the devs

jefftk 5y ago
You curse the devs who build excellent libraries and give them away for free, because they do not accept the additional burden of never making changes that would require you to adjust your usage? You could choose not to use such libraries, but as long as they're up front about it I don't see how you can hold it against them.
- gfxgirl 5y ago
  it upsets me because they are the only library of 100s I use that do this. They consciously break things every month. they don't give a fuck how many man hours it burns of other people's time. No other project I know of does this. It's not just time using the library. Every volunteer that wrote docs or a tutorial their work is now out of date. every answer on S.O. is more than 3-4 months old is wrong. Every new user now has the burden of trying to learn where any content they find is practically guaranteed not to work correctlt with the current version. it's mean in the extreme to piss on so many people
fulafel 5y ago
I'm sure you know this but for the record, they don't actually ignore backward compatibility, it features often as a concern in the dev discussions and they're conscious of when they break it, and there can be deprecations[1].. it's a conscious tradeoff allowing development to go faster. I'd argue Three.js beats native to make low upkeep & long lived end user (=portable) 3D apps, given all the churn, constant driver breakages necessitating new workarounds, proprietary-ness and fragmentation in 3D APIs. With Three you update to newer versions on your own time (if ever) and stuff doesn't break from under you. And in the decades long view WebGL 1 / WebGL 2 / WebGPU is used behind the scenes. [1] example: https://github.com/mrdoob/three.js/pull/21777 https://github.com/mrdoob/three.js/pull/21777

AtlasBarfed 5y ago

Well, it probably starts with a stable secure cross-platform language and ecosystem. Things like this are best explained by a very grey neckbeard who's been through a lot. Author is not particularly old, author programs in python (something that has broken compatibility once)... "stick to the standard library" is about at ground level an opinion on this subject that a typical HN probably thought of as they clicked the link. 1) need a stable, mature, secure language Stable eliminates Rust, Python, Perl. Go hasn't been around long enough. Secure eliminates C, C++, probably D and most compiled languages No dependencies: yikes there goes Java because of the JVM dependency. Assembly isn't portable. ...Seriously, are we left with Ada? Secure, strict, mature, reasonably portable. And virtually unknown outside of government/military. 2) UIs age, but CLIs really don't So make it a CLI, and use unix philosophy 3) OS churn Hopefully with the practical cessation of gigahertz scaling, the industry will settle down. Then again, the ARM hordes are approaching to wipe out all the x86 stability in desktops and servers. Perhaps the best thing for this in the industry has been Windows subsystem for Linux. If we could get an OSX subsystem for Linux, that would be a real boon. Even Linux on it's umpteenth release is still changing quite a lot, and NIH reinvention like Wayland and Systemd and toss-it-all-out Gnome 3. Well, it kind of keeps us employed. I'd adhere to assuming a unix-ish os, like maybe POSIX. So: Ada, CLI/text, POSIX compatibility. That's not limiting at all, is it? Sigh.

jhgb 5y ago
> 1) need a stable, mature, secure language > Stable eliminates Rust, Python, Perl. Go hasn't been around long enough. > Secure eliminates C, C++, probably D and most compiled languages > No dependencies: yikes there goes Java because of the JVM dependency. Common Lisp, then?
- AtlasBarfed 5y ago
  Almost, but Java has a VM. Yes you can technically compile Java to an executable but it's not done much. CL has a VM too, does it not?
  - jhgb 5y ago

anyfactor 5y ago

I think the idea of we need continuous improvements to maintain a steady userbase is a myth. And it is disturbing to me that we talk about browser support and the global reach internet services yet I struggle to access many popular websites on my galaxy S2.

karmakaze 5y ago

The most important things I've learned is that the system has to be (1) self-maintaining (e.g. security updates, rotate logs, alerting monitors so safe to ignore), (2) super easy to make and deploy a change. This one consists of a README that word-for-word works so you don't have to think. Minimizing steps with push-to-github auto-deploy is great, as is an easy setup for a hot-reloading local dev. Following those two points, I've been able to use whatever languages/tools tickle my interest with little impact on maintenance effort. Dependencies are few, but not minimal, pinned to exact versions. FYI, these have run the range of Vue/TypeScript, Node, Go, Java/Kotlin, Clojure, F#, and Php, with typically MySQL (sometimes multi-master).

shoo 5y ago
> (2) super easy to make and deploy a change. This one consists of a README that word-for-word works so you don't have to think. Minimizing steps with push-to-github auto-deploy is great Great advice. Minimise the energy barrier to being able to ship a patch or a new feature after neglecting the service for months/years and forgetting how you set everything up. I always have a bit of a laugh after patching my hobby project and doing a handful of iterative automated deploys while making small changes, then switching back to the day job where aspiring to do a single deploy will typically require hours of meetings and general bureaucratic overhead. Another possible tip is: understand your deployment environment and learn how to take advantage of what it offers out of the box without adding additional layers or dependencies. E.g. I deploy a hobby webapp to my own debian server. Debian comes with apt for package management and systemd for managing services, so I use both of those (yes, even apt for python package dependencies).
- karmakaze 5y ago
  Totally agree. Systemd .service unit files are super simple to understand and setup. Well worth the small time investment.

unknownOrigin 5y ago

Designing for minimum maintenance is nice when you're working on something for "yourself"... not so much if you're creating something for someone else. Since designing for minimum maintenance takes additional time (and money) and most stakeholding entities (contractee, employer) will want the job to be done as soon as possible (and only THEN worry about upkeep), this can easily prevent you from "doing the right thing" simply by the nature of competition. Not to mention the elephant in the room - that minimum maintenance often means less money for the person doing the maintenance, i.e. "you". Vendor lockin is an issue only if you're not the vendor.

cblconfederate 5y ago

php+mysql has been low upkeep for at least 15 years is that enough

milofeynman 5y ago

An interesting take-away from this is designing low upkeep business services. I have been dealing with trying to onboard to a service at work, and there are so many dependencies! Several are pinned to versions that require python2, but I can't install python2 because another service I use calls "python" and assumes it's linked to python 3! And sometimes someone will add a new feature, and include an entire library for that single method call, instead of trying to write a little JavaScript. I almost think there should be a non-trivial process at most businesses for including new libraries so that web devs would try to do it without adding a new library, so they didn't have to deal with red tape!

forgotmypw17 5y ago

Lindy Effect says that software will continue working for as long as it's already been working so far. I use this to my advantage by writing software AS IF it has already been working for 20 years, meaning I use only languages and technologies which have not introduced breaking changes in 20 years or so. It's worked out quite well so far!

est31 5y ago
Which language or technology has not introduced breaking changes in the last two decades? Windows has good backward compatibility, but even Windows can't run really old programs. C++ has evolved dramatically in the last 20 years. Even C had changes. Do people still use tables to create websites, or jquery?
- forgotmypw17 5y ago
  Now, to be fair, I'm talking about carefully writing code with the knowledge of the previous 20 years, which is a little bit like "cheating", whatever that concept means here. I use ASCII/textfiles, HTTP, CGI, SSI, access.log, Perl, Bash, and small subsets of PHP, HTML, JS, CSS, and almost no third-party libraries. Certainly no JQuery or anything similar. I'm not hardcore enough for C or C++, but I know that a program written 20 years ago in these languages would most likely compile today.
  - aahortwwy 5y ago
    > a program written 20 years ago in these languages would most likely compile today Not without continuous attention over those 20 years. I'm currently struggling to get a C program last touched 10 years ago to compile.
    - forgotmypw17 5y ago
      Like I said, I'm cheating a little bit, and using the knowledge I have today to "retro-edit" this hypothetical 20-year-old program.

tomerbd 5y ago

Sounds good for personal projects.

oftenwrong 5y ago

On this topic, what happened with https://www.jefftk.com/nextbus/ https://www.jefftk.com/nextbus/ ? Seems to be a 404 now. Was there a breaking change or disruption in the nextbus API? I loved its bare-bones UI. It was definitely one of the most usable webapps. I assume its a11y was nearly perfect as well. If anyone would like to see it, here's an archived snapshot: https://web.archive.org/web/20170622033104/https://www.jefftk.com/nextbus/ https://web.archive.org/web/20170622033104/https://www.jefft... I also admire https://diskprices.com/ https://diskprices.com/ for the same reasons.

tored 5y ago

Few weeks ago I wrote a lengthy post here on HN about how to write low upkeep software in PHP https://news.ycombinator.com/item?id=28232146 https://news.ycombinator.com/item?id=28232146

6510 5y ago

For fun personal projects I like using php to manipulate static html documents. I have quite a few projects with broken admin areas written in ancient php but (besides from user contributions) the html/css/js/xml/json just continues to work as if nothing happened. Sometimes I just delete the admin tools.

XCSme 5y ago

I am not sure writing your own build tools is the way to go, if there already exists something that does that thing even better.(as the author mentioned, he wrote a templating engine that updates all static files when a shared component is edited)