9 ms·
In Pursuit of Production Minimalism
- 1996 8y agoThe list of suggestions do not match the headlines. Introducing new technologies to replace old ones, preferring services to local hosting - I just do not agree! I use the oldest technology I can get away with, and host everything on dedicated servers as it is often more performant and cheaper. Simplicity needs bounds. The suggestion of reuse also needs that: If you end up storing binary content on SQL to avoid having to roll a file hosting, you are taking the simplicity mantra too far.
- meesles 8y agoAgreed, minimalism and age/maturity do not correlate in any meaningful way. It's much too circumstantial. In some cases, running a time-tested Java app might be much easier than spinning up a modern JS equivalent. In other cases, maybe the legacy options don't provide the functionality you need. Not to mention contradicting ideas: "Retire old technology" and "Don’t use new technology the day, or even the year, that it’s initially released" What is old technology? You're on a UNIX system, right? 0_0
- shuntress 8y agoI think the "retire old technology" is more in the context of recognizing when something you are thinking about adding could replace something you already have.
- grey-area 8y agoIntroducing new technologies to replace old ones I took that to mean if you must start using new solutions, make sure you remove some similar tech from your stack for every new solution you start using (e.g. replace RabbitMQ with Kafka) so that you don't end up with 50 ways to store data say. That was certainly the context in the article. preferring services to local hosting There is an argument for using hosted services too though - it reduces the ops workload - the correct solution probably depends on scale, staff etc. I agree this isn't one size fits all. If you end up storing binary content on SQL to avoid having to roll a file hosting, you are taking the simplicity mantra too far. I don't think the article mentions storing binary content in Postgresql does it? They're just talking about all non-ephemeral data, I imagine at one point they started using some data stores better suited as caches or message queues for permanent data storage, and discovering that when data was lost was the impetus for simplifying their storage - I like that they've moved to using their own hosted Postgresql solution to store their customer data, it simplifies their infrastructure, and makes sure they feel any pain customers feel too - a good kind of simplicity to strive for.
- whitepoplar 8y agoI really enjoy old, stable tools that are maintained and made incrementally better, day by day. Postgres is one example, Elixir is another (which doesn't re-invent the Erlang VM, but builds on top of it). I also really like hosted cloud services, but they need to be replaceable with little to no effort. For instance, I don't want a new NoSQL platform, I want high-availability, managed, on-the-fly-scalable Postgres.
- abraae 8y agoUnfashionable I know but I feel strongly like this about Java. We chose Java maybe 2 decades ago when it was newly minted. Some decent swathes of code written back then are still running in production. Its enormously satisfying looking at anything in your stack which has really stood the test of time. I harbour similar feelings about the Oracle DB but I'm a little more conflicted there.
- mmt 8y ago> need to be replaceable with little to no effort. For instance, I don't want a new NoSQL platform, I want high-availability, managed, on-the-fly-scalable Postgres. I don't think the latter is easily replaceable, though, either, since what you've described isn't Postgres, but, at best, something custom with a Postgres interface. For example, the Citus DBaaS offering may fit the bill, but what could easily replace it? A self-hosted version would fail at being "managed" (though this is a bit academic, since, presumably one would hire for that) and at being on-the-fly-scalable [1]. Is Amazon RDS a drop-in replacement, or does it have compatibility and performance edge case caveats? [1] I read that sort of "wish" often and have always found it borderline naive. That is, what I think they really mean is they just don't want to worry about scalability. The dominance of expensive, few-sizes-fits-all cloud infrastructure means that custom building an over-engineered database server, is not a practical option, and for most cases, neither short-notice scalability beyond that, nor runaway growth are a realistic problem.
- radicalbyte 8y agoStoring binaries in SQL works great for small scale systems; it makes the backup/migration process simple.
- briandear 8y ago> host everything on dedicated servers as it is often more performant and cheaper. It depends on what you are serving and at what scale as well as the opportunity costs of managing it yourself. If I have a developer spending some hours maintaining a server each month, it isn’t cheaper at all. You can also have issues of under or over capacity. Heroku, for example saves us a ton of time we could be using for shipping features rather than maintaining infrastructure. If really depends on the application.
- wvh 8y agoI think the "libraries over frameworks" adage definitely should be extended to SAAS as well. Use the tools you can get away with, but with an eye on the future and without painting yourself in the corner. Sometimes convenience makes us give up too much control and know-how. Of course the article is about a PAAS company, so the advice ought to be taken with that in mind...
- zerotolerance 8y agoMinimizing what code you own is a naive optimization. When you consider the liabilities that third-party library and service dependencies introduce it becomes quite attractive to build your own minimal systems. I'm not sure when programmers became so averse to writing simple code. But leftpad was just the beginning.
- paulddraper 8y agoBuild vs Buy has no universal answer. Do you build? Or do you buy into the support level, the features, the bugs, the availability, the security, etc.
- sbjs 8y agoMy rule of thumb is, write it yourself until it's painful, and spend a day looking for a good library to reduce the pain. If you can't find it after a day, or if you found one but it's taking longer than a day to configure it for your needs, try to write one yourself. This rule has not done me wrong yet. It's how I ended up moving away from Mobx and Redux in favor of RxJS and setState (just submitted a post about how to do this), and why I moved from Vue to React. But it is very subjective and subjectivity gets harder the more people you have on a team working on the same code.
- zerotolerance 8y agoI've actually come to prefer the opposite. I'd argue that the liability associated with third-party deps makes them debt. Debt is great for growth and experimentation. It works in a pinch especially if the experiment fails and you can just toss the dependency. So, today I'd rather buy, then build when I know I want to take on the pattern for better ops and long-term ownership.
- darkerside 8y agoThis is the top comment, but I'm confused about how it relates to the linked article. The article doesn't really advocate outsourcing your software to third parties, just consolidating the infrastructure that you do own (sometimes in clever ways, like recursing your own PaaS) to make it as simple as possible.
- mLuby 8y agoFantastic suggestions. Not only does having fewer technologies in your stack mean there're fewer to maintain, it also means they're used more so your devs are more expert at them.
- maxxxxx 8y agoThese are not bad. But "Use services" seems like a recipe for long term pain. It's an external dependency you can't control. I would prefer to use the external service to learn what you really need and then try to insource it again.
- mercwear 8y agoThis article mentions Clarence "Kelly" Johnson. If you don't know who he is and care to learn more about one of the best engineers that ever lived (SR-71 is his work), check out his book: https://www.amazon.com/Kelly-More-Than-Share-All/dp/0874744911 https://www.amazon.com/Kelly-More-Than-Share-All/dp/08747449... His second in command at Lockheed, a man named Ben Rich also wrote a very good book: https://www.amazon.com/Skunk-Works-Personal-Memoir-Lockheed/dp/0316743003/ref=pd_lpo_sbs_14_t_0?_encoding=UTF8&psc=1&refRID=24ZEA753MP8VAPZAA7KC https://www.amazon.com/Skunk-Works-Personal-Memoir-Lockheed/...
- ThenAsNow 8y ago> (SR-71 is his work) One unfortunate trait of the original Lockheed Skunkworks culture was to make Kelly Johnson (brilliant as he was) the only face of the organization. As just one such example, many know of the clever inlet system on the A-12/SR-71, but it, like many things Skunkworks in the pre-Rich era, is incorrectly ascribed to Johnson. The real person who deserves the credit as the chief for that system is David H. Campbell, but how many people know his name?
- theoh 8y agoThe form of the T-38 was apparently determined by Lee Begin at Northrop. Who, you ask? Well, good luck finding information about him on the Internet.
- hirundo 8y agoIf you remove everything that doesn't break the widget then everything left is functional. Pursing minimalism equals pursuing function. Therefore engineering equals applying minimalism to technology.
- nine_k 8y agoThat "doesn't break" needs an "under which conditions" clause. A lot of complexity arises from having to handle the imperfections of the environment. But I agree with the general idea. Good design usually arises after a worse, messy and more complex early design is produced, analyzed and simplified away, often radically. Coming to simplicity by a shorter route is very rare.
- chrisweekly 8y agoYes. "Perfection is achieved, not when there is nothing left to add, but when there is nothing left to take away." - St Aubrey(?)
- nine_k 8y agoAntoine de St Exupery; quoted in the article :)
- 0xbadcafebee 8y agoFwiw, the SR-71 was not engineering excellence. It was slapped together to do what needed to be done. It literally leaked jet fuel onto the ground when it wasn't in the air, in order to allow the metal body panels to literally stretch and close gaps in its skin when it was in flight (because physics). The air frame couldn't handle a takeoff with a full fuel tank, so it had to be filled up right after taking off. 90% of the parts manufactured didn't work. It was difficult to maintain and eventually developed a "speedy" one-week turn-around time to get it to fly again after a mission, assuming a base had the specialized tools and support staff to do so. The SR-71 is an example of how the sheer will to get something done can indeed accomplish great feats, if at great cost and complexity. It's also an example of how innovation is no guarantee of success; since the Blackbird was retired, we still have no airplane as fast as one built in the 1960's. A great plane is just what it needs to be. It may not be the most minimal it could be, but it will be the most effective. Another example, the A-10 Warthog, is not "minimal", but it is an amazing plane. I think to get the best result based on the desired outcome, you have to pursue craftsmanship and excellence through continuous education, practice, and improvement, and not worry how it looks so much as how well it works. Complexity is a sin, except when it is necessary.
- maxxxxx 8y agoAgreed. I have read a bit about Skunkworks and they are really good at building planes quickly but the SR 71, the U2 and F117 are all very difficult to.maintain and/or tricky to fly. Not suitable for mass production and use.
- tonyarkles 8y agoIt’s funny you mention the U2. If I’m remembering right, the U2 went from idea to flying in 18 months. That is my benchmark for what 18 months of development by a talented team looks like. And yes, you’re right, it’s not an airplane for mass production, but it filled a niche and did a damned good job of it.
- thatfunkymunki 8y ago
- simonw 8y agoThe rule of thumb that I go by is that in order to add a new technology to a stack it can't just represent an incremental improvement: it needs to be a 2x improvement in productivity or (even better) it needs to make it possible to build something that isn't possible to build without it. The cost of introducing new technology (in terms of training, tooling and now-you-have-two) is so high that it's just not worth it if it will only incrementally improve how things work at the moment.
- toast0 8y agoI agree with the thrust, but these need more nuance: > Avoid custom technology. Software that you write is software that you have to maintain. Forever. Don’t succumb to NIH when there’s a well supported public solution that fits just as well (or even almost as well). Software you run is software you have to maintain; until you can burn it in the fires of Mordor. If there's a public solution that is a good fit for your problem, and was built along the same lines you would build it, then that's great; but if it's a great fit for the problem and also pulls in 70 dependencies and updates three times a week, that's probably not worth the integration mess. > Use services. Software that you install is software that you have to operate. From the moment it’s activated, someone will be taking regular time out of their schedule to perform maintenance, troubleshoot problems, and install upgrades. Don’t succumb to NHH (not hosted here) when there’s a public service available that will do the job better. You don't need to host everything, but you better have a backup plan when the service you depend on decides that they're no longer going to provide the service. Chances are, you're going to need to debug the service too, sooner or later, and it's harder to debug a black box. That said, it's great when you can lean on a rock solid service to do things that are important, but not a 'core competency'
- pcmaffey 8y agoYup, also "publicly supported solution" often means not perfectly customizable, and may only be publicly supported for a limited time. I find that most services get me 80% of the way, 2x faster than a custom solution. But the final 20% is an unknown abyss that will more often than not, have insurmountable limitations. And so, if the software is in my wheelhouse, I'm almost always better off building it custom.
- 0x8BADF00D 8y ago> Software you run is software you have to maintain; until you can burn it in the fires of Mordor. If there's a public solution that is a good fit for your problem, and was built along the same lines you would build it, then that's great; but if it's a great fit for the problem and also pulls in 70 dependencies and updates three times a week, that's probably not worth the integration mess. I’ve gotten flak for rolling my own library. Basically, I was trying to get the base clock rate of a server for benchmarking purposes. Instead of using an external library, I simply wrote some inline assembly to read from the timestamp counter. I was told this will create a maintainence headache, but zero fucks given on my end. I got to write some inline assembly today. :)
- deleted 8y ago[deleted]
- stockkid 8y ago> Use services. Software that you install is software that you have to operate. On the other hand, a software that you use as a service is a software you do not control. Therefore whether to use services depends on how mission critical the software/functionality is to the whole system.
- tharakam 8y ago"It seems that perfection is attained not when there is nothing more to add, but when there is nothing more to remove." - Antoine de Saint
- jonahx 8y agoThough less literally accurate, I prefer the translation: > Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.
- pdog 8y agoLooking at the SR-71, despite all its engineering "flaws" (documented excellently in a sibling comment: https://news.ycombinator.com/item?id=17675996 https://news.ycombinator.com/item?id=17675996), I wonder... Have we really progressed technologically, as in, doing the impossible, since the late 1960s? The Blackbird is still the fastest jet in the world more than fifty years later. I'm not even sure we could return to the Moon successfully on the first try if we tried.
- Barrin92 8y agothe disparity between progress in the 'world of bits' compared to progress in the 'world of atoms' really has become quite stark. I would agree with you that there even seems to have been a regression in some domains. We've lost an engineering culture that connects to the physical world and real materials we deal with every day. Another good example is probably the kitchen. A kitchen mid 20th century looked very different from a kitchen at the beginning of the century. Kitchens nowadays really hasn't changed that much for a very long time. Dan Wang has written a very good (albeit lenghty) essay on the topic: https://danwang.co/how-technology-grows/ https://danwang.co/how-technology-grows/
- skybrian 8y agoMaking a plane faster is only one dimension for optimization and it's not actually a goal in itself. It sacrifices other goals. Nobody has done better because they are optimizing for something else. The purpose of the SR-71 was reconnaissance. Do you really think the U.S. military hasn't gotten better at that, using drones and/or satellites?
- varjag 8y agoSR-71 was meant to be flying over USSR and China. Few existing USAF drones would have survived 20 minutes there even in 1960s. But sat reconnaissance have certainly got a lot better to the point of rendering the bird obsolete.
- skybrian 8y agoIt's interesting how often good-sounding advice is contradictory. The article advocates standardization. Using fewer programming languages means fewer toolchains to maintain and less fragmentation of engineering knowledge. On the other hand, the frequent advice that we should "use the correct tool for the job" suggests being familiar with a lot of different, specialist tools. More programming languages? I guess it all depends. Knowing when to make a move versus when to do the opposite comes down to taste and experience.
- castlegloom 8y agoGreat article, classic comment section: Half of people arguing about the plane analogy Other half arguing about jQuery
- dionna767675 8y agoMake money online Internet marketing opportunity plus try our marketing courses learn how to run a business from the ground up plus make money online get your business certificate low affordable payment plan real legitimate company Internet marketing opportunity the proven online destination for small business consulting and tools for small business owners and entrepreneurs for more information click below to join https://internetmarketingsolution.club https://internetmarketingsolution.club
- contingencies 8y agoSome good architecture related quotes here. Added to http://github.com/globalcitizen/taoup http://github.com/globalcitizen/taoup
- dionna767675 8y agoFree free free to join this is a real money making opportunity get paid weekly by PayPal free to join click below for more information when you become a free member watch all the tutorial videos join the amazing facebook group learn step by step training from the best of the best http://souniqueincome.club http://souniqueincome.club
- mark_l_watson 8y agoI really like managed app hosting like AppEngine and Heroku. I have spent decades doing what we now call devops, or ‘you build it then you own it.’ Now I like to save my time for other things. AppEngine wins on cost while Heroku wins in that if you use standard things like Postgres, Kafka, etc. it is not too bad moving to your own dedicated services. Edit: I also like the article’s advice on minimizing the number of components in systems. Using Postgres as a ‘Swiss Army Knife’ for information storage is a good start.
- edpichler 8y agoGreat article. I always tell to my clients: Software projects are cheap, the expensive is the never endless Maintenance.