33 ms·
Buy Don't Build
- lawwantsin17 6y agoMore nonsense for non-tech founders posing as nonsense for tech founders. All of these points leave out developer happiness and buying just does equal good knowledge. It equals Googling around a black box.
- jvanderbot 6y agoBuild the secret sauce, buy the packets.
- philjohn 6y agoAlso worth noting that "buy" in this context can also mean using a piece of open-source software, not just buying something commercial.
- didip 6y agoI agree. If you are in decision making position, I struggle to see why would you want to spend so much energy installing Prometheus + Grafana stack instead of just using DataDog. Even worse if your company attempts to build Prometheus/Grafana from scratch. Just why? What a huge waste of money. If it contributes to your core business then ok, I can see why you may want to build custom solutions.
- rantwasp 6y agoi think there is a huge difference between using prometheus and paying for datadog. the article argues for not building prometheus which makes sense in 99.95% of cases. btw, the build vs buy should be a topic of discussion whenever people want to build something. at an absolute minimum they need to understand (and use) what’s already out there before they embark on the journey of building themselves
- YetAnotherNick 6y ago> so much energy installing Prometheus + Grafana stack instead of just using DataDog. Because it could be done in few commands the second time. And learning is only possible when wasting time on this kind of things, rather than trying hard to custom fit your usecase using some buyable tool. That being said, I kind of partially agree with you and the author though. Don't maintain service just to feel that you are in control, but with the exception that if you are doing it for the learning purpose.
- flumpcakes 6y agoOn premise Prometheus/Grafana could be lower than 5% TCO compared to DataDog, at least at places I have worked, and I am including employee time. This isn't particulary a discussion about cap ex. vs. op ex. either.
- lhopki01 6y agoInteresting you should bring this up as we are currently in the process of moving from Datadog to Prometheus/Grafana. What we've found is all the time you save in not setting up Prometheus we lose in support tickets to Datadog to get them to explain undocumented functionality and hidden "magic" they put in place. With Prometheus/Grafana the answers are always out there because of the sheer number of people out there. We've even resorted to sending known metrics on known intervals just to understand what Datadog does differently for monitors vs graphs. Also the majority of the time for a monitoring system is not spent on setting it up but rather on generating the dashboards and alerts. Also with all buys systems there is a hidden costs in managing the legion of user accounts to access these new services. It's rarely as easy as just setting up SAML or single sign on. The final thing that people never talk about is cost. Yes there is the cost of an engineer to set it up but often that costs is cheaper than the bought service. This is especially true of pay per api call type services.
- phsource 6y agoI'm totally on board with the overall message that building (i.e. engineering) your own internal tools come with lots of overhead, but take issue with this one point: > My counter-argument to that is there is also lock-in with internal systems. The most common version of this is the keeper of the spreadsheet. The author then disparages spreadsheets as becoming the exclusive domain of one employee who wouldn't want processes to change. In reality and my experience, though, spreadsheets are one of the most versatile and accessible systems, and their close cousins (Airtable, Notion) great as well! You can customize it to your own processes and they're pretty universally understood, so the barrier to change is pretty low.
- jrott 6y agoAuthor here I actually think spreadsheets are great because they are so accessible. What isn’t great though is when there is a business processs that is a spreadsheet and knowledge that exists in one persons head.
- holri 6y agoIt is a management failure when this is in only one person head. Outsourcing tools will not solve internal management problems.
- giantDinosaur 6y agoThe issue with excel is that there's no standard construction that anyone can query; most likely it's extremely hacked together since it's not a database, but made to look like one. Spreadsheets are great for doing stuff related to exports or before importing, but so, so often they lead to data islands built to either actively or passively protect someone's job.
- rantwasp 6y agoi don’t think it’s the spreadsheet itself. it’s about the process and people who defend the process no matter what
- 6y ago
- cortesoft 6y agoAll of this is true, but sometimes people forget that buying a solution doesn't mean there will be no work. Sometimes, the integration and maintenance of that integration ends up being more than building and maintaining your own solution, especially if you have to do any large changes to how your systems work in order to integrate.
- victor9000 6y agoWhile I agree with this comment, if someone buys and ends up in the position you describe then they did a poor job at buying.
- dnautics 6y agothere may be no options. For example, I (profitably) ran a small cloud. My supervisor kept questioning me "why don't we just use openstack instead of your software"? The answer is even in this article! > Most enterprise systems require an engineering team to keep them running. If we wanted a team of 3+ to run the cloud then we could buy openstack, or cloudstack, probably 2+, but that's also without pushing features. And suddenly we wouldn't be profitable anymore. I left, so I guess they will find out.
- jeromenerf 6y agoNot always a poor job, but sometimes a bet that turns out wrong. I have seen great products and companies disappear in unforeseen mergers or API prices skyrocketing for instance. Risk is low for commodities, but high for anything "disruptive".
- JMTQp8lwXL 6y agoThe people that have to perform the technical integration between two independently developed products (and could warn that the integration work is more involved than starting from scratch) are a different group from who decides the acquisition is happening.
- aeternum 6y agoNowadays you're often buying SaaS so not only do you need to ensure the product is a fit currently but you must predict the fit going into the future. It's not an easy thing to do. I've found that open source libraries are often more stable, require less ongoing maintenance due to API changes and have better support lifetimes compared to the SaaS equivalent.
- jrochkind1 6y agoGood article, I am definitely filing away to pull out in some future team discussions. This paragraph is confusing me, either it's missing a 'not' or something, or I'm just confused. Is anyone following? > The question you should be asking is what else could be done instead of tuning your own stuff or building a new internal system. The answer is usually spending more time coming up with the correct architecture instead of fighting fires or developing actual customer-facing features. Instead of building internal systems you would be coming up with the correct architecture instead of fighting fires or developing actual customer-facing features? Wait, what? there are too many 'instead of's in there and i'm not sure they are all meant how they are said. I think developing actual customer-facing features is the goal, but here it sounds like the thing you would like to avoid... and I'm not really sure about fighting fires (ideally you want a system where you do less of that, right?) or coming up with the correct architecture (?).
- jrott 6y agoYeah developing actual customer facing features is the goal. That paragraph needs to be edited
- deleted 6y ago[deleted]
- blakesterz 6y agoI think he's right, if you're at a small/medium sized place, all the reasons he lists there are solid reasons to buy and not build things for yourself. That being said, there's no reason to make that an absolute rule, I think it's ok to build some things and buy some things. Just think of all the things we've ended up with because someplace built some thing and it ended up being some big thing that we all know. Isn't that how Slack and Docker both got started?
- whoisjuan 6y agoI own a Screenshot API service (shameless plug: https://getscreenshot.rasterwise.com/ https://getscreenshot.rasterwise.com/) and this exactly the argument that I make for small utilitarian services like mine. You shouldn't build them because buying them is several orders of magnitude cheaper. I have a small paragraph from a blog post where I explain this with some math that is probably crappy but captures the idea: "When you spend time in areas that are not the core of your product, you're actually being financially inefficient. Taking screenshots is likely not a core task of your business so it doesn't makes much sense to waste development resources in this area. Here is a brief example of how inefficient it gets: A mid-level developer in a small market earns $90K USD per year, working 40 hours per week. His/her effective hourly rate is $47 USD per hour. Writing a basic implementation of a screenshot utility will take at least 5 hours. But this is just a simple prototype. Many use cases will need to be addressed, and there's likely going to be a large amount of time spent in optimizing, securing, provisioning, testing, etc. A realistic utility that can be used for a development workflow is going to take at the very least 30 hours of development time, but likely more. Other un-accounted areas of development time are: documentation, training and maintenance. Given this scenario is very likely that writing a semi-good solution will take more than a week and maintaining it, will take at least an hour every month. This means that writing this solution will cost almost $2000 USD of development time plus another $500 USD or so, just to support it every year. And of course, this doesn't include the cost of the infrastructure you're paying to run it." My argument is that the more utilitarian the service, the more cost-effective that is to buy it instead of writing it. Writing it could be simple but there's a drain in engineering resources when you try to maintain it in the long run.
- lexicality 6y agoOne of the few things I appreciate from my abusive first boss was making me acutely aware of the value of my own time. He was incredibly stingy with money and always wanted to know how much it would cost him to have his engineering team implement something vs how much it would cost to buy it and integrate it. Admittedly he was pathologic about this to the point of having us create a timer that counted in pounds sterling that he'd start running at the start of every meeting, but I feel like it made me way more efficient with my time (and hate long meetings)
- human 6y agoI had a similar client early in my career. Working for him was a love/hate relantionship. He tripled my hourly rate telling me my time was highly valuable but then had very high standards and aggressive timelines.
- FpUser 6y agoI think you had great boss (assuming he did not start timer each time you stopped typing on keyboard). Those endless meetings become unproductive pretty fast. I suspect that in big part they become a self sustained cancer used to justify time and salaries of unproductive company members. As for the subject itself: frankly I do not see any advantage of hosting on say Azure over hosting myself. Either requires a good deal of maintenance. And no you can not really rely on Azure doing it for you. Uptime for me is probably just as good on my servers as each one I have in my office also has ready to use up to date shadow copy located elsewhere. Azure is way more expensive of course. The amount of hardware/processing power I have on my own server would cost me a fortune to have on Azure. Scalability does not matter much either because: I am not Google and do not have to serve the rest of the world. On top of that my servers are usually high performance native C++ applications that can handle thousands of requests per second sustainably and without breaking a sweat. Vertical scalability with modern CPUs and multilevel storage is absolutely insane.
- lexicality 6y ago> frankly I do not see any advantage of hosting on say Azure over hosting myself. In my opinion, cloud hosting only makes sense if your systems are architectured around being hosted on the cloud. If you can build your application in such a way that it runs in the free tier of serverless, then it's going to be considerably more cost effective than renting a dedicated server - but if you can't, then it absolutely won't be.
- wenbin 6y agoTotally agree. Some engineers love to say "I can build this over a weekend". But the main cost is almost always on the maintenance side. I forgot where I saw this number - in the entire lifecycle of a piece of software, (on average) maintenance cost is at least 8x of the initial dev cost. Yes, buying a solution also costs time (and $$$) to integrate and maintain, e.g., using a 3rd party API - time to write code, time to set up monitoring / alerting, error handling... There are always exceptions, edge cases, special situations... But in general there are fewer and fewer things that a web company needs to build from scratch. It takes less engineering time to launch a web product than before, thanks to all those out of box solutions, e.g., SaaS, apis... I like building a software/web business today than 10 years ago :) One-person (or tiny team) web businesses will be very common (I'm running one [1]), and running an API business is not bad because it actually saves customers time & money(vs building one in-house) [2] thus they are willing to pay a small fee (compared with hiring one or more full-time engineers). [1] https://www.listennotes.com/blog/the-boring-technology-behind-a-one-person-23/ https://www.listennotes.com/blog/the-boring-technology-behin... [2] https://www.listennotes.com/blog/how-i-accidentally-built-a-podcast-api-business-46/ https://www.listennotes.com/blog/how-i-accidentally-built-a-...
- satyrnein 6y agoI think "maintenance" somewhat mischaracterizes it. In my experience, it's often that an engineer can build the typical use case over a weekend, but will have to continue pouring in more dev time while it's in production to handle a huge number of edge cases that weren't properly handled originally. That work is only "maintenance" because you already claimed the work was completed, and finance has it marked as having been put into service, so you can't capitalize further costs against it.
- wenbin 6y agoYea, in software development, it’s always work in progress. There’s not a clear finish line. The word “maintenance” can’t perfectly capture the meaning of all the continuous development, small incremental improvements, bug fixes, operational tasks...
- dilyevsky 6y agoBuy if you want to save money, build if you need it to work.
- thinkharderdev 6y agoThese sorts of discussion always seem somewhat confused to me because "buying" vs "building" is not a binary choice. It is more often a choice between: 1. Buy some platform/framework which you will then need to hire a small army of costly consultants to integrate and customize for your particular business need. Or 2. "Build" your own solution by orchestrating a bunch of open source technologies to solve your problem. Moreover, it is not quite clear up front whether 1 or 2 will be costlier in terms of development effort and overhead.
- recursivedoubts 6y agoexcellent point
- barnaclejive 6y agoThis is 100% correct
- dvtrn 6y agoIt would seem to me there are still two choices to make, meaning it’s still a binary choice-merely with more caveats and words involved to describe the options?
- thinkharderdev 6y agoYeah, that was phrased somewhat awkwardly. The binary choice I was rejecting was between 1. "Buying" a solution that solves your problem exactly and doesn't require engineering resources to implement and maintain 2. "Building" a solution where you have to solve every problem from scratch where you don't have any particular expertise.
- localhost8000 6y agoI understand your general point of there being a continuum of paying for building blocks to be integrated. But, > is not a binary choice Then, offering two options. :)
- 6y ago
- api 6y agoThis is usually true except for two cases: 1 There is some other expense you can save significantly on by DIY. An example would be DIY k8s on bare metal via bare metal providers for a massively bandwidth intensive service where paying AWS or GCP outbound data rates would be thousands and thousands a month. 2 You are at sufficient scale that there is an economy of scale that can be accessed. This overlaps a bit with the first point but can also happen if you are just big enough. The first point is more about some special need.
- x87678r 6y agoThis is where free & opensource has really changed the world. Its not buy vs build any more its download vs buy vs build and its best to use a free package that is out there. Its incredible what is available now, in the future I wonder if anyone will need to build anything.
- mirekrusin 6y agoNonsense without a context. Sometimes "building" means writing 6 shell scripts (each <10 LoC) and "buying" means getting CKA/CKAD certificates for all people involved and spending half a year on training.
- fuzzy2 6y agoI'm a little confused. When I have to build software (for a client), it's usually to support some business process or make it automated or whatever. What am I supposed to buy, except some PDF/Excel/whatever library every now and then? But maybe I'm just not the target audience for this article.
- satyrnein 6y agoYeah, doing work to automate some idiosyncratic process, either for clients or internal users, may be different. You do have to be careful you're not slowly implementing a CMS, CRM, email marketing tool, etc, feature by feature, though.
- z3t4 6y agoI think there is a balance in the "Yak Shaving" business... There is a swaying "the best code is the code you did not write".
- leowoo91 6y agoI have to disagree.. Not everyone has enough time to learn "ready" systems either, they can get pretty overwhelming. When I decide to build instead of buy, it's mostly because I don't believe in the design of the product that is favored by thousand people with reason being "it just sounds cool".
- lexicality 6y agoThis works great until you hire a second person who now has to figure out how your system works from the documentation you wrote, rather than the documentation and blogposts on the internet written by the thousands of people who thought the other thing was cool.
- human 6y agoFrom an engineer perspective there is a learning opportunity that can’t be underestimated. If you have free time to build it’s always interesting how much you can grow by doing it yourself. However, if you are working for a startup and time to market is essential, than buy it for sure.
- recursivedoubts 6y agoEvery build v. buy decision should ask: "Is this a place where we want to bet on our ability to derive competitive advantage." If the answer is no, lean towards buy. If the answer is yes, lean towards build. For ops, netflix would say "yes" whereas most companies would say no.
- magicalhippo 6y agoIt's all trade-offs. At work a customer could only send some data via mail, our integration could only accept via SFTP. I had just had a look at Azure and had seen their Logic Apps, and quickly whipped up one which downloaded the mail and uploaded the files to SFTP. Nice. Until a few months ago when it suddenly stopped working because for some reason Azure doesn't like the key exchange algorithms Bitvise SSH server provides. Or at least, that's as far as I've been able to determine. Zero logging on the Azure side so no clue what's wrong. After spending hours trying to figure out what went wrong, I made my own program that does the same. It took a bit longer than the Logic App, but at least I can fix it when it breaks...
- papito 6y agoThe most egregious cases of violating this involve trying to create your own CMSs, search engines, and God forbid - databases. There are of course, exceptions. AirTable wrote their OWN database engine, but they knew what they were getting into, and they did it right.
- hansvm 6y agoI can't speak to CMS, but are search engines and databases really such a big deal? If postgres or influx or something kinda fits your workload then it's probably not worth the time, but on the other hand I've seen both custom databases and the cludges people have written to work around off-the-shelf DBs, and I vastly prefer the former both in terms of performance and maintainability in a problem space (similarly with search engines).
- papito 6y agoThe problem with search engines is that people think it's just tokenizing the search query and then doing an AND query for the words, but then the fun starts. First of all, there is specific search theory that one has to dive into and have the Knuth tomes nearby. Then, different languages, thesaurus, taxonomies, "did you mean". And we are not even at indexing speed, replication, performance, etc. There is absolutely no chance you would be able to create and maintain a reasonably good search engine part time.
- blargmaster42_8 6y agoThis an ad
- rq1 6y agoSorry but no. It’s cost dependent and you should carefully study both options. It’s part of the engineering: study fixed and variable costs. Outsourcing is cost effective when the market is mature enough. You should maybe not rebuild cloudflare’s core services to operate a website. Depending on your scale, you certainly should build your own ML stack for instance.
- mlthoughts2018 6y agoI think all four points in the article are deeply wrong. 1. It is easy to run your own services. You should be investing in internal developer tooling that makes this easy, and in fact that developer tooling should sit in front of any vendor solution you ever buy so that the way it integrates with your alerting, monitoring, data exporting, etc., is completely standardized to be uniform with service delivery in any other system in the org. 2. and 3. You do need complete control over what the application does because it absolutely always is unique and special on a per-use-case basis every time. A good example is search. Anyone who thinks search is a commodity service you can just throw ElasticSearch or Algolia in front of is sorely mistaken and dangerously naive. Every different search use case is going to have different success criteria, different data privacy concerns, different timeliness and freshness concerns, etc. and you need business software to control these elements in ways that fit into standard internal product management and QA procedures. 4. Vendor lock-in is a critical problem. If you choose GCP vs AWS, you are defining culture and you are defining experimentation and exploration that you cannot do. You’re essentially cleaving away many future possibilities from even being testable. It’s much worse than just having a crufty old system to maintain, it’s about brittleness and lack of ability to appropriately empower engineers to consider whatever part of the solution space they decide is needed. Companies that “get it” will prioritize “ease of swapping” so that you can constantly improve and leverage autonomy without needless parochial constraints on what can be considered. Thinking, “yeah but just buying it solved our problem today” is such a death knell of weak leadership who cannot fathom strategy or how to leverage real solution ideation from their staff.
- lta 6y agoThank you for your answer. I was desperately scrolling in search of one like this to support. I'm especially angry at the dismissal of the vendor lock-in problem in the main article. I've seen quite a few start-ups being trapped by a vendors which were very cheap at first (trial period, starter plans, etc.) and ended up eating much of the profits. Not to mention the innovation cost...
- libria 6y ago> The desire to build custom versions of everything seems to come from a few places: 1 [...] 2 [...] 3 [...] 4 There are personal reasons as well: 5. Innate curiosity and excitement about technology 6a. Get experience 6b. Resume talking point Many of us picked a career in IT because just like building things out of Legos is fun, building a streamlined full CI/CD pipeline is fun, building a full stack application is fun, etc. Speaking of career, one needs to acquire experience in the new technologies to pad their resume and it's convenient to do it on company time. I'm not justifying putting your interests ahead of your company's, but understand that's what some of us do. I'd say #5 and #6 are often stronger drivers for decisions than #1-4.
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- mchusma 6y agoThe answer here is not so simple as "always buy". It depends heavily on context. I think there are some things that do make little sense for anyone to build unless it's super core. These tend to be low level, cheap developer tools that do a couple of things well. Trying to "buy and then integrate" larger pieces of software is often more work. Lots of people "buy" because they don't understand the problem. It's a mental opting out. This leads to disjointed experiences internally and externally. But it all depends on your situation.
- monadic3 6y agoThis doesn't account for vendor incompetence at basic customer support, which is far and away the largest reason to avoid renting commodity services. Being able to debug and fix things on demand without waiting on another party is massively valuable.
- coward8675309 6y agoThe way I put this to clients is that you should own the bricks that your product is made of and not worry so much about the mortar. The bricks are the SQL queries, the logic inside the container images you run, the data stored in the tables and storage buckets. Oh yes, and the business relationships and brand equity. As long as you're wiring those things together in ways you can replicate across cloud providers or in your own datacenter or on your own server in the closet, you've achieved the sweet spot in terms of deployment portability. Running your own (Kubernetes cluster|database instance|pubsub system) on a cloud provider is crazy given that you now have at lease $150K+/yr piece of overhead in the form of at least one (dev)ops person. Save that expense for when or if you decide your needs have become predictable enough that you can start building your own software/hardware/network infrastructure. Unless advanced devops fu is an important part of your company's mission or valuation, don't do it unless its saving you a lot of money over the alternatives.
- jtsiskin 6y agoI thought one of the main advantages of “running your own (Kubernetes cluster|database instance|pubsub system)” was for exactly that reason - so that “you can replicate across cloud providers or in your own datacenter or on your own server in the closet”.
- coward8675309 6y agoI don't pack a desk in my messenger bag: a laptop, pencil, and notebook suffice.
- woleium 6y agoA lot of the maintenance issues change with low code solutions, as do a bunch of other assumptions made in the article. We will be seeing a lot more low code developed without rigor by end user teams in coming years.
- drej 6y agoI used to have a boss who took this logic to an extreme and basically preached “always buy, never build” Problems mounted - cost, security, integration issues, performance, ... - we couldn’t do anything about any of that, because we just wrote glue code to combine all the SaaS stuff. I think about that job from time to time and I have two theories as to why this guy was like this - a) lack of trust/knowledge his engineers can build stuff, b) bragging about using a shiny-new-tech is better than talking about writing a Python script that saves data to Postgres. All in all - good points in the article, but there is more to the decision (as hinted in the summary), building stuff is not just about scratching one’s itch.
- deleted 6y ago[deleted]
- veesahni 6y agoFor Enchant [0], we followed the "buy don't build" philosophy a lot in the early years. However, the biggest challenge has been reliability. For example, a service provider will have a 5H outage without any clear indication about what's going on or when it will get resolved. This leaves us in a difficult position to raise an incident with an unknown ETA where we can do nothing but wait. Then when it's all resolved, we either don't get a post-mortem or a hand-wavy one. Or a service provider will lose data, which may have been minor to them but critical to our operations. And all we get from them is an "oopsie!" So over time, we've ended up insourcing mission critical pieces (within reason) since we can engineer solutions with guarantees that are inline with customer expectations and our own targets. Separately, in some cases, buying and integrating turned out to be a ton more work than what building in house would have been. Because when we build in house, it can be built to meet our full requirements in the first place.. whereas when buying, we may find ourselves working around limitations in the APIs provided to us. 0: https://www.enchant.com https://www.enchant.com - shared inboxes, knowledge bases, live chat
- rachelbythebay 6y agoIt’s not buy vs. build. It’s _rent_ vs. build. Unless you are actually purchasing whatever it is you’re using... and I’m guessing they aren’t doing that.
- lambdatronics 6y agoThat's really incisive. Just as with rental housing, the responsibility for the maintenance gets shifted, not just the responsibility for building. It seems to me like software maintenance has gotten more and more important relative to software building over time... but it might just be an optical illusion. (I mean, Windows 3.1 didn't update itself overnight...) I wonder if this is true, and if so, what's driving it?
- paragraft 6y agoYes, and you only organisationally learn this distinction really once you've been say, bitten hard by a contract renewal that completely up-ends your architecture, requiring huge efforts to stay in compliance with the new terms because by this point you're well and truly locked into the product.
- taylodl 6y agoSystems of Differentiation should employ a build-first strategy. These are the systems that differentiate you from your competitors, they are your competitive edge. As such you typically need tight control over these systems and don't want to rely on 3rd parties - either vendors or contractors. Systems of Engagement & Systems of Record should employ a buy-first strategy. These are typically more operations-focused systems and as such you're usually better off buying them, but not always (see below). It's possible that some of these systems may also be Systems of Differentiation. What is a buy-first strategy? It means consider buying first. How much would your operations be impacted by the necessary changes to your workflow? How much customization will be required in order for it to be usable and how are those customizations maintained throughout product upgrades? How much integration is required with existing systems? Buying software is rarely a buy, deploy, and done proposition! Usually the TCO is lower if you buy, but not always - depending on how much customization and integration is required.
- bdcravens 6y agoThe phrase AWS came up with for this, "Undifferentiated Heavy Lifting", is quite apt.
- taylodl 6y agoI should have pointed out this is the strategy that should be employed by an established company. If your company is just starting out then buy everything if possible - your competitive edge is the fact you're small and nimble. You can respond to market changes faster than the established companies can. Over time you'll recognize what are to become your Systems of Differentiation and then and only then should you consider building that yourself. When you're starting out you typically don't have the time or capital to wait and build the software you may need. For such companies doing business and generating revenue now is more important than having a perfect application portfolio. And yes, this is why systems are never as perfect and pristine as developers would hope.
- doonesbury 6y agoLike the post, and use of definitional concepts. Useful. Thank you.
- bdcravens 6y agoThe hardest part of learning to not look through your developer eyes. Often the 80% solution is "good enough" for your customers or your stake holders, and they'd never appreciate the extra 20% a fully custom solution provides.
- CapitalistCartr 6y agoBuilding anything is step 5 of a good process, not step one. Whether you buy or build is far secondary. First, check the motivations. Emotional or logical. If the driving motive(s) is ego, that's already a big, red flag. Consider running like a bull in reverse. Second,intel. Gather information about the situation; educate yourself. Find the apples on the road. Third, formulate a vision, an image of the final result that all the stakeholders (those whom you cannot ignore, no matter how hard you try) agree on. This is where beautiful diagrams are created. Which don't matter. All that matters is each stakeholder signs & dates it. Fourth, plan. Use your knowlege gained in step two to formulate a plan for getting from the current situation to the "vision". Fifth, execute said plan. Find more road apples. Sixth, debrief. Analyze the outcome. Get a drink with the ones who did the work. This can all boil down to just you spending 2-3 days running around doing it yoursels, or multiple teams spending most of a year herding cats. The important part is knowing the steps, and mentally looking for them. And for those who push to skip.
- burnte 6y agoMeh. In 2007 I wrote a webapp for a friend to help him manage his business. It's in PHP and other than a few fixes here and there, it's running just fine in 2020.
- bird_monster 6y agoI work at a small startup. My boss comes from only enterprise level companies. My boss believes literally everything we touch has to be custom, and we have to make it. It's an incredibly exhausting process of getting in front of and/or retroactively nullifying any planning the company does based on his suggestions, because most of the time they're quite harmful to our startup's tech. I've gone to our CEO about it and he agreed that we have to curtail this behavior whenever we see it. It's just an extra layer of "stuff" I have to think about. No, we don't need k8s, Kafka, a home rolled CDN, custom authentication, etc.
- deleted 6y ago[deleted]
- zapataband1 6y agoThere are some good points in there, but it's worth noting build/deploying is becoming easier than ever. For a simple static site you could pay 300$ a website-builder-thing or you can build an app inside a docker container and your host will reboot it for you if it crashes. I use a docker-compose.yml file and I was able to quickly download an open source service that automatically renews my SSL cert. But yeah it depends on the scope, and the project, and what skills you have freely available or wouldn't mind learning.
- zapataband1 6y agogood real-life example i just ran into: i host my own website, and now I want to run a newsletter/email campaign. I don't desire doing this work and it's cheeeaapp/free from certain services that guarantee high delivery.
- temporallobe 6y agoThis is not a universal truth. I’ve worked with COTS products that required a ton of work to customize and get working properly - sometimes costing as much in time and manpower as it would have been to build the same thing from scratch. Often this happens with managed server all-in-one “business solutions” systems that let non-developers build UIs and reports (and charts/visualizations, etc.). Nearly 100% of the time this is inadequate and often completely fails, requiring management to bring in actual software engineers to “fix” things and eventually just rebuild an entire dashboard from scratch using real development tools and techniques. Many of these kinds of shops underestimate the importance of strong software engineering practices and fail to understand the complexity of their own requirements, much less the complexity of their data.
- tus88 6y ago> It’s usually a major mistake, that ends up costing a ton of time and money LOL this is hilarious. Because outsourcing to consulting firms (cough IBM cough) has never resulted in blow-outs and poor delivery. Buying almost always means being trapped in a snake-oil contract that requires ongoing consulting costs that quickly outweight the costs of the actual software. > This can be avoided by making sure that the key differentiators for your business are in-house. Oh....so you are saying we should build now? Blogpsam.
- blindm 6y agoSome people take lifetimes, even multiple reincarnations to come to the conclusion that they are buyers, not sellers. Some people are born marketers and are very good at persuasion, hyperbole, and other tools to sell a product/service. Others are natural consumers: they develop tastes and grow to be rabid shoppers. It's good to sit in the middle of these two opposites, but often better to decide which extreme you want. I know I've made the decision too late in life, when I wasted a vast portion of it trying to sell some product/service/idea when really I just wanted the latest Porsche!
- dasil003 6y agoI'm sure a lot of people would love to go all-in on the buy side. Where do you get the money to buy something if you don't sell anything though?
- blindm 6y ago> Where do you get the money In this case we assume there is a cashflow of some sort. How the cashflow comes into being is a public matter for most, but sometimes a private matter. It is assumed that a certain amount of disposable income can be used to acquire tastes and become a 'shopaholic' of sorts.
- einpoklum 6y agoThis is Capitalist logic, and more specifically - an argument for the concentration of capital in the computing/IT sector. We, software developers and users - people and small organizations - should adopt the opposite slogan: Built, Don't Buy. When we build, we improve our knowledge and understanding of systems. We help affect them, through engagement with their developers or through modification and forking. When the challenges and overhead of building exceed our abilities - this will drive us to: 1. Strive for better-fitting software - easier to use, deploy, build and maintain; and 2. Cooperate and collaborate with related organizations and communities/groups of developers and enthusiasts, either to share knowledge on how to get things done more easily, or to distribute the load of work between many parties, each of which can't do it on their own. Even if a lot of FOSS contribution these days are made at for-profit corporations - the above is key to the future of both software freedom and social freedom.
- doonesbury 6y agoI like this comment a lot. It's new perspective here. Of course capitalism is great. The US is only 25% of China's population and still we produce well. China copied of lot of capitalism. Silicon valley is hard to replicate in other countries. But we Americans (I'm one at an American company) do have a penchant for confusing R&D and longer term investment distorting it through the lens of a short term bean counting dollar mentality. Usually in the medium to longer term is the loss of customers and market credibility by anemic products of crappy quality. And as far as that goes hasn't this been talked into the ground through the 1990s under the broad topics of SPC and TQM? I could mention pithy quotes from Iacocca, Ishikawa, Deming, Drucker and so on but I wont. To compete world wide usually requires deep knowledge in teams, organizations that lead to client centered services that require mature processes and engineering talent to build. You cannot buy that. That deep knowledge helps demarcate build vs buy. Let me mention one story here to make the point in the small. During the 80s/90s HP did selectively less OEM work. They bought Fujitsu wave soldering and pick n place machines. That is something like 1million per line. That kind of equipment comes with a serious stack of user manuals. HP threw those out and provided their own set with training. Why? Any fool can plunk down money. Smart OEMs needed a way to get their line engineers to "know how the line thinks" and to move jobs in and out of the line smoothly in an integrated process with attention to spc, quality and metrics tracking. Yes they bought but not by going ignorant.
- je42 6y agoYou can only buy stuff if you know that stuff is aligned with what you need. Further, you need to consider that the stuff you buy also needs to aligned with your needs over time. However, you also need to consider this when building stuff. The alignment is the tricky thing it is easier to achieve when building in-house as long as you engineering is up to the job. Also, note that building in-house doesn't mean engineering "from scratch". Usually it means use lower level components over higher level components/services.
- Silhouette 6y agoThe longer I run small businesses, the more sceptical I become of this sentiment. I wrote a cathartic rant here at first, about all the different ways external services have let down my businesses over a period of many years and how often our in-house systems have been the only thing that kept us going in those situations. But instead of letting rip and naming names in rather undignified fashion, I have decided to settle for just posting the conclusion. I want to agree with the principle that you build what is essential to your business and you buy in the rest. As both an entrepreneur and a developer, I want to focus on the unique, value-adding parts of whatever my business is doing. I want to let someone else handle the mechanical stuff and the formalities, and I have no problem with paying a fair price for that. But that whole argument is predicated on the idea that outsourcing will get an overall better result than building in-house, so that in some way it saves time and/or money that you can better invest elsewhere. So many services we've used have changed or discontinued functionality and forced us to relearn or reintegrate just to avoid going backwards, so many services we've used have pushed their prices up in real terms while adding little of extra value to us, so many services only ever reach 75% of what we'd like from them and the missing 25% of functionality hurts more and more, so many services we've used just don't work so often that you wonder how they manage to stay in business at all, that I now find it hard to trust or expect good results from any external service at all. Sometimes you have no choice about outsourcing. No small business has the resources to do some things, particularly in heavily regulated areas like payments and tax, or in large-scale infrastructure like operating a data centre, unless that is part of the purpose of the business. So you choose the least of evils and hope for the best. But today, as someone just starting to set up another business, I am looking for services that handle everything in a particular area where for practical reasons we can't do it in-house. I want to specify what we need, and have it done and the results delivered. I don't want to care about anything in between. I don't want to be distracted by any related legal and regulatory compliance matters. I just want the job done, properly and legally, for a reasonable price. I have little interest in working with anyone offering less than that, unless I have absolutely no choice. Maybe this means I actually do agree with the original principle after all, and I've just learned from experience that an extreme interpretation of it is the way to go.
- zmmmmm 6y agoIt's interesting that the whole article is pitched around a context where there is a basic assumption that there is a whole engineering team, devops etc., and the question is about how their time is best utilized. But if you chose to buy instead of build everything most of that would not exist. So there's a paradox I see which is akin to the credit system where only people who don't need credit are offered it: you can only "afford" to buy instead of build when you have the engineering competence to build - that's when you can intelligently choose not to. On the other hand, if you lack the internal competence to build and for that reason choose to buy? That's when all the bad things happen. You're going to get screwed by your vendor - they are going to know you aren't technically capable of supporting yourself, they will give you stupid timelines, blown out costs, unreasonable constraints ("it has to have a 16 core, 256GB RAM server or we won't support it") etc etc. You will end up with the worst case of vendor lockin and a system everybody hates. So one of the pitches I make when people are making this decision is that you should be building at least some things, because the strategic cost of not doing that affects everything else you do.
- bobbydreamer 6y agoThis blog has only one article
- simonw 6y agoOne thing to pay close attention to when making a build v.s. buy decision is the impact that billing models will have on your usage of a tool. Take logging for example. If you buy a log aggregation platform like Splunk Cloud or Loggly the pricing is likely based on the quantity of data you ingest per day. This can set up a weird incentive. If you are already close to the limit of your plan, you'll find that engineers are discouraged from logging new things. This can have a subtle effect on your culture. Engineers who don't want to get into a budgeting conversation will end up avoiding using key tools, and this can cost you a lot of money in terms of invisible lost productivity. Tools that charge per-head have a similar problem: if your analytics tool charges per head, your junior engineers won't have access to this. This means you won't build a culture where engineers use analytics to help make decisions. This is a very tricky dynamic. On the one hand it's clearly completely crazy to invest in building your own logging or analytics solutions - you should be spending engineering effort solving the problems that are unique to your company! But on the other hand, there are significant, hard-to-measure hidden costs of vendors with billing mechanisms that affect your culture in negative ways. I don't have a solution to this. It's just something I've encountered that makes the "build v.s. buy" decision a lot more subtle than it can first appear.
- tixocloud 6y agoVery interesting perspective. Ran into the same issues where Salesforce was charging us by the number of columns we had that ended up forcing the team to overload a column with a JSON object so we could get it to work without overpaying. Integration and maintenance turned out it be a downright nightmare. Would be interested to hear various pricing models to understand the pros/cons for each especially in the logging/analytics space. Does per storage, per event, per user make sense in certain cases?
- nitrogen 6y agoFor things that are directly generating revenue, it makes sense to have per-X pricing because it aligns your vendors' interests with your own. In that case, you want to incentivize the vendor to maximize your usage, because that usage is making you money.
- figers 6y agoA huge customer went with our custom build software because we controlled everything, all built in house, we could integrate with them exactly how they wanted vs a competitor who bought their technology platform and couldn't control the changes asked for...
- doonesbury 6y agoInteresting read. Coincidentally this is a current conversation at our office (10K+ FTEs): use/extend open-source or develop institutional proficiency at aspects of distributed computing? Ours is the usual case: we have homegrown brokered message queues plus Kafka and 21 more. We have homegrown single-master replicated DB, but also use DB2, Oracle, Postgres, MySql. We have private in-core caches with speciality code to knit together data and also use Memcache, Redis. Some of that code is installed from DPKGs; some if it slightly edited code and installed as service so internal users aren't bothered with the icky-details. We have the Jekyll & Hyde behavior mentioned in OP's write-up and from commenters: on stuff considered core to the company and clients for a sufficiently complex system, we're not going to rely on 3rd party trouble ticketing systems and support. It's in-house. But that resolve falls away in stages and never lasts. We also have people who strongly push the business practical side: we're not here to write Azure or Kafka. We're here to make business apps. Get focused, and get client focused. That comes with an interesting blend of boredom and brand awareness gone wrong: I looked around & I saw or heard people in Dept X are working on caching solutions so ... what'd be the point of you doing it? Now, here's the interesting question: suppose we needed a distributed ledger (without blockchain) or we needed 2PC for some homegrown caching. Now what? Do we write that in-house as a reusable components? Dig it out of PG or MySql code if they have it and we know were to look? Suppose we make a business case to upgrade a cluster to kernel bypass NICs. Should we go into Redis and self-modify the code to support that I/O path? No? Frankly, what then is the purpose of a CS guy/gal with a MS in distributed computing otherwise? My own perspective on this: - The first 1/3rd of this back and forth is the proxy conversation that flies above reality which is more about risk management and perception of failure i.e. company reputation & gossip mongering. - The second 1/3rd of this is what people talk about because we can't talk about the underlying CS concepts. Most company employees usually have no ability to formally describe, or otherwise even sketch 2PC giving pseudo-code for it on a whiteboard. Or take this another way: see if you can get your lead on your homegrown replicated DB to explain how it works and why it works like we'd see in a senior level CS class: clear, specific, and yet abstract enough not be talking about superfluous implementation details. Can the audience understand it? - Insufficient grounding in customer needs so one can better determine what clients really need, and what's lacking in the current offerings. Failure here will send you on goose chases to no where. Gedanken experiment: you and I start a new software company broadly in distributed ledgers, caches. We are able to get 25% of MS' distributed computing leads --- not all --- but none of these guys and gals are slouches. What now? Do we write it in house or expand and extend some IBM offering? The the difference is we have talent, and there's some reason to believe they can deal with a distributed system's worth of risk. To end this with humor, here's a nice analogy I once heard: On a Saturday, mom and dad were sick of the noise of their three kids. And the house was a mess. So they send them outdoors to work their energy off. At sunset the house is clean, and they're sitting on the porch watching the sun go down over some wine. They see the kids coming back: they are filthy dirty. They see work: all 3 kids needs baths. The clothes have to be washed. And they'll probably destroy the bathrooms they just cleaned. So the father looks at the mother and says: As I see it, we could clean these kids up or make a new one. Not sure if making a new one is wisdom, but it's a hell of a lot more fun.
- dmantis 6y agoThe very important point of today's world - data protection - is not covered at all. For customers, it's better to give their data consciously to one provider within one chosen jurisdiction than moving personal data, documents, activity logs, etc within multiple 3rd parties, as well as to do due diligence on any web service they provide any data. Please, respect your customers and don't share their activity when it's not necessary and cost just a few hundreds bucks per month. Treating humans as "units" instead as people with rights is a big problem of IT businesses.
- ummonk 6y agoThe problem happens when Ops software tends to be GUI-based and overly complex, and you just want some programmatic / command line tool you can properly integrate into your software / processes. It's often simpler to take open source software and adapt it to your needs than to buy commercial software.
- hootbootscoot 6y agoMmmm probably depends upon what it is. Core competencies of your business? Probably not the best thing to use an external service for. (Assuming your core competency is not cloud computing & storage) Your core will need to constantly be extended, honed, refined, redefined, poked at, prodded etc...
- say_it_as_it_is 6y agoAnd then the day comes when you've been laid off and your lack of product development expertise keeps you from a world of opportunity.
- effnorwood 6y agoBut muh open source. Build don’t buy.
- XCSme 6y agoI think the answer is somewhere in-between. I really like the idea of "paid open-source", where someone creates a software product and you pay a small license fee to self-host it. This way you still have full control over the software, very low costs but the product is still externally maintained so you don't have to worry about it breaking. The only thing you have to maintain are the servers, but with all the VPS and managed cloud platforms out there, running a server, backing it up or monitoring it is done in a few clicks. The biggest selling points of such a distribution system should be: 1) It's REALLY easy to set-up, no technical knowledge required. 2) It doesn't require any type of maintenance (eg. can run perfectly at least one year without changing anything, this means that the software should allow for OTA updates). 3) It is a lot (10x) cheaper than a SaaS alternative.
- jokethrowaway 6y agoThis implies that employees actually have more valuable stuff to work on or that the leadership in the company know what's valuable, which is not always the case. Given the limited number of projects creating values, it's often a choice of working on the next failed project or saving the company 100 grand.
- dave_sid 6y agoNah
- Ataraxy 6y agoI tend to favor abstractions so that services can always remain agnostic. It's one thing to buy, but it's another matter to be locked into something. Especially if it's core to your business.
- marsdepinski 6y agoIf you know what you're doing building and running yourself is usually a good path
- mjgs 6y agoGreat article and some really interesting comments in this thread. I wrote a piece just a few days ago talking about the same topic, although it was aimed at earlier stage developers, so there is less detail, but the overall message is similar: Deciding when to build a custom solution in web development https://blog.markjgsmith.com/2020/12/11/deciding-when-to-build-a-custom-solution-in-web-development.html https://blog.markjgsmith.com/2020/12/11/deciding-when-to-bui...
- inshadows 6y agoThis is very often misinterpreted to mean that you should use exiting stuff even if it doesn't fit your purpose. Obviously there need to be products that fit your purpose BUILT by someone for you to buy one. Several times I've been in a position where people around me pushed me into using something that remotely looked like it solves my problem. Since I was the one doing actual analysis and work, I knew I'd spend a lot of time digging through internals of said product to bend it for the use case. They also claim that they don't believe that there's not some existing software that exactly solves my problem and I'm creating something just for the sake of it. Let's be honest, these people are full of crap thinking about software engineering only superfluously. There's no silver bullet.
- talkingtab 6y agoIf you are talking core technology then I disagree. My example is Oauth. There are lots of libraries that make it easy to use Oauth, but then what you have learned is not Oauth, but some library and how to use it. Libraries can also add a layer of complexity because they are generalized. And finally, what if the service you buy and don't build needs to be modified for some use case that is particular to your application? While I agree that all of these points should be considered in the general case, core technology for your business presents a very different set of considerations.
- ideamotor 6y agoTalk Don’t Do So much effort is spent on hypotheticals. If you think about a problem for too long, it will seem like a worse problem than it is.
- watermelon59 6y agoI hope this doesn't get buried, because there's something that I feel like I'm a bit of a unicorn about. > The problem then with that is everyone who is working on those services is usually trying to get off of them. After all, no one wants to work on something that their boss doesn’t care about. > Yes, your CI/CD system is absolutely critical, but it’s easy for executives to not think about. This leads to a failure mode where you have a lost garden of internal tools. I love maintaining stuff. I don't care that my boss doesn't care about something. I care about... what I care about :) I don't mind the "boring" tasks like keeping library versions up to date, being on the latest runtime version, and using a recent version of the framework we've adopted (we're currently running on an ancient version of it). I like doing that stuff. I like keeping things tidy. If things are up to date, I can move on to making our CI better, or improving our test coverage, or really anything that improves the whole team's productivity. There's always something to update or polish or improve. My dilemma is that as much as I'm willing to do all that stuff, I'm essentially not allowed to. My lead and their boss say my skills are too valuable to be spent on that, so instead I must do things like lead the new team they're forming to build a new service... which I'm not interested in at all. I'm not keen on the whole lead thing. Glad to be a follower. I'm also not thrilled about building new stuff. I like to say I'm more like a car mechanic rather than an engineer: I like tweaking and tuning and fixing existing stuff, not creating new stuff. Does anyone else feel like this? I've tried bringing it up in chats with coworkers and everyone looks at me like I have two heads.
- presentation 6y agoI more or less feel that way - I don’t have a lot of interest in churning out yet another feature in the never ending treadmill of feature development.
- madrox 6y agoConsider that while you may care about maintaining something, whoever comes after you likely will not. Just like writing and documenting good code, solutions should last beyond a single engineer. So yes, your skills are too valuable. All engineering skills are. That's the point of buy vs build. If you truly feel you aren't being allowed to work on the the things you care about, it's possible you're simply in the wrong role or company.
- rmason 6y agoLike all things challenge absolutes. I agree you shouldn't always build, otherwise you'd build your own office suite! But do you know how many times engineers started to build something and made a discovery that resulted in inventing something significant. Or created something that gave their company a significant advantage. A not small number of times.
- nitwit005 6y agoThis misses the rather obvious issue of price. Building your own software is the cheaper option some of the time. This is particularly true when you have to pay to add some missing feature you need. They will, naturally, try to ask for as much as possible to get what you want.
- madrox 6y agoWhat this doesn't mean, though, is Buy Everything. Buying everything is also how you get poor quality and poor process. It's important to recognize what's core to your culture. It's why I prefer discussing this as a what is core vs commodity decision. If there's a general rule for making these decisions, I haven't found one. What's helped me, though, is asking whether or not the problems my team has is any different from most other teams. Sometimes it's true, like when I was at an entertainment company working on asset management. Sometimes it isn't, like when I was at the same company figuring out an AB testing solution.
- bern4444 6y agoDSS?
- trabant00 6y agoThere are several good comments in here explaining cases and reasons to build instead of buy. But they are all techical and miss one critical thing: human motivation. Nobody cares about your bussiness as much as you. The motivation behind your product/service provider is to take your money. Buying will always be a battle with the other party to get value for your money. They are strongly motivated to offer as little work for as much money as they can get away with because there is where profit is made. The provider might even be more competent than you in building. But think about hiring a good programmer that just doesn't want to work vs a mediocre one that is highly motivated. I think everybody has come across these examples and knows the outcomes.
- doonesbury 6y agoAmen to that! Giving a damn is absolutely needed. See longer response elsewhere this comment section.
- nine_k 6y agoWhen you have more money than time, prefer buying. When you have more time than money, prefer rolling your own. It's a well-known principle. For a VC-backed startup with millions galore and a pressure to beat competition, buying is natural. For a bootstrapped company or a side project, building your own may be best, especially when the advanced features that make market-leading solutions more expensive does not add a lot of value at the company's current scale, and the growth is not exponential.
- beshrkayali 6y agoThe article makes some good points, but the title is somewhat clickbaity and reductive. Blanket statements such as "Buy don't build" are unlikely to be a good rule-of-thumb to follow imho. However, at the end, a somewhat more sensible advice is presented: > I suggest that you prioritize buying to building, unless building will provide a real sustainable advantage for the business. I generally only buy things to save time when I really don't have it, and usually as a temporary solution. Buying isn't an all-in or all-out solution. There are many levels at which you can decide to either buy or maintain the thing yourself. In a lot of average/usual situations, reading how to properly setup the service rather than just buying it from x-as-a-service will provide valuable learning not just for the setup itself, but also for tooling used around it, which will make developers even happier and more productive. For example, I wouldn't buy Redis/Elasticsearch as a service if I can spare a couple of days to read how to set it up (and of course there are always exceptions). Care about security first, then efficiency. It will take time, but it's not as difficult as it's made to seem. Quick example, a few years ago, I witnessed a company save more than 80% of their hosting cost by switching from Heroku to a simple EC2/Ansible setup. They're still buying, but there are levels. Also, it helps to know what are the actual requirements when you're buying. Thinking that buying everything as-a-service will save time is a misconception imo, as you'll likely end up having to spend time reading the docs of whatever you're buying. Decision changes will also cost more time because unless that service was using an open standard, you'll likely end up having to change a few things. There's great value in learning open standards and how to automate setups. With that being said, no one can learn everything at once, so if the need is urgent and you're short on time and have properly studied the financial model of the service, yes, buying is a good option. Not related to this, but I've been having a lot of fun messing around with Nomnad. I've also learned some very interesting things about how docker networking work because of it.
- PeterStuer 6y ago'Buy' often is not available at the granular cost:benefit price-point that your specific application needs, especially in the startup and scale-up phase. For the service provider there is a fairly fixed to linear base overhead so it does not make sense to make an special offering for the 1/10th or less of functionality you might require at 1:10th of the price. It doesn't make economic sense. So for the small business it can mean they get away with a homegrown minimal service at a fraction of the cost, even though the 'quality' of the produced for general applications might also just be a fraction of integrating the 'off the shelf' offering. I've seen some startups running in major deficits simply because they went all out on integrating external services the aggregated cost far outstripped the margin their market allowed for.
- lumberjack24 6y agoAnother great read on the topic is from Calm's CTO - https://lethain.com/build-vs-buy https://lethain.com/build-vs-buy Some interesting takes 1) There's more to it than the 'risk' factor, consider value & costs! 2) Ask yourself "What can I actually build right now?" and be brutally honest before attempting any comparison (vs external tools) 3) Solutions that hinder your efforts to move any component outside their ecosystem are best avoided. These prove to be costly in the long run.
- codebolt 6y agoCompletely disagree with this sentiment. I personally built a small program that scans our production environment for logs with errors and sends the team an email if something's up. Spent two days writing the initial version, have only made a handful of small extensions since then. Been using it for years now, with no complaints from anyone. But countless problematic situations that were resolved before they could cause major havoc. Would have probably spent a lot more time wrestling with a more general solution to make it fit our particular environment.