8 ms·
Hidden costs of constantly shipping new things
- 29athrowaway 6y agohttps://en.wikipedia.org/wiki/Feature_creep https://en.wikipedia.org/wiki/Feature_creep https://en.wikipedia.org/wiki/Scope_creep https://en.wikipedia.org/wiki/Scope_creep
- arbitrage 6y agoThat's not a particularly good way to encourage discussion. What is it specifically about Feature & Scope creep that you would like to say?
- 29athrowaway 6y agoI think this article is great and it does a good job characterizing what happens to products that fall victim to feature creep.
- lbriner 6y agoThis is kind of interesting but it sounds like the perpetual worry of the perfectionist developer. Of course you end up with tech tech, you develop new things when you could improve existing things, you deploy something that could be iterated but you don't. The truth is that the world is a complex place and you don't always know whether you can keep your existing customers by improving what you already have or get new customers to decrease the risk of declining income. It is hard to measure what is acceptable tech debt and what is worth addressing. In one sense, the proof is in the income. If you can help customers with an 80% OK product then you just need to live with the dodgy 20% bit. I agree with the danger of big customers though! If you ever think something is a way to avoid hard decisions, pivots and annoying some of your customers, then it is too good to be true ;-)
- at-fates-hands 6y agoI worked at a large web development company. The one thing which continually hamstrung the company was we had several large customers who were paying us a good chunk of money to maintain their web properties. They sucked up so much of our resources to keep them happy that other smaller customers suffered as a result. The amount of churn we had was always the smaller clients who were still paying good money but felt neglected and would be open to changing vendors. We went out of our way to mind the big clients and thought if we lost these few big clients it would so dramatically adverse the companies bottom line, we couldn't afford to lose them. This resulted in a situation where they were able to hold our company hostage, make unreasonable demands on time and energy and hamstrung our ability to keep our more loyal clients (who only bothered us on occasion) happy. It was a good lesson there is a danger when you have a small company servicing a much larger company that you get into a position where your fear of losing them is a detriment to your other, more faithful clients.
- michalbugno 6y agoI'm far from being a developer perfectionist, I wouldn't spend 10 years in this company if I was one, trust me (btw we fired a lot of them along the way). The point of the article is not to convince anyone to slow down as much as possible and work on bugs/etc. It's just that at some point (3, 4 years in?) there comes a time that you just have to put more effort into the things I described, otherwise it gets complicated fast. Obviously there is a tradeoff, my opinion is that this tradeoff wasn't correctly balanced (especially further down).
- sreekotay 6y agoI think the article starts out sounding that way --- but reading all the way through, it's about how to DO BETTER if you want to ship more often (the key take aways from the article being: document, iterate/revisit, and be intentional)
- cnfsd22 6y agoFiring perfectionists is a strange signal. If the customers are sufficiently large, you should do everything possible to please them; beyond revenue, they are themselves an asset (sometimes a company is acquired for this only). In many industries it can take years to establish yourself as an approved vendor for a large org, and especially to a large enterprise company, this can be worth more than the underlying business. Not suggesting that’s the case for Base, but if you’re not gunning for an IPO, your exit strategy isn’t going to be one that necessarily resonates with a product or eng team.
- montagg 6y agoI work at a company that grew from a few hundred to a few thousand people in a few years' time, and there was an identical mindset and identical problems over that growth period. I started during that growth period, and 10 years later, we're still digging ourselves out of those mistakes. One reason for this was how heavily we prioritized first mover advantage. We pushed extremely hard to get into new markets, and while strategically this worked out extremely well for us, it meant that later development was hamstrung by earlier decisions. Any iteration on old features almost always meant rebuilding it entirely, and we've only just started to get to a place where previous efforts to do this are now paying off in increased velocity. I think there's an argument that this could be good, since you're only improving features when there's some other value you can ship. However, I think it's short-sighted, because it means prioritizing small iterations has an unnecessarily large cost, simply because that cost wasn't paid down slowly over time. While I do think prioritizing new markets early on was a good call, that mindset bit us in the ass in a big way when we tried to ship a very flashy new feature later in our growth cycle that crashed and burned because of tech debt, and it took us over a year to fix everything to release it. If we had taken a mindset of building for the long term five years ago, which was already years after we'd gotten to a comfortable place in revenue and marketshare, we would probably have paid off most of the major tech debt by now. I honestly don't know how a company can get past the addiction to shipping, at least not during its growth period. It can be strategically necessary, and I imagine in only a two-year period at the early stages of a company, like what's in this article, it's impossible to avoid. But leaders must have in the back of their heads the idea that they will need to start tipping the scales toward standardization, building to last, and building infrastructure as well as features sooner than they'd probably like to avoid larger problems in the future. And I suspect the only way leaders will really take that seriously is if they've gone through it before. EDIT: Clarified a few points.
- jamil7 6y agoApart from the one catastrophic failure you mention it sounds like, from a business perspective this worked? You were able to extend your tech debt long enough to start generating money and are now in a position to pay it back. From an engineering perspective I agree it's a grind to pay all this back and lose so much velocity. Are their business strategies that favour slower more careful engineering practises? mission-critical systems come to mind but maybe also nth-movers could be put in there? Where you know what you're building and competing with but want to make the best version of X.
- pkteison 6y agoWhat an awful article. Expensive conclusions are drawn with no apparent consideration for their cost. The experience of a company which appears to have successfully balanced complicated tradeoffs in choosing features is discounted because there were bugs. And almost no examples or useful stories about any detail, or suggestions on how to do it better other than to go (much) slower and to spend (much) more or to not take rich customers money. I suspect good arguments could be made for these things, but they need to acknowledge and justify their cost.
- michalbugno 6y agoAuthor here. Sorry you didn't like it :) It's tough to suggest detailed solutions to a problem which is in general very vague. Btw I hope the article shows my appreciation for what the company achieved. One small insight from Uzi (CEO + founder) when I discussed this with him (after publishing article): Base should've focused on much less features but with greater detail. It somehow confirms my guess that we didn't work on existing features as much as we should, but we "spread too thin".
- deleted 6y ago[deleted]
- Covzire 6y agoOh good, I was afraid this was going to be an intervention for me regarding my steady stream of almost daily deliveries from Amazon/HD/Walmart/Ebay etc.
- renewiltord 6y agoEvery company suffers under the weight of the debt of being feature focused. That's because everyone who didn't do that isn't alive to tell the tale.
- jmull 6y agoHm... this article actually seems like a better argument to obsessively focus on constantly shipping new things than the opposite.
- coldcode 6y agoBeing bought out for large money often covers up the nightmare that would have been, and shifts it to someone else.
- hn_throwaway_99 6y agoAlso could be titled "Hidden costs of moving to microservices because the cool kids are." Very glad (most of) the cool kids have re-evaluated their stance on microservices.
- michalbugno 6y agoI have a plan to write a short article about specifically this topic at Base. I personally fucked up a ton of things with microservices there :)
- troelsSteegin 6y agoI think this is a solid critical reflection that at least in enterprise relates well. When it comes to your customers, I believe that you want to understand as much as you can, as early as you can, about what they are actually trying to do and about how they are actually faring with what you have sold them. I'd argue that the success of your initial customers is more important than the features for your next deal, and that feature investments should reflect that. The problem, as the OP said, is in estimating the difference between the incremental feature requirements of your initial customers and what your strategic feature set will be. There may be no difference, and your strategy is to ship features in order satisfy each new customer as you get it. Perhaps you can design for that. Chances are though that your first customers are not wholly repsentative of your strategic, target market. So, in the early stage, you have to make choices about allocating dev for tactical "now" or for strategic "later". In the early stage, if there is not enough "now", there is no "later". The question is when to begin budget for later. I'll argue that you are attempting to build a business, not a suspense movie, and that you budget a fixed amount for "later" from day one. Plan to succeed. I agree with the recommendation of a strong Professional Services function as an buffer for dev, but will note that culturally you have to work hard to not silo. Your Support and ProServe teams become a primary channel of learning for dev. As a business, though, you may have to give away ProServ hours. All in all my biggest takeaway here is that you attempt to "hire" customers strategically. As you journey together, will this customer want to go where you will want to go? What time is it when your "Big Customer" sits on your roadmap? ... "Hire" customers strategically is easier said than done when there's salaries on the line -- accordingly look at how you compensate Sales and budget away from early stage big deals.
- deleted 6y ago[deleted]
- digitallogic 6y agoInteresting! I especially enjoyed the insight into the evolution of the company. Though with this insight I have slightly different conclusions than the author: 1. A bias to ship and a bias to ship new things are not one and the same. A lot of the problems, such as a failure to iterate on existing products/feature, sound very much a product of the later, not the former. If anything, the issue "Insufficient Iteration" is probably not correctable without a bias to shipping. 2. A bias to ship and a bias to ship things that impact your customers are not one and the same. There is a note about an early shift to micro-services. I can't speak to this organization, but, generally speaking, spending time on internal engineering work to the detriment obvious missing features is a common issue with early stage companies. 3. A bias to ship and a bias to ship to the right customers. Specifically in regard to the high value, high demand customers. 4. A bias to ship and... idk what to call this? "The product specs were well thought through, sometimes crafted for months." TBH, the problem with this one feels like a lack of a bias to ship. As described, I think the real culprit was a lack of or poor prioritization. FWIW, I suspect the author and I may actually be in violent agreement as I did find my self nodding with most of his lessons learned. Though I'd be careful about letting too much hindsight bleed in (eg - do situations that'd be improved by more decision documentation justify the effort of documenting all product decisions, especially in the early phase when the product is rapidly evolving)?
- RandoHolmes 6y agoI think the problem is that the leaders in the company (technical or otherwise) didn't recognize that software systems are like children: They have phases. The ship it constantly is probably ok when you're small, but at some point you need to become more mature than that, both as an organization, and as an approach to the software itself.
- bleonard 6y agoWhen I was running a product team, I often talked about "nuance" being the enemy. Every difference (or arrow in a flowchart) was a new piece of product debt to have to think about later. This article is in that direction. > … the difficulties started piling up fast, and before we knew it we had to deal with a lot.
- Slartie 6y agoI always wonder why nobody appears to try a hybrid model between product and project development when it comes to satisfying large customers which need very specific customizations that often don't fit well into a product development roadmap aiming at satisfying a more generic audience. Like: don't just add weird features that make sense for just one customer, but are eventually delivered to all customers, but instead maintain a "core product" which provides a lot of generic, core functionality, but which by itself isn't actually usable and just serves as basic building blocks to construct the actual end products which are then built and packaged individually for each (big) customer. Each of these "end products" is a development project in itself, has its own team, its own codebase, but they all draw from a core set of base functionality developed and maintained by another dedicated team that is not directly in contact with any of the customers (or at least not permanently). This - at least in theory - prevents custom functionality for specific customers from polluting the common core product that aims to please a general audience, while still allowing for the necessary freedom to satisfy big customers demanding custom functionality, but also offering big checks in return.
- pyromine 6y agoI mean you're describing what most Enterprise Resource Planning deployments look like. There's a lot of enterprise software that follows this model
- Slartie 6y agoYou are right; probably I have to state the question in a more precise way, which would be: why is it such that this model seems only to be used in huge "enterprisey" contexts? The model of course depends on a certain modularity of the software in question, but hasn't that been a core concept not just for the biggest players, but also the smaller shops for a while now?
- rahimnathwani 6y agoImplementation costs a lot, not just in coding, but in all the extra work that goes into sales, gathering requirements, training etc. Whoever nominally pays for all that work, it translates into a high price tag for the customer. Only large enterprises will get enough value from the customisations for that high price tag to be worthwhile. All else being equal, a smaller company will spend a lower absolute amount to work around imperfections, and get lower value from any product overall. So of course they will not be willing to pay as high a price. Hence, for commodity use cases, use commodity software rather than fully/semi-customised.
- trentnix 6y agoThere's a lot of insight packed into this post, and really it transcends way beyond the mindset of constantly shipping new things. This particular line resonated with me: Big customers certainly sounded good on paper, but they came with a cost (complexity and a very hands-on relationship) that we weren’t really prepared for, and that we didn’t manage to put boundaries on fast enough. This is an issue I've dealt with more than once in my career. It seems to occur especially when an organization values the sale more than it values the business. It's virtually impossible to successfully apply boundaries after the customer has been trained that there are none. Irrespective of how carefully you do it or how much you try to reassure your customer, it always feels like you are taking value away. I've encountered this recently in an organization that values customer happiness, which it calculates as an absence of customer complaints, more than customer success. The squeaky wheels get all the grease at the expense of just about everything else. And the software inevitably has become a maze of settings and IF-THEN-ELSE blocks and SWITCH statements, increasing the cognitive load required to make what should be simple changes. When new features are added, the impulse is to code defensively and build inevitable customization into everything up front. And further down the spiral of technical debt and increasing complexity we go...