11 ms·
The Fallacy of Move Fast and Break Things
- mtreis86 7y agoIn summary; don't move so fast that you stumble, and be wary of what you break. I think a better one would be "don't run your company based on aphorisms".
- hef19898 7y agoAlso, make sure you are running in the right direction. Ideally before moving fast.
- Traster 7y agoMove fast and break things has always been more marketing than real ethos. It's an obviously obtuse thing to say to signal people you're an asshole savant (whilst writing mediocre php in your bedroom). I don't know why people bother with it so much. People act as if because "Move fast" is a good thing, it has anything to do with "Break things". Does everyone at facebook sprint around the office and break limbs when they bump into each other around a blind corner? No, because they're not morons.
- codingdave 7y agoIt is not a motto, it is an anti-pattern. When you are still prototyping and have not found product-market fit, it might work. (And that quote comes from the early days of Facebook, when it was still morphing into its final form.) Once you have established yourself as a viable product, found your fit, and have paying customers, "break things" becomes a path to poor service.
- Hokusai 7y ago> Once you have established yourself as a viable product, found your fit, and have paying customers, "break things" becomes a path to poor service. Yes. I have seen this too many times. People are creatures of habit. I have seen people from small companies try to run after the growth a company 10x or 50x bigger like if nothing has changed. I have also heard of, not seen by myself, directives from big companies going to smaller ones trying to apply all their processes and checks to just a couple of teams. Each strategy is correct for the company situation. There are no silver bullets.
- dasil003 7y agoI feel this reasoning easily slips into no-true-Scotsman territory. Did FB have product-market fit in 2006? 2008? 2013? They didn’t just move fast during prototyping, this went on for years and years of hypergrowth. There was customer backlash at each major release as they overhauled the product but it turns out they knew better than the customers themselves knew what they wanted. Friendster and MySpace by contrast slowed way down once they hit traction, but it didn’t enable them to solve their scalability issues.
- jcelerier 7y ago> it turns out they knew better than the customers themselves knew what they wanted. do we live in the same universe ? facebook has already one full step in the big storage room with myspace, omegle, etc. most teens I know laugh at you if you say you're using facebook like you're some decrepit old person.
- hinkley 7y agoWell, I mean "It's better to burn out than to fade away" is a motto too, but it's one that people backpedal on when their organs start to suffer the consequences of their hard-living.
- comprev 7y agoOriginal article: https://devops.com/the-fallacy-of-move-fast-and-break-things/ https://devops.com/the-fallacy-of-move-fast-and-break-things...
- gorgoiler 7y agoGreat article. Move Fast and Break Things always meant “and don’t be afraid that you might accidentally break things, because you eventually will and you should just fix them if you do”. The saying isn’t itself a fallacy but doing the moving fast part without adequate resources to do the fixing part is indeed reckless. Don’t push on a Friday night. Don’t push without metrics on error rates. Talk to your team. Append instead of overwrite. ..etc.
- danesparza 7y agoAgreed. I always saw this as an extension of "fail fast" -- or an encouragement to experiment (even when those experiments aren't as successful as you'd like). Sometimes, teams get stuck in 'analysis paralysis' -- and this was meant to be a type of antidote. To the folks that say this is a type of anti-pattern, I say: Either you haven't seen this implemented correctly (it helps if you have kids that you want to encourage to explore) or "There is more than one right way to be creative".
- mtberatwork 7y agoAdding: Hire and retain enough experienced staff. Too often these things boil down to one or two engineers shouldering the entire burden of keeping these systems online.
- lonelappde 7y agoYour last paragraph is good advice in many cases but it's saying to not move fast and break things. Twitter rose to prominence on an infamous fail whale. Reddit also was constantly broken. Facebook broke privacy and democracy ;-)
- brosinante 7y agoThe HN crowd (and related ecosystems) have been using this as a mantra for a decade or more now. Interesting to see it only took worldwide attention and derision of this "model" to make us even consider it as a fallacy. To me it further confirms it - don't trust either the industry, or the people in it.
- vbtemp 7y agoI might not have a multi-billion dollar company, but Think deliberately, Code carefully, Test thoroughly seems to maximize success for minimal stress...
- ragebol 7y agoI've always thought that you can only apply this mantra when you can afford to break things. Which you can only do when the thing you break is not critical to anything.
- tmcb 7y agoWhat are the consequences when you try to move fast and not just break things, but fail? - It takes longer to resolve incidents. - You lose customer confidence and sales. - Employees burn out, and you have a high turnover rate. The main idea behind the approach is making sure that you can move fast and break things whilst minimizing those consequences. It means absolutely nothing if you just let things break; that would be even unethical. - Taking longer to resolve incidents will allow you to anticipate minor issues that could add up during a worst-case scenario, which is a valuable thing. - Impact on customer confidence is less critical in earlier stages, when you are not expected to make a profit. Once again, these minor impacts on it will, ideally, bring attention to things that matter. - Employee burn out is uncorrelated to this approach. It is all about having the right policies in place, replacing blaming by accountability and treating other people with due respect.
- jasode 7y agoThe "move fast and break things" is just a web app version of previous sayings in other domains: - "A ship in harbor is safe but that's not what ships are made for." - "If I've made more shots than you, it's because I've _missed_ more shots than you." -- variations of this from Michael Jordan, Wayne Gretzky, other athletes - "All great writers got better by writing a lot of _bad_ sentences." - "If you want to learn how to ride a bike, roller skate, ski, snowboard, etc you're going to have to _fall down_ a lot." All of them are trying to be provocative in the same way: expect repeated failures when experimenting/exploring what success looks like. So MFABT when compared to variations of previous quotes is pretty standard advice. Nevertheless, everybody likes to poke holes in it because it came from a source a lot of people don't like: Mark Zuckerberg & Facebook. Should programmers[1] at NASA "move fast & break things"?!? Of course not. I know what the original context of MFABT was about so it doesn't apply to NASA. [1] https://www.fastcompany.com/28121/they-write-right-stuff https://www.fastcompany.com/28121/they-write-right-stuff
- sremani 7y agoI think it comes down to volume and rate of corrections. You will inevitably make mistakes in many endeavors and most mistakes are not fatal, but while making mistakes the rate of correction during this time puts you on the right path. Most of the criticism of "move fast and break things" is word thinking. The move fast part includes "rate of correction".
- lnenad 7y agoIt's also the size of the corrections, doing daily builds probably means you've changed a small piece of functionality and even if stuff breaks it's usually a quick fix, whereas doing them monthly/bimonthly carries a much higher risk of completely breaking stuff and is much harder to fix.
- braythwayt 7y agoWith respect... I would say that a great deal of the criticism I’ve seen is of behaviour that can be summarized as, “Move fast and break things for other people. But if we’re achieving our objectives, no need to correct.” It is not fair to characterize all “Move fast and break things” as falling into this egocentric bucket, but My anecdotal experience is that this is what attracts the most criticism.
- vanderZwan 7y agoI think it was someone on HN who said "I wish we focused more on moving slow and fixing things" and I still think of that remark whenever something breaks or isn't quite working as intended on my computer or phone. So quite often, I guess. I do think "move fast and break things" has a place, but that place is mostly limited to so-called transcendental problems: situations where the status quo don't work and we need to try something outside of the established solution space. It's kind of comparable to the over-use of brainstorming; quite often brainstorming is not going to help you find the solution to a problem.
- bigpeet 7y agoMy current company seems to apply the following variation of the motto: Move slow, break things anyway, disable tests to hide it. Yeah, I've quit.
- karatestomp 7y agoHaving the wrong things broken is a great way to move slow. Heavily manual (so, insufficiently documented) deployment processes, difficult, manual correctness testing, slow feedback loops in development, that kind of thing. Poor rollback planning.
- bigpeet 7y agoPretty much spot-on.
- commandlinefan 7y ago> disable tests Ha, amateurs - my last company never wrote any tests in the first place.
- thedance 7y agoHey looks like we both worked there! Break everything and be paralyzed, that was our motto.
- dahart 7y ago> The 2019 report showed elite companies have 46 times more frequent code deployments than low-performing teams and a 2,555 times faster lead time to move from code commit to deploy to support their initiatives to move faster. > Yes, elite teams ship faster. But, their changes are 1/7 as likely to fail, and they recover from incidents 2,604 times faster than low-performing teams. Since the report is behind a form, and the form is having trouble loading for me right now, I’ll just ask if anyone knows the context behind these numbers? These numbers seem very large for any kind of average; are they choosing the most extreme cases? I feel like they also only make sense in units of seconds or minutes, otherwise we’re comparing hours to weeks or days to years.
- fleddr 7y agoFrom the article: "As consumers and end-users of software, our expectations are continually rising." A very agreeable statement, but no they're not. Expectations are that your service works, makes sense, is easy to use. This is not a rise in expectations, it has always been there.
- PaulRobinson 7y agoLot of answers so far seem to have missed the points being made in the book the article references. If you have the ability to ship things fast - and so do regularly - you will find that your releases: - Have a small blast radius if there are bugs, because how much blast can you have in an hour's work versus a quarter's? - The tooling that you build to let you go fast will tell you that a blast is indeed happening right now - Your Mean-Time-To-Resolution (MTTR) is lower because you can just roll it back. It's not days or weeks of work to get a rollback out, it's minutes. - That having a culture of being able to release continuously and quickly allows you to go fast, and the occasional breakage is tolerable, because the cost is low. You can go slow if you want. And if you don't have LEST metrics or have slow release processes you must, but if you work to get to the point where you are able to move fast, you'll find that you break things less and fix them quicker when they do break. The alternative is you still break things, but the problems incurred are much, much greater. Also note: there are some environments where continuous deployment is never going to be an option or desirable. Thankfully these environments tend to be ones where the market will tolerate the higher costs of formal methods and a more deliberative quality control process (medical devices, aerospace, nuclear reactors, etc.).
- altcognito 7y agoI like everything except the small blast radius point here. You can do a lot of damage in a short period of time with computers.
- PaulRobinson 7y ago`sudo rm -rf /` is a very big thing to do in a very small amount of code, sure. I'd adovocate thinking about probabilities then: If you have a shorter period of time to produce a smaller change, are you more or less likely to have a smaller blast radius than if you have a longer period of time to produce a much larger change? There are no absolutes, but we're talking about risk here, and risk is about impact and _likelihood_. Your impacts are going to be smaller and your likelihood of big blast radiuses are _less likely_, but not ruled out.
- 7y ago
- Koshkin 7y agoThere are some things I wouldn't want to break, ever. Think before you move.
- Finnucane 7y ago"Move fast and break things"--said no carpenter, ever. Imagine if doctors, engineers, plumbers, electricians, and all the other real world folks we depend on decided this was a good way to operate. Would you drive across a bridge knowing that was the motto of the people who designed and built it?
- tmcb 7y agoThing is, they do, but not in the ways you would expect. Cadavers on medical schools. Wind tunnel modeling. Computer-aided designs. The main difference to those is that the software industry can indulge on testing its own assets with little to no cost, and especially if that does not impact your ability to generate revenue.
- mdorazio 7y agoThere's a rather big difference between breaking things in a test & learning environment (all of the examples you provided) and breaking things in a production release.
- tmcb 7y agoThere is also a big difference between a profession that respects the intelligence of their peers and one where people assume that their peers are ignorant and lack common sense. Surgeons must act fast and be good at multitasking. I bet they are not in a forum discussing if those traits make them vulnerable to hasty decisions and to loss of concentration on the task at hand.
- mdorazio 7y agoI'm genuinely not sure what your point is. Can you explain it? The two surgeons I know generally have extreme trust that their tools, monitors, etc. will "just work" when and where they are needed, every time, and that those things have been properly vetted by the medical industry and community prior to going into a live hospital environment unless it's specifically known that they are testing something new.
- downerending 7y agoReminds me of a quote I cannot locate, but will paraphrase: Any career, no matter how successful, will ultimately feel like a series of failures. That's not meant as a downer, but rather just that it's often hard to tell. As to MFABT, the proper adjustment of that knob in a particular situation is difficult and is what expertise is all about.
- zackmorris 7y agoThe basic assumptions behind move fast and break things: * We don’t know what the final solution will look like, because of things like scope creep and client demands. * The more agile the team is, the higher the velocity and ability to iterate. * The more specialized roles we have (like project manager, team lead and junior developer) the more we can focus on completing tickets to increase velocity. Which I counter with: * We actually do know what the final solution will look like. I normally have an idea of what clients are asking for before they finish their sentences. Then I extrapolate to the fundamental problems and meta concepts around their specific problem. I’ve already formulated solutions and am solving them in my mind, until I isolate the unknown steps (the hard steps), whose odds of me being able to solve them are the basis of my estimate. Most of my work today goes into research, setup, glue code, infrastructure and maintenance. Solving the client’s actual problem constitutes perhaps 5% of the work I must do. * We can always iterate faster in our minds than in the real world. With a proper architecture budget, we can take time in the beginning to brainstorm the complete solution before we ever start writing code. Then if we follow proper quality assurance practices like test-driven development, we can write code that is free of technical debt and bugs. All the while exerting substantially less effort than writing the same thing several times. * Agile programming wasn’t really a thing until the rise of large internet companies and the marginalization of individual software developers. It’s curious to me that the people with the most say in writing solutions have the least clout in organizations. I feel that this is due to supply-side economics and wealth inequality. So today we cater to client first, then organization, then software developer. That’s what the specialization in agile is really all about. But we should be doing something more like movie production, so the client is the producer, the organization is the director (or movie studio) and developers are the actors. The industry should have more confidence in the process and have conventions which mitigate the churn caused by clients’ fickleness and low understanding of the problem domain. But right now it feels like the emphasis is on getting every scene filmed as fast as possible. We pretend that developers have no gravitas and are just interchangeable cogs in the machine. It’s no surprise to me that most productions bomb in the box office. To use construction as an analogy: programmers used to be akin to architects, spending time in the beginning to draft a plan that includes all contingencies so that the building doesn’t fall down (a right-brained or politically-blue approach based on creativity, insight, seeing outside the box, etc). Today we’re being told to be framers, hammering out solutions as fast as we can over and over again in order to get things done until the building is finished (a left-brained or politically-red approach based on a strong work ethic, discipline, loyalty, not getting distracted, etc). I think these are both important, and over the last year I have been working on my “getting things done” side to improve my execution ability. But I will always be a dreamer and value the intuitive side of software development. The idea that we can manifest solutions in the real world with the power of our minds is what got me into programming in the first place. The trend towards application and turning programming into just another job has soured me on the field in general. I’m actually not entirely certain that I can do it anymore. So to me, the true cost of move fast and break things is the loss of people like me who thought that this was all going to go a different direction.
- saurik 7y agoAt an F8 keynote, many years ago, Zuck admitted on stage that when you "move fast and break things" all the time, well, everything is constantly broken (duh) and it made it so hard for even their own engineers to maintain their own applications (which exist on a very large number of platforms) that he appreciated it must have been impossible for third parties to use their developer platform, and so he formally changed the motto to "move fast with stable infra".
- gentleman11 7y agoDo you have a link to that by chance?
- dang 7y agohttps://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=%22move%20slowly%22%20%22preserve%20things%22&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
- tabtab 7y agoIt's a matter of knowing your audience and knowing your funder's/owner's goals and expected risk-reward trade-off profile. A "cowboy" mentality may indeed help a start-up edge out competitors. But applying that to an established company can ruin your reputation: you have more to lose by playing fast and loose. Startups by nature are a gamble, but if an established corporation gambles too often they may poke a hole in their dam that they can't patch fast enough. Venture capitalists usually factor in a high failure rate to startups they invest in. Their plan is that enough will succeed in a big way to counter the high failure rate. A lot of "fads gone wrong" happen when a technique or technology that works for one situation or size is misapplied to another. Your office printer inventory tracker doesn't need microservices and web-scale databases. Pilot projects with new technology is often a good thing, but don't bet key organizational functions on such a project (sometimes called line-of-business systems).
- gentleman11 7y agoI am reading Bob Martins Clean Architecture right now. The introduction takes the opposite stance: that there is no such thing as moving fast in a sloppy way, because even in the short term quality code yields faster development times. This is meant to include changeability: quality code should theoretically yield faster iterations too. This “break things” aspect is something I believe he opposes. Is he right? Wrong?