4 ms·
It's amazing to me that something I threw out in a meeting six years ago became an entire engineering culture. It all started when we were talking about what t
by jedberg 6y ago
It's amazing to me that something I threw out in a meeting six years ago became an entire engineering culture.
It all started when we were talking about what to call the folks who were building things like Chaos Monkey/Gorilla/Kong and I said, "Let's call them Chaos Engineers, since they are engineering chaos". And so we adopted that title at Netflix, and now here we are.
I should also point out here how valueless what I did was -- I literally just came up with the name for what we were already doing inside Netflix. Everyone else actually wrote it down and spread it outside of Netflix. Their contributions to spreading the word are far more important than the name I came up with.
- jackblemming 6y agoWhat's the difference between "Chaos Engineering" and fault injection, which has been around since 1970~?
- nyokodo 6y agoChaos Engineering is fault injection being incorporated as a core aspect of a production system. So whereas in the 1970s fault injection might have been more used in the development and test phases, Chaos Engineering also includes a bunch of persistent services knocking out key aspects of your production system.
- hammock 6y agoSo basically you are writing tests that break things, and then keeping that code in the final compile?
- ulam2 6y agoYou nailed it
- hammock 6y agoReminds me of these kinds of tables: https://www.google.com/search?q=broken+glass+table&rlz=1C5GCEA_en&sxsrf=ALeKk0306UvLBo4WafGo4fY2pnkcGP0F5g:1603801893558&tbm=isch&source=iu&ictx=1&fir=-p8n4PjYmb69GM%252CZB7Q3vDYL49qlM%252C_&vet=1&usg=AI4_-kTBwXyqfi_Tm_8AOpiXYDLYAb5QGg&sa=X&ved=2ahUKEwi47p7R49TsAhWahXIEHeNODMoQ9QF6BAgJEE0&biw=1227&bih=648#imgrc=-p8n4PjYmb69GM https://www.google.com/search?q=broken+glass+table&rlz=1C5GC... Wherein the table is made, then shattered, but keeps its functionality :)
- spooneybarger 6y agoFault injection is a technique. Chaos engineering is a discipline.
- clr42 6y agoYes; to expand on that a bit -- you can do fault injection in a chaos experiment. You can also do many other things within an experiment that would be difficult to classify as fault injection; for example, a sharp increase in the number of customers making requests to your service isn't really a fault. Chaos Engineering then wraps all of those techniques in a framework of experimentation and analysis. That's the discipline, which encourages you to proactively make improvements to Availability, and now Security as well.
- bigiain 6y agoMostly better marketing... I'll bet "Chaos Engineers" get paid two or three times whatever the role that did "fault injection" got.
- mxyzplk 6y agoWhile I agree with the other replies, I think there is some interesting alignment with fault injection - while it's "been around," if you consider the big three concerns of availability, performance, and security, there is a rich set of tooling and frameworks and discipline and best practices around the latter two, performance testing and management and security testing and management, but for availability not so much. There's "uptime monitoring" and there's "write your own ad hoc failure injection" and not a lot of agreed techniques around it. A number of the open source "chaos engineering" tools are really "just" failure injection tools but that's still pushing the state of the art forward IMO.
- deleted 6y ago[deleted]
- jeromenerf 6y agoNaming things is notoriously difficult and you nailed it. Bravo. I remember clearly the chaos monkey posts and how it resonated for our team, as we were building an ambitious real time project at the time. Thanks for the inspiration booster o/
- toomanybeersies 6y agoAt my old job, we were creating a new mobile app, and as a joke I suggested a name pretty close to grindr (it was a completely unrelated app). The CEO was oblivious to my sarcasm and decided it was a great name for the app, so that's what we ran with.
- clr42 6y agoThanks, Jeremy! Having a name for it made it a lot easier to write the principles: https://principlesofchaos.org/ https://principlesofchaos.org/ And now we have a canonical book. ;-)
- 1t1m3thr0w4w4y 6y agoLink only allows for strict one-time GET. Potential readers with security components in their infrastructure (such as URL Defense) or previewing features in their mail client will never get to the content.
- simonebrunozzi 6y agoThanks for the details, and the much appreciated humility.