3 ms·

I largely agree... but to offer more nuance you need to know what the right amount of automation versus toil is right for your project. It's worth reading the G

I largely agree... but to offer more nuance you need to know what the right amount of automation versus toil is right for your project. It's worth reading the Google SRE [0] books, especially the workbook, if you're thinking about running and operating a service. Complexity is the enemy in software engineering as much as it is in operations engineering. Sustainable reliability is about doing as little as you can get away with to achieve the reliability results that make your customers happy. That does mean following the advice of TFA and avoiding going straight to k8s for a small indie site that could be deployed with a simple Ansible/Puppet script. However it also means defining what your targets are and having indicators so that you're not relying on your gut instincts either: aim for 60% toil, 40% automation at first. Whatever the right balance is review it regularly as you scale your project up. You can do a lot with a couple of beefy VPSs and a solid database these days. [0] https://landing.google.com/sre/books/ https://landing.google.com/sre/books/

lostphilosopher 8y ago
"Simplicity is a necessary precondition for reliability." - Edsger W. Dijkstra (handwritten annotation to EWD498)
- ken 8y ago
  That's in EWD1175: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD11xx/EWD1175.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD11xx/E...
  - lostphilosopher 8y ago
    Ok, maybe you can help settle this for me. I know that phrasing appears in EWD1175, but years ago I came across this[1] which claims it was originally a handwritten annotation to EWD498. So that's how I've always cited it (I prefer Dikstra's phrasing to Hoare's.) Some searching today found a wikiquote page[2] which led me here[3] also claiming an EWD498 handwritten annotation as the source. Do you know definitively whether the annotation appears in EWD498? (I thought at one time I actually found an image with the handwritten annotation, but I can't find it at the moment... Maybe I dreamed that...) 1. http://www.cs.virginia.edu/~evans/cs655/readings/ewd498.html http://www.cs.virginia.edu/~evans/cs655/readings/ewd498.html 2. https://en.wikiquote.org/wiki/Talk:Edsger_W._Dijkstra https://en.wikiquote.org/wiki/Talk:Edsger_W._Dijkstra 3. http://web.archive.org/web/200011201643/http://www.cbi.umn.edu/inv/burros/ewd498.htm http://web.archive.org/web/200011201643/http://www.cbi.umn.e...
- heymijo 8y ago
  If one is looking to triangulate here's that same tenet in systems theory known as Gall's Law [0]. "A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system." [0] Systemantics by John Gall https://books.google.com/books/about/Systemantics.html?id=ql1bAAAAMAAJ https://books.google.com/books/about/Systemantics.html?id=ql...
JohnFen 8y ago
> Complexity is the enemy in software engineering For decades, I've been saying this a bit differently: "Software engineering is primarily an exercise in complexity management".
- combatentropy 8y ago
  "Controlling complexity is the essence of computer programming." --- Brian Kernighan, 1976
kokokokoko 8y ago
My rule has always been: "write down the entire manual process". Follow the steps and do a deployment and time it. Do you have that much time? Are you likely to make an error? If not, just do it manually. If so, break out the easiest parts and automate that. Then reassess. Automation should follow the same rule as other things in a business. Only automate when it becomes painful not to. Or if it is trivially simple to.
- rakoo 8y ago
  Reminds me of an article I saw sometime back on HN: "Manual work is a bug" (https://queue.acm.org/detail.cfm?id=3197520 https://queue.acm.org/detail.cfm?id=3197520). Just like you, the author's advice is to go iteratively, with the first step being documentation. As documentation improves you act more as a cpu running instructions, at which points automation becomes clearer, and so on. Not everything can or should be automated, but we definitely could do more.

lostphilosopher 8y ago

"Simplicity is a necessary precondition for reliability." - Edsger W. Dijkstra (handwritten annotation to EWD498)

ken 8y ago
That's in EWD1175: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD11xx/EWD1175.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD11xx/E...
- lostphilosopher 8y ago
  Ok, maybe you can help settle this for me. I know that phrasing appears in EWD1175, but years ago I came across this[1] which claims it was originally a handwritten annotation to EWD498. So that's how I've always cited it (I prefer Dikstra's phrasing to Hoare's.) Some searching today found a wikiquote page[2] which led me here[3] also claiming an EWD498 handwritten annotation as the source. Do you know definitively whether the annotation appears in EWD498? (I thought at one time I actually found an image with the handwritten annotation, but I can't find it at the moment... Maybe I dreamed that...) 1. http://www.cs.virginia.edu/~evans/cs655/readings/ewd498.html http://www.cs.virginia.edu/~evans/cs655/readings/ewd498.html 2. https://en.wikiquote.org/wiki/Talk:Edsger_W._Dijkstra https://en.wikiquote.org/wiki/Talk:Edsger_W._Dijkstra 3. http://web.archive.org/web/200011201643/http://www.cbi.umn.edu/inv/burros/ewd498.htm http://web.archive.org/web/200011201643/http://www.cbi.umn.e...
heymijo 8y ago
If one is looking to triangulate here's that same tenet in systems theory known as Gall's Law [0]. "A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system." [0] Systemantics by John Gall https://books.google.com/books/about/Systemantics.html?id=ql1bAAAAMAAJ https://books.google.com/books/about/Systemantics.html?id=ql...

JohnFen 8y ago

> Complexity is the enemy in software engineering For decades, I've been saying this a bit differently: "Software engineering is primarily an exercise in complexity management".

combatentropy 8y ago
"Controlling complexity is the essence of computer programming." --- Brian Kernighan, 1976

kokokokoko 8y ago

My rule has always been: "write down the entire manual process". Follow the steps and do a deployment and time it. Do you have that much time? Are you likely to make an error? If not, just do it manually. If so, break out the easiest parts and automate that. Then reassess. Automation should follow the same rule as other things in a business. Only automate when it becomes painful not to. Or if it is trivially simple to.

rakoo 8y ago
Reminds me of an article I saw sometime back on HN: "Manual work is a bug" (https://queue.acm.org/detail.cfm?id=3197520 https://queue.acm.org/detail.cfm?id=3197520). Just like you, the author's advice is to go iteratively, with the first step being documentation. As documentation improves you act more as a cpu running instructions, at which points automation becomes clearer, and so on. Not everything can or should be automated, but we definitely could do more.