21 ms·
Do-nothing scripting: the key to gradual automation (2019)
- stanislavb 5y agoBraintree's Runbook seems to regard exactly this case of "Do-nothing scripting" https://github.com/braintree/runbook https://github.com/braintree/runbook
- silvester23 5y ago(2019) Discussed at the time: https://news.ycombinator.com/item?id=20495739 https://news.ycombinator.com/item?id=20495739
- SeriousM 5y agoUsually I write instructions in markdown files and test them by redo all the steps. This saved me already many times because I will forget how to renew this damn certificate two years later, or setup the logging engine after 18 months ago. No need for a script.
- abledon 5y agoi wish you could extend markdown syntax so certain things colored the editor differently. # colors it color A ## colors it color B
- qwertox 5y agoI'm so scared of my certificate renewal script. It's really cool and sends me a summary via email of the executed tasks and runs automatically every month. But when I look at the code I don't know what the hell I was thinking when I wrote it. I mean, it even fetches the certificates prior to renewing them and then re-fetches them afterwards in order to make sure that the servers are actually using the new ones.
- dnautics 5y agoI think this is eventually going to be how people use elixir's livebook library.
- JeremyNT 5y agoLengthy previous discussion (2019): https://news.ycombinator.com/item?id=20495739 https://news.ycombinator.com/item?id=20495739
- patleeman 5y agoOne could pretty easily create a tool that takes Github Flavored Markdown and interactively displays the text and make checkbox items appear one at a time in a manner similar to this. Would be much more useful and easier to configure.
- Jtsummers 5y agoHow would it be easier than the proposed Python example? Which only relies on bog standard Python (unless your steps are more complicated) and can (by virtue of encapsulating steps into a class) be wrapped up in nearly any user interface.
- deleted 5y ago[deleted]
- gorgoiler 5y agoNice. A class with a single method can be nicely modelled as a closure. Returning higher order functions and using format’s kwargs is an even more succinct version of this. def explode(**k) def run(): return TheBomb( ’{user} is!’.format(**k)) return run run(user=‘gorgoiler’)
- nerdponx 5y agoI'd almost always rather have a class than a function returning a function, in a language where classes are typical and natural. It imposes more structure on the code, provides some degree of namespacing, and makes it easier to extend.
- gorgoiler 5y agoPersonally, classes feel particularly unnatural in Python. Closures are also weird though, I get your point. I wouldn’t use them in code that’s widely shared. The upside is truly private data.
- deleted 5y ago[deleted]
- iFreilicht 5y agoA class with a single function can also be nicely modelled as a function.
- gorgoiler 5y agoAhem, encapsulation of both data and code :)
- sovietmudkipz 5y agoI did this after seeing this before on HN. There were a few processes that were manual that benefitted from the technique in the article. Learn from my folly: I even called them “do nothing scripts,” referencing this article. However; I was judged by peers for not writing the full automation versions as they didn’t appreciate the idea of gradual automation (programmer hubris?). Saying “do nothing scripts” in meetings did catch the awkward attention of leadership. As a description, “do nothing” communicates a lot. As a brand, “do nothing” can use some improvements. My short prescription of turning “do nothings” into “do some things” into “do all the things” didn’t help. We had some new people join the team and they had fun turning the do nothing scripts into a document. * Sigh * I still build these type of process description scripts still. I usually don’t advertise them to peers until they do some of the things nowadays.
- scrooched_moose 5y agoSomething like "Checklist Script", as that's what they really provide?
- andrewflnr 5y ago"Runnable checklist"
- randomswede 5y agoI ended up landing on "runnable plan".
- BariumBlue 5y ago"Process Automation" would be a good, encompassing term I think. Even if there's no code being run, having all the steps down and easy to follow is a way to make the process "automatic". The term is extendible to completely automated computer code, while including non-computer-code (human) automation.
- simonw 5y agoHow about "scripted playbook"?
- throwaway20371 5y agoThis is the way. I wish this were taught in computer science class, development bootcamps, operations team onboarding, anywhere there is a procedure that is even slightly complicated to automate. It is the absolute best solution there is. * Documentation of the entire procedure is contained in one place. No need to go sifting through 20 different sources of documentation. This lowers the human emotional barrier to "just get it done", as people will always avoid things they aren't comfortable/familiar with, or don't have all the steps to. This central point of documentation also enables rapidly improving the process by letting people see all the steps in one place, which makes it easier to fix/collapse/remove steps. * Automation in small pieces over time avoids the trap of "a project" where one or more engineers have to be dedicated to this one task for a long period of time. Most things shouldn't be automated unless there is demonstrably greater value in the cost of automating them than the cost of not doing so. Automating only the most valuable/costly pieces first gives immediate gains without sinking too much into the entire thing. * One unified "method" to encapsulate any kind of process means your organization can ramp up on processes easier, reducing overall organizational cost. * In the absence of any other similar process, you are guaranteed to save time and money. I would say that the only potential downside is if someone decides to "engineer" this method, making it more and more and more complicated, until it loses its value. KISS is a requirement for it to be sustainable.
- csdvrx 5y ago> only potential downside is if someone decides to "engineer" this method It can be engineered, if you follow a gradual process. On servers, I keep a log of what was deployed in a root directory following the sequential number _ goal format (ex: 00_partitions ... 90_web_server) It is not fancy, most of the logs are not even scripts: many are just ASCII text files, that will only be used as a checklist if the same "goal" has to be achieved again. For example, 00_partition may be "gdisk /dev/nvme0n1" followed by a copy-pasted list of the partitions and some quick description about why it was done that way. But that's on the first iteration only: the next iteration turns that into "do-nothing" script, the next iteration into a better script with basic checks (supporting both /dev/nvme0n1 and /dev/sda), then exception handling (if partitions already exist, etc), and so on: this gradual complexification process avoids the "premature optimization" of creating infrastructure-as-code for what you rarely need, while optimizing and fine tuning the parts you most often need. Someone will certainly mention Terraform, or Ansible, or something else - yes, they exist and they are nice, but if you are doing everything there, you are over-engineering and wasting time: not everything needs your equal attention! If you only install a webserver once in a blue moon, make a .txt checklist of the steps you followed. But if you leave and breathe nginx options and certificate deployment, fully automatize all that, including the obscure details of what may fail if you use let's encrypt with some specific DNS configuration! And if you don't know yet which is which, start small (a .txt checklist will cost you a few minutes) and the next time you find yourself doing the same thing, do it better using the previous artifact (the .txt file) to create a better one (a script, then a better script etc)
- simonw 5y agoI like Observable https://observablehq.com https://observablehq.com notebooks for this kind of thing - quickly building automated scripts that simply accept user input and use it to dynamically construct a copy-and-paste output. You can run and edit them entirely in the browser, they can do anything you can do with JavaScript and they're easy to share with other people and create forks.
- orf 5y agoTerraform. My god, just use terraform. Stop scripting random stuff to create multiple untracked resources in different places by hand. Slap this[1] on top of this[2] and add whatever other services you want, and boom. Done. 1. https://stackoverflow.com/questions/49743220/how-do-i-create-an-ssh-key-in-terraform https://stackoverflow.com/questions/49743220/how-do-i-create... 2. https://registry.terraform.io/providers/1Password/onepassword/latest https://registry.terraform.io/providers/1Password/onepasswor...
- avisser 5y agoI think you're focused on the domain chosen by the author and not the more general technique. Replace the steps with "Releasing a build to my app store" or something else.
- scubbo 5y agoThis type of "helpfully-correcting disagreement" (rather than pointless-but-satisfying snark) is one reason why I greatly prefer Hacker News to almost-every other social network. Thank you!
- c7DJTLrn 5y agoYes, but I think you've been spoilt if every shop you've worked at already knows and uses Terraform. It's a real uphill battle to introduce it to a place where nobody has the mindset of treating computers as cattle, has never played around with declarative tools or config management before.
- iso1631 5y agoCongratulations, you've just made the situation worse by rushing to what you think a solution is, rather than seeing that the actual problem is.
- orf 5y agoThe problem is you’ve got a bunch of things that need to be in a particular state (for every employee, for every office, for every foo). There are a few solutions to this. Documentation with manual actions, a bunch of imperative code to create but not manage these things, or something that describes what you want without the “how I get it”. If you’re at the stage where you think writing a bunch of random Python code to prompt a user with steps is an acceptable solution then I can’t see how the situation can get much worse.
- mr-karan 5y agoThe author has written a Golang version of the same as well [1]. 1: https://github.com/danslimmon/donothing https://github.com/danslimmon/donothing
- kaycebasques 5y agoI also independently stumbled on this idea. My rational was that it turns the process into more of a checklist, without all of the hassle of handling edge cases in my code. Because usually whenever I try to fully automate I re-learn the wisdom of that xkcd cartoon about "automation expectations versus reality". Another wise thing about it is that it's a relatively low-friction way to start documenting institutional knowledge. Your engineers might hate writing docs, but maybe they won't mind writing a bash script like this. Which is essentially documentation in disguise. And since it's (presumably) living with the rest of your version control system you increase the chances that your engineers will explain changes to the process over time (via commit logs).
- ImportOllie 5y agoI really like this, but in the example is there much value defaulting to wrapping each function in a class?
- Jtsummers 5y agoIt gives a uniform interface (the run method) for each step. You can also, then, encapsulate more logic around that particular step if it needs more than just a single run function or some other stateful component.
- _dain_ 5y agoA plain function has a uniform interface too: just calling it.
- Jtsummers 5y agoTrue, but that doesn't help you with encapsulation or the other (potential) benefits of using classes like inheritance.
- Too 5y agoLocal variables inside a function are very much encapsulated. Unless you need to keep state between two distinct actions on the same object, but in that case a single run() entry point will not be sufficient anyway. Inheritance can be replaced by just calling a common function. There are many “what if” here, which will likely never happen and until they do it’s premature abstraction. Most likely some other abstraction is needed by the time script grows.
- _dain_ 5y ago"Lots of times people think they might need something later. You don't. Or you can just do it later if it comes up." https://youtu.be/o9pEzgHorH0?t=339 https://youtu.be/o9pEzgHorH0?t=339 For glue code scripts like this you usually don't end up using encapsulation and inheritance or any other fancy OOP design patterns. It's usually grungy IO-bound procedural code, the sort of thing you'd write in bash if bash weren't terrible for long scripts. Don't burden it with premature abstractions. If you end up needing a class, just write the class later.
- phaedrus 5y agoMy experience in corporate IT convinced me that even if an entire release could be automated down to a single press of a giant red button (and even if undoing the release could be another, single button), then culturally our organization would still figure out a way to turn the pressing of that button into a six hour ordeal requiring the participation of twelve people.
- wing-_-nuts 5y agoThis is so true it hurts.
- oscribinn 5y agoMy empirical evidence backs this up, I had deploying new releases to all servers completely automated down to just creating a tag in GitLab and every time the decision of actually running it turned into a series of emails and meetings throughout the week.
- tossaway9000 5y ago> Create an SSH key pair for the user. > Send the user their private key via 1Password. Why are you generating *private keys* for users, then sharing them? Not that this impacts the automation bits but IMHO users should known how to generate and maintain a key pair, and send you the public key.
- DantesKite 5y agoWhy?
- tossaway9000 5y agoIt's not a "private" key if its shared. Only the end user needs to know the private key details. Consider the usecase of generating a user their initial password for a service, this almost always results in the user needing to immediately reset their password after initial login, this doesn't happen if someone is generating both parts of the key pair for you. My sysadmin doesn't need to know my password, and doesn't need to know my private key or passphrase, that would allow them to impersonate me.
- ghostly_s 5y agowell, your sysadmin created your AD account, and probably has some means of root access to your box too, so they could already impersonate you. * *this may or may not be true depending on how Enterprise-y your workplace is.
- nerdponx 5y agoIf nothing else, they need to be able to access your files in case you leave.
- lucb1e 5y agoUnless it's all encrypted with the user's own password: sure, you can always invoke the admin powers and override the password field in the database or whatever. But the default mode should be that an account is only accessed by the person it belongs to. Even as a security firm we have some exceptions in our company (disk encryption password for a physical device in the office is in a shared vault, for example), or support might do a remote control session with the user present, but it's definitely not the default setup procedure for a new employee to have shared credentials for example.
- qwertox 5y agoThis was very enlightening for me. But I wonder if it wouldn't be better if it were something like a ncurses list with checkboxable list items which one would progress trough. Take a checklist for a pilot for example. Wouldn't it be not as good if the next item on the list only appeared after an item has been checked? It could be beneficial if you could peek a couple of items down the line in order to group your actions or get a better overview of the overall task. "Do I put the water for the coffee in the microwave now or do I first finish this quick item and let the next long-running item start before I go and make my coffee?"
- slightwinder 5y agoPresenting one task after another has the benefit to enforce a focus on it. Planing ahead is fine, but can be also distract or even mislead.
- glacials 5y agoYou lost me at ncurses. IMO this idea works because the task is so simple no one would bat an eye at the thought of doing it. All it takes is one person to question why we're not using <technology> instead, or why we're not spending this ncurses amount of effort towards something else instead, to jeopardize the effort.
- hollasch 5y agoI think this would be a great idea if the intent was a checklist that would never be automated. If the idea is to create a scaffold for future incremental automation where tasks start falling off the checklist (until ideally none are left), then I'd rather start with that framework instead.
- crispyambulance 5y agoCool. A nice medium on the way between "seat-of-your-pants" manual and completely inscrutable automation! People like to brag that they never do something manually more than once, but in reality it takes time and practice to be able to fully automate any non-trivial process. You usually can't do it in one go. I really like that this article embraces that. I especially like how it's broken down such that any particular function can be swapped out with full automation once the author gets enough confidence to be able to do that.
- nicbou 5y agoI really like this idea. It goes hand in hand with an older post of mine: "no script is too simple" [1]. A script is a recipe that your whole team can follow. It's easier to update a recipe than to propagate changes across the whole team. It's also similar to a flight check, which is a fancy name for a reusable to-do list. [1] https://nicolasbouliane.com/blog/no-script-is-too-simple https://nicolasbouliane.com/blog/no-script-is-too-simple
- bazhova 5y agoThis is a really nice solution. For the 99% who don't work for a place with a massive DevOps team, this approach is a nice middle ground that actually works. If management doesn't like it they don't have to know. These scripts are for getting through the day with 10% more energy to spare at the end.
- tmerr 5y agoI do something similar but take it a step further and statefully track the progress of the workflow. The code generates a file that is like a TODO list, and a separate command runs a single step before marking it complete. Great for longer running workflows (days).
- dctoedt 5y agoCould crude shell scripts be used for this purpose? For example, in zsh [0]: $ echo "Do the first step, then press any key."; read -s -k EDIT: Apparently so: [1] [0] Modeled on https://stackoverflow.com/questions/5215343/how-can-i-pause-in-zsh/14867987 https://stackoverflow.com/questions/5215343/how-can-i-pause-... [1] https://news.ycombinator.com/item?id=20495739 https://news.ycombinator.com/item?id=20495739
- dradtke 5y agoBraintree created a tool for doing something similar: https://medium.com/braintree-product-technology/https-medium-com-braintree-product-technology-runbook-be6f072cfc0d https://medium.com/braintree-product-technology/https-medium...
- oehpr 5y agoHey I independently came to this as well! We were performing some migrations to our servers and we weren't sure we had nailed the process down. Automating the procedure has a problem in that I can't just "reset" the server and start again. I could try to make all my operations idempotent or using a babushka style "Check - Set - Check" methodology, but that would greatly expand the work for a one off procedure that once it's done it's done. I wasn't even sure of the generality of the steps, and they were complicated enough that copy and pasting them with all their parameters was tedious. So I created a structure just like this! I called it a living procedure. An excerpt: #!/bin/bash source ./procedure-lib.sh question "Which client is this?" old_client question "What worker is $old_client on?" old_worker question "What is the targets name going to be? (it's fine if it's still $old_client)" new_client question "What worker is $new_client going to be on?" new_worker question "What branch is $new_client going to be on?" new_branch question "What should we name the migration? (typically something like migraiton-$(date --iso))" migration_name #================ Initialize the client on the new worker ================ step "Login to $new_worker and Initialize $new_client in legacy mode using run.sh init_legacy, set their branch to $new_branch"\ 'eg. ssh '"$new_worker"'.lightship.works "sudo -i bash -c \" These question functions also remembered the answer you put in last time (it was based off what the text of the question was, so questions that depend on the answer of previous questions did not presume anything) so that if you just hit "enter" then it would load the last one. This was vital because when a procedure ended up incorrect or needed to be amended, you could ctrl+c the script, fix it, then spam enter until you reached the point you were last time. I've been thinking about creating a better structure to these scripts that allow you to gradually transform these living procedures into babushka style "Check Set Check" automations.
- Flenser 5y agoThank you for introducing me to babushka style "Check - Set - Check" methodology. I assume you meant this: https://github.com/benhoskings/babushka https://github.com/benhoskings/babushka
- gregwebs 5y agoI like to do this with a checklist. In Confluence you can clone the checklist page for the onboarding/release, etc and check off the cloned page. Confluence is wiki-style (edit first, ask questions later), but I could definitely see git-style (pull request first) as an improvement for some situations.
- catern 5y agoThe idea here can be generalized to a primitive programming construct: The magic function "wish", which can do anything you want, just give it a string description. For example: wish('copy these files to this host', files, host) You can augment this further by allowing the program to specify a return type from the wish, and magically the wish will produce the type you want. So for example: data = wish(Path, 'the path of the data files I need') Of course, the wish is actually implemented by sending a request to some human user. In my "wish" library for Python http://rsyscall.org/wish/ http://rsyscall.org/wish/ you can have a stack of handlers for wishes, just like exception handlers. If later you want to automate some wish, you can specify a handler which intercepts certain wishes and forwards the other ones on (by wishing again, just like an exception handler can re-raise).
- ilovetux 5y agoI like your idea. Im gonna try to use your project soon, but due to my companies policies ill probably have to slowly implement your ideas. If it comes to that I might rename your "wish"es to "requirement"s as that would fit with my company's culture a bit more.
- bowmessage 5y agoNo offense, but this seems like unnecessary indirection harmful to script legibility, e.g. it would help me as a reader to see, instead, `scp.copy(files, host)`, instead of having to browse through a list of wish handlers to see which one implements file copy.
- catern 5y agoYeah I agree, you'd only want to automate it that way if you couldn't modify the original script, which is kind of a niche use case... anyway I just think it's neat
- Too 5y agoSimilar to behavior driven testing. For example behave framework. It works exactly like wish does above but creates a whole language out of it. For a human it reads very natural but understanding which combination of human words that map to underlying handlers is a hopeless and unnecessary level of indirection, and is why anyone shouldn’t touch these tools with a ten foot pole.
- mig39 5y agoThis is a lot like the checklists a lot of industries use, just automated on your shell instead of being on paper. I'm thinking of a pilot's checklist, or an operating theatre checklist.
- mypalmike 5y agoThere's a difference though. I've worked in ops teams where there were many such checklists in Word documents, wikis, or whatever. Standard procedures involved reading from these runbooks and following the steps. But here, you have code that plays the same role, but which can eventually be changed out with code that actually does the work for you. It's a seemingly small difference but it's actually huge if you have some decent developers who can spend a bit of time every once in a while pushing the needle forward in automating these manual steps. Another note: I've found that it's hard to even get teams to consider the minor change of using the code version instead of the wiki. Inertia is a real problem.
- didip 5y agoSo... I am confused about something, isn't it better to start writing some automation scripts and document the missing piece for later?
- throwaway20371 5y agoThat's basically what the do-nothing script is. The difference is that before you write any automation, you document all the steps in the script. Right there - when you've got it all written down, and no automation work has been done yet - that in itself is a very valuable piece of work. Now you can point anyone in the company to that script, and they can all accomplish the task without having to figure it out for themselves. You can now scale that process N times (N = the number of people in your company). Just writing down the steps has become a force-multiplier of repeatable work. Then as you begin automating each step, people automatically receive the benefit of that automation. Because the documented steps and the automation are in the exact same place, both will always be up-to-date.
- slim 5y agoThat's implied. Optimisation is not premature when obvious.
- bionhoward 5y agoMakefiles can be great for this sort of thing. Over time you build up a really nice library of shell commands and it's trivial to organize them into a DAG of dependent steps. Lack of makefiles is a strong reason to avoid windows dev machines like the plague...(IMHO)
- guhidalg 5y agoWindows has MSBuild that is just as expressive as Makefiles, don’t use the lack of make as an excuse.
- shadowfox 5y agoOr use a `make` from any of Msys, Msys2, Cygwin, WSL, Gow or any of the assorted GNU util ports.
- bionhoward 5y agoOk, I'll think of a different excuse next time
- lenkite 5y agoThis is nothing but a checklist right ?
- andrewflnr 5y agoIt's an interactive checklist with a gradual upgrade path to full automation.
- kristianpaul 5y agoThe lesson for me here is to actually have scripts written where the business logic is clear to see and change. This is a workflow in a python script but could also be a tool just by adding argument parsing. But also tools can be called from software pipelines and have tools to execute this for us (automation). Look at https://aosabook.org/en/500L/a-continuous-integration-system.html https://aosabook.org/en/500L/a-continuous-integration-system...
- dragontamer 5y agoHow about you just... cut out the middleman and write an expect script (or pexpect script, for you Python programmers) ???
- Arch-TK 5y agoI like the article. This is a great idea. I'm going to implement it over the next week. I have high hopes. That being said, I have absolutely no clue what the advantage is to the python programming style exemplified in the article. Can someone explain why on earth I would ever want to convert: def do_something(context): pass ... do_something(c) into class DoSomething(object): def run(self, context): pass ... DoSomething().run(c) What is the point here?
- ttymck 5y agoIn that contrived example, there is no point. In the real world, there are plenty of reasons to use a class instead of a function. Extensibility is the first that comes to mind. What answers are you expecting (i.e. performance)?
- Arch-TK 5y agoI want to know about this hypothetical extensibility. What extensibility does this approach achieve over just a bunch of functions?
- Jtsummers 5y agoPotential reasons to use classes over just a bunch of functions (and the syntax is rough and probably wrong, I don't use Python often): 1. You have a complex step that you want to refactor. Without classes you end up with something like: def Actual_Step(context): part1(context) part2(context) part3(context) def part1... def part2... def part3... Great, you've now polluted the larger namespace and further exacerbated the need to thread data through a bunch of function calls. With a class, you could do something like: class Actual_Step(object): var1, var2, var3 # things needed for this def run(self, context): # set the vars somehow self.part1() self.part2() self.part3() def part1(self): use var1 def part2(self): use var2 def part3(self): use var1, var2, var3 It's not stateless, but it works just fine. No polluting the larger namespace, and less threading a bunch of data through a bunch of calls. (Do you need the self.part1() bits in Python?) 2. You want to inherit. Maybe you have a collection of steps that all have the same setup/teardown bits, you can do this: class Common_Base(object): def setup(self, context): ... def teardown(self, context): ... def custom(self, context): # should be implemented by each derived class ... def run(self, context): self.setup(context) self.custom(context) self.teardown(context) class Specific_Instance(Common_Base): def custom(self, context): ... And reuse that in multiple places. 3. Customizing at construction time. Maybe you have a step that goes out to some database, but what database may vary depending on what you're doing (for instance, at work we have test and ops modes): class Get_Data(object): def __init__(self, db_name): self.db_name = db_name def run(self, context): ... And of course, since this is Python, you can change all of this to use a decorator or decorators and remove most of the boilerplate, but still retain the potential benefits. In theory you could grow this (though at that point you may be better off with another system) so that different reports can be constructed from the steps themselves, without the need to execute them. Add elements like documentation strings that can be returned and prettified for different user interfaces or reporting purposes. All without ever having to touch the actual methods doing the work, just the way you interact with the classes holding them.
- lifeisstillgood 5y agoWeirdly I have something like this called SOPys (usual python joke). It is pretty much that for pretty much the same reason only this is waaay better explained. So - got my external validation . Will expand that repo when i get my own time back
- paulmooreparks 5y agoSo, this is a wizard, basically. https://microsoft.fandom.com/wiki/Wizard https://microsoft.fandom.com/wiki/Wizard
- tomrod 5y agoAs others have stated, "do-nothing" may have some baggage. Consider something like "incrementally inclusive" or "eventually complete" or similar.
- hyperpallium2 5y agoAnother benefit of this interactive to-do list: you'll find out pretty quickly if the steps really are that simple. The test is you. Many "slogs" require too much human decision to be automatable, but too little to be interesting. tedious. tiresome. irksome. In a way, also trite, bromidic/
- sakesun 5y agoSoon AI will generate do-nothing script for us.
- andai 5y agoI was just thinking, I should make a tool to make these on the fly, I have ADHD and constantly forget where I was in a process. I also have many projects running at any one time, it would be great to be able to leave them open and pick up where I left off.
- nathanathan 5y agoCombining this technique with an automation website like Jenkins and a "no code" programing language could be an interesting product. I imagine having a UI where prompts can be specified and predefined components like send X an email can be dropped into the workflow. Acknowledgments that steps and complete can be tracked, and long running workflows could be shared between users.
- cuddlecake 5y agoThe underlying concept would be akin to Business Process Modelling. Using something like https://camunda.com https://camunda.com [0], you can execute a business process model. Such a model can include sending emails, waiting for responses, human action, custom API requests, etc. Camunda is not no-code, but it comes close to what you imagine. I suspect that when the existing API landscape of a company needs to be used for workflows, coding/integration is inevitable. There's also (truly) no-code platforms like https://bryter.com https://bryter.com [0]: I only know of this because there's a department at my current job that specializes in implementing workflows using camunda. I got explanations for it during moderate drinking, so my recounting of it may be incorrect in parts.
- sullyj3 5y agoSide note: - I don't understand why classes are needed here. instead of procedure = [ CreateSSHKeypairStep(), GitCommitStep(), WaitForBuildStep(), RetrieveUserEmailStep(), SendPrivateKeyStep(), ] for step in procedure: step.run(context) Why not make them functions instead of classes. The URLs can be locals instead of class variables and the context argument becomes an argument to the function rather than a run method. procedure = [ createSSHKeypairStep, gitCommitStep, waitForBuildStep, retrieveUserEmailStep, sendPrivateKeyStep, ] for step in procedure: step(context) The OO is just gratuitous. Personally I probably wouldn't even bother with the for loop/context and would just pass the data that's actually needed by each function into it manually. createSSHKeypairStep(username) gitCommitStep(username) waitForBuildStep() email = retrieveUserEmailStep(username) sendPrivateKeyStep(email) I get that it feels more modular - "oh all I need to do to add a step is add it to the list!" But like, all you need to do to add a step is add a line of code. There's no difference. Much easier to see how the data is actually flowing that way. I found the fact that the context was mutated in a function it was passed into kind of gross. It's just globals with extra steps. Besides, this is less LOC anyway.
- dullcrisp 5y agoI assume the point is to let any one of the steps turn into something more complex and automated, as they say, without affecting the rest of the script. If each step is its own class, that gives you a place to go in and expand any one of them without worrying about conflicting with unrelated changes. If the idea were to leave the script as it is, then of course your way is simpler.
- geysersam 5y agoI see what you mean. But in Python a function can be turned into a full fledged callable class at a later point without any change to the rest of the script. Not that it matters much either way.
- sullyj3 5y agoSeems like premature abstraction to me. It's not difficult to make that change if you need it. In any case, I'm really struggling to envisage a scenario where you would need them to be classes - why couldn't a more automatic version still be a function? In my opinion, in order to justify instantiating an object, it should actually contain data, and that data should actually be accessed by code other than methods internal to that class. This rules out using an object to represent a procedure - regular functions subsume that functionality. Before first class functions became commonplace, it did make sense to use objects as procedures in some OO languages, to allow you to pass them around as values. Since most languages these days do have first class functions, this has become less necessary. If you want a class in order to to split the procedure into multiple methods with access to shared mutable state, that's totally fine - you can just use nested functions to do the same thing.
- rgrs 5y agoI use Bash aliases to do the same
- NationOfJoe 5y agoAmazingly obvious concept, love it. I feel i could use this in my personal life. Just wondering, i know it isn't the point, but does this exist as a service? For example is there a web service where my sales dept could add all the steps and it just goes through them one by one, but can include a start/completed webhooks.
- thr0waway37373 5y agoThis approach doesn't work well in the long term. I've worked as a successor in one of the author's previous company. All the "do-nothing scripts" diverges from the reality as time goes by. The scripts became a false prophet, not a single source of truth. When I came in, the fully automated parts are doing fairly well. On the other hands, most the one-off scripts doesn't work. Falsy docs are worse than no documentation. We've tried looking at the scripts, but small divergents in the scripts destroyed our confidence in the scripts. Instead we figure how to do task by looking at the source code and actual deployments. The lesson we learned from this is we need to treat scripts as a real software instead. We make scripts idempotent and run them periodically to see if anything breaks. For infrastructures we make them immutable and embraced the cattle mindset(vs pet mindset).
- NationOfJoe 5y agoWe make scripts idempotent does this mean change the commands that run to be idempotent? so the script/echo concept stays the same but you make sure the commands it echo's wont break anything if run more then once?
- dbmikus 5y agoNot the parent, but I'm guessing it's the idea that the script always moves the system to the same steady state. Similar to how Ansible tries to be and how Kubernetes controller loops work.
- hollasch 5y ago> All the "do-nothing scripts" diverges from the reality as time goes by. Not sure I follow. In my experience, we have _tons_ of process that's written up in random pages in our company wiki. I assure you, nothing diverges as quickly as a set (regrettably) of duplicated wiki pages outlining a process. If you're making the argument that a fully automated script is superior to a checklist, then ... well, sure. But while you're spending the resources to fully automate one process, duplicated wiki pages are being written for the 100 other checklists (at least in my experience). The key here is that I fully agree with you on the divergence problem. But if a scripted checklist diverges, people complain and it's trivial to update the script. If you don't have these, then people either add a comment to one of the wiki pages, or a side note, or worse, write up a "better" wiki page with the real actual this-time-for-sure steps. I believe the key here is to implement a consistent tactical approach to incremental development, capturing divergence along the way. If you just let them lie fallow, then yes, that's bad. But surely a fully-automated script allowed to lie fallow and diverge from reality is no better off.
- randomswede 5y agoThis is basically why I ended up writing Runnable Plans, over at https://github.com/vatine/runnable-plans/ https://github.com/vatine/runnable-plans/, the main differences are "you define the plan in YAML" (yeah, horrible, but better than hand-chasing a parser, that MAY come at a later date) instead of "in the script", "the plan has no inherent order" (to allow for future parallel execution), "saves success/failure to allow for later restart", and "can generate a GraphViz graph of the plan dependency ordering". But, whatever works, works. Start somewhere, get it into a script, plan, whatever. Then, it is easier to identify steps that can be turned to entirely machine-operated.
- deleted 5y ago[deleted]
- rthomas6 5y agoWhy a class for each step instead of a function? They don't have individual contexts and don't even have any members. They don't even need to persist, it's literally a command line script. Object oriented style for no reason is my pet peeve.
- PaulHoule 5y agoI have a hobby of printing "three-sided cards" that have a front side (maybe a photograph, art reproduction, or anime character) and a back side that describes the image and has a QR code (and maybe an NFC tag) that points to a "web side". This process has a number of steps, some of which are physical (handling paper), some of which are digital and scriptable, and some of which are digital but not scriptable (serious image modifications with photoshop.) I use a process very similar to what's described in the article to manage the whole thing.
- toby_tw 5y agoWhy don’t you just write this down into a checklist? Why codify it? You need the interpreter to run it. It’s less accessible to non technical people. Actually running the script is an extra step that’s not needed. This feels like an engineer overcomplicating something.
- bones6 5y agoI immediately recognized this as a nice scaffolding to start the automation process of a slog task. There's lots of things that are difficult to fully automate. But breaking it down into step by step functions, you can identify and automate the low hanging fruit. Then maybe you could request some help with the ones that are a stretch for you to automate yourself.
- hollasch 5y agoIt addresses the false dichotomy between checklist and full automation. Documentation loves to spawn future copies and variants, while scripts tend to remain single-source.
- hollasch 5y agoIn my work, we have develop on Linux, MacOS and Windows. Another advantage to this approach is that it allows you not just to incrementally automate steps in the process, but _steps on platform X_. It eliminates yet another barrier to starting the process of automation.
- hollasch 5y agoSo naturally, my knee-jerk reaction is to create some kind of checklist language and interpreter. Which is almost wrong, of course. I say "almost", because it occurs to me that many steps are shared individually across many different workflows. If you see a single task as an automation target, then you end up writing simple scripts that perform very focused operations -- a strong advantage over a large complicated script that does many different things, some that are repeated for other complicated scripts. In this setup, you end up with checklists that remain forever, with automation replacing some or all of the print commands. Of course, my spidey sense tells me that I've just pushed the problem down a level, so ¯\_(ツ)_/¯.