25 ms·

In praise of --dry-run

• Smaug123 5y ago
  (Author here.) A whistlestop tour of the technique "insert a lightweight API boundary down the middle of a tool, for fun and profit". It leads you towards a structure that has lots of benefits, such as meaningful `--dry-run` output and more ready librarification.
  • tasogare 5y ago
    Nice to see F# in posts from time to time. I might try the technic in the future.
  • b3morales 5y ago
    This was a great read! It's not material to the point, but I think there's a small typo on line 12 of the second "Finishing the example" snippet: it looks like let instructions = gather args should be let instructions = gather inputGlobs
    • Smaug123 5y ago
      Thanks! You're quite right; thanks for letting me know. I'll get that fixed.
• nicoburns 5y ago
  For particularly dangerous operations, even better is to make dry run the default, and force users to explicitly opt-in to actually running it.
  • kirkules 5y ago
    Definitely true. In this case, it's important to vet guides/examples to make sure you minimize copypastable examples that have the --no_dry_run flag
  • llimllib 5y ago
    I've written a couple tools that default to dry run and require `--yes-actually-delete` and similar, but I can't think of any publicly available tools that follow this pattern - anybody have suggestions?
    • jerf 5y ago
      rm has "--no-preserve-root", which isn't quite "--no-seriously-delete-my-root-directory" but is at least heading in that direction.
      • llimllib 5y ago
        if you squint, you can kind of see file permissions as implementing this pattern? rm /usr fails unless you sudo rm /usr
      • gbrown_ 5y ago
        Obligatory Cantrill reference https://youtu.be/wTVfAMRj-7E?t=5046 https://youtu.be/wTVfAMRj-7E?t=5046. The whole series of these are well worth a listen.
    • TFortunato 5y ago
      Not quite the same, but apt commands like apt-get require user interaction to confirm what you are about to do, unless you explicitly add a "-y" / "--yes" to the command.
      • stevekemp 5y ago
        apt will sometimes prompt you to confirm really dangerous options too: sudo apt-get purge login .. WARNING: The following essential packages will be removed. This should NOT be done unless you know exactly what you are doing! login 0 upgraded, 0 newly installed, 1 to remove and 303 not upgraded. After this operation, 1,212 kB disk space will be freed. You are about to do something potentially harmful. To continue type in the phrase 'Yes, do as I say!' ?]
    • 5y ago
  • ben509 5y ago
    I've done that and called the counter option --for-real, but I'd like both args so runbooks can spell out: 1. do-step-one --dry-run 2. (do validation) 3. do-step-one --real-run 4. (do validation) Many commandline parsing libraries assume you're doing booleans, and I think --dry-run and --no-dry-run is confusing. And, internally, you have a boolean flag so there's always the possibility of some code getting it backwards. Internally, I'd like an enum flag that's clearly dry_run or real_run, so the guards are using positive logic: switch(run) { case dry_run: print("Would do this..."); case real_run: do_real_thing(); }
    • Smaug123 5y ago
      coughs gently I use `DryRunMode.Dry` and `DryRunMode.Wet` in the OP.
      • kelnos 5y ago
        I think the point here, though, is that the user needs to be explicit on the command line: they have to specify one of `--wet` or `--dry`; specifying neither is an error. It's not clear from your code if you do that, or if you interpret `--dry-run` as `DryRunMode.Dry` and the absence of an option as `DryRunMode.Wet`. While I kinda like this idea in principle, I haven't really seen any CLI apps that require an explicit option for both modes, so it might be a bit of unexpected UX for people.
      • ben509 5y ago
        I noticed that after I posted and had read a bit more; but you're also using F# which has proper ADTs. I'm typically writing support scripts in Python or the like, and even very good libraries like click[1] nevertheless reflect the idioms popular in the language. I really came back to this post because I was struggling with Gradle and if there is a more perfect illustration of how a dry run mode could help than that, I don't know what it could be. Gradle builds are a fucking mystery, every time, and there's no excuse for it. [1]: https://palletsprojects.com/p/click/ https://palletsprojects.com/p/click/
  • steego 5y ago
    It’s a good opportunity to use long options like: —i-acknowledge-this-is-dangerous-and-i-gave-this-more-than-cursory-thought
  • jakebasile 5y ago
    Yep, this is what I do now. The dry run is the default, you have to flag `--really` to do anything permanent.
  • globular-toast 5y ago
    I don't like this because it trains users to assume that commands can never do dangerous stuff by default.
• overshard 5y ago
  At my work I've gone through and setup a variety of "replacement" commands for commands that may have dire impact on production systems. All of these replacement commands dry run by default (or fake dry run/try to create a dry run the best they can) and require a "--do-it" (think emperor palpatine voice) flag to do the intended operation. I've had multiple coworkers thank me for this setup as it's saved people from silly typos and mistakes, one such being the different handling of programs in the use of and ending "/" in folders. ex. "/etc" is different than "/etc/" sometimes for certain operations. I'd like to one day see a global flag on operating systems to dry run nearly all commands that changed anything.
  • danmur 5y ago
    Ditto :). I use --really though.
    • MAGZine 5y ago
      --void-warranty
      • andrewshadura 5y ago
        --deliberately-do-insane-thing
        • matsemann 5y ago
          React.__SECRET_DOM_DO_NOT_USE_OR_YOU_WILL_BE_FIRED
          • Sohcahtoa82 5y ago
            How about Handler.runWithScissors()? https://android.googlesource.com/platform/frameworks/base/+/android-5.0.0_r1/core/java/android/os/Handler.java#461 https://android.googlesource.com/platform/frameworks/base/+/...
        • wtetzner 5y ago
• lbhdc 5y ago
  I have been incorporating dry run flags into the tools I have been working on (typically etl like tools). Typically this flag only stops the final call to external services that would change the internal state of those services. Its been really helpful when hacking on things, adjusting logic, or validating that input data does what I think it will.
• im3w1l 5y ago
  First I found myself nodding along, but then a troublesome question popped into my mind. What about Toc/tou race conditions?
  • Smaug123 5y ago
    Indeed, that's the thing I rather swept under the rug. I'll definitely take precautions if it's at all easy to avoid the assumption that just because my `--dry-run` stage said something was safe, it will definitely be safe by the time execution passes to the "do-it" stage. However, in general you are indeed doomed the instant you decided to split the tool up this way. I have never actually encountered such a race condition while using any tool I've written this way. But I do remember one place where I specifically didn't add a feature to the tool because it was so vulnerable to a TOC/TOU race condition. Instead, I turned that feature into a "did I just manage to do the thing successfully, and if not, why not" check at the end of the "do-it" stage. So yes, that concern is extremely valid, and if you're doing an operation whose validity is liable to change after you've done the check, then you do just need to follow a different approach for that operation. (But then there's probably no possibility even in principle of a `--dry-run` for such an operation anyway!)
    • mst 5y ago
      Something I've found can work is "check, output plan, prompt for confirmation, if 'yes' execute plan in a way that maximises the odds of bailing out if anything changed while you were displaying the prompt" - for bonus points, recalculate after each action that had to be simulated and do something policy-defined if anything differs from your prior expectations. Obviously the effort of doing this becomes a trade-off but it's worth considering.
  • iudqnolq 5y ago
    I think this can be mitagated with more abstract descriptions, like: "Would have gotten all servers tagged foo (currently 1,341) and deleted them"
    • Smaug123 5y ago
• amirkdv 5y ago
  I'm adamantly pro dry-run and like OP I've found that you really need to design for it. From my experience, OP's two main ideas are spot on: librarification and pushing dry-run logic deeper into library code. Here are two other things I've found: 1. Regardless of where you push your run/dry-run dispatch to, the underlying "do it" logic really needs to be factored properly for individual side effects (think functional). Otherwise, you inevitably end up with loads of "if dry-run / else" soup or worse, bugs _caused_ by your dry-run support. 2. You still need safety nets for when someone is doing dangerous operations without dry-run. Pointing and calling [0] is a great trick for this. For example, say your tool is about to delete X, Y, and Z. Instead of a simple "Yes" confirmation which the user would quickly end up doing on auto-pilot, you could have a more involved confirmation like "Enter the number of resources listed above to proceed" and then only proceed to delete if the user enters "3". Very curious to hear about design idioms folks have come up with! [0] https://en.wikipedia.org/wiki/Pointing_and_calling https://en.wikipedia.org/wiki/Pointing_and_calling
  • kbenson 5y ago
    Usually I just use dry run directives to make sure stuff is working right when I first develop it, so all that is worked out from the very beginning. I also like to develop with a lot of debug output gated behind debug as well, and just leave that there for the inevitable but at some point that would make me add it if not already present. What I don't like is passing those as params as args. Currying this stuff around is very error prone and cumbersome. I prefer to set environment variables for DRY_RUN and DEBUG (with debug accepting higher integer levels for more debugging). This works wonderfully for me since I never have to worry I didn't pass something along correctly, I'm always asking the global set at runtime.
    • Buttons840 5y ago
      So, for a top level bit of code to pass arguments to lower level bits of code you set environment variables? Seems a bit odd to me Using environment variables for the user to configure their environment or pass information into a program seems normal, but code passing arguments to other bits of code in the same language with environment variables seems odd.
• mr-wendel 5y ago
  For all you Bash heads out there: DRYRUN=1 VERBOSE=1 cmd() { [ $DRYRUN -eq 1 -o $VERBOSE -eq 1 ] \ && echo -e "\033[0;33;40m# $(printf "'%s' " "$@")\033[0;0m" >&2 if [ $DRYRUN -eq 0 ]; then "$@" fi } And now you can put this all over your scripts to very easily implement dry-run behavior without any quoting worries. Note that you can't invoke aliases this way. cmd echo "don't worry about quoting" cmd do_something_dangerous "$scary_arg" "$scary_arg2"
  • tremon 5y ago
    Why not simply DRYRUN=echo cmd() { $DRYRUN "$@" } ?
    • mr-wendel 5y ago
      The printf version will give you automatic quoting making copy'n'paste really easy to test. It' also mixes well w/ a verbose option for "--verbose" or such.
      • ebeip90 5y ago
        For those using zsh, you can get better quoting (and only when necessary, and handles non-printable characters and newlines better) than using printf. run() { if dry; then echo "${(q-)@}" else "$@" fi }
    • croehrig 5y ago
      I use this all the time (I call it $EXEC instead), and have gotten into the habit of starting all my scripts with EXEC=echo and put it on every command that does any file changes. It has saved my bacon many times. It doesn't work all the time as the previous poster noted, but it is very low friction which is especially important when writing quick throw-away scripts.
• SavantIdiot 5y ago
  Duplicity has this option. I love it. Especially when something bad can happen, like blowing up my disk because an ISO image is in the backup area.
• zbentley 5y ago
  Even better than a dry-run flag is to have the noop mode be the default behavior and have e.g. "--execute" enable non-read-only behavior (or, for really scary commands like programmatic data destroyers, "--nuke-my-data-i-solemnly-swear-i-know-what-i-am-doing"). Prevents plenty of accidental mistakes by sleepy operators. That's a quibble though, the article is well-put and quite correct about the value of a dry-run mode.
  • zbentley 5y ago
    Oops, I read the article but not the comments. I see I'm far from the first person to suggest this, apologies for the duplication.
• yuchi 5y ago
  Fantastic article, and I promote a stricter approach: `--dry-run` is the default, and you need to disable with `--no-dry`. For irreversible (such as deletions) actions I even go one step further with a `--no-dry=<some-non-trivial-value>` in order to force some thinking. Edit: apparently this is way more common than I thought, by reading comments here!
• agumonkey 5y ago
  I wonder how it would feel to have every program dry-runnable but composable.. basically FP/virtual systems. You can (dry-run cmd-0) | (dry-run ...) | (dry-run cmd-n) and it will yield a potential system state.
• notafraudster 5y ago
  How do you all write dry-run logic for utilities that perform file operations (edits, deletions, creations) and have some kind of sequential dependency in operations such that later steps depend on the previous steps having been executed? I have found it a frustrating nut to crack because it seems like it needs to involve either writing a simulation layer that tracks the state of all the file calls virtually, or else copy all the files to a temp folder (so the dry run isn't dry, it's just on a separate version of the data). Both of these seem like bad solutions.
  • programmarchy 5y ago
    Copying to a temp folder sounds pretty reasonable to me.
  • lamontcg 5y ago
    Yes, if you call a black box outside of your code base which does something then subsequent operations can't easily be dry-run'd. So if you need to install a package which sets up a systemd service, the subsequent dry-run of managing that systemd service would fail because the service doesn't exist yet. Or you need to make assumptions in the service dry run that it should have been set up before. You can just assume that the service would have been setup and report that you were, say, enabling and starting the service. Or you could require some kind of hint to be setup from the package installation to the specific service configuration (for well known things this could be done by default, but for arbitrary user-supplied packages this cannot be done). Or you could list the entire package contents and try to determine if the manifest looks like it is configuring a service. And it still may fail because you don't know the contents of that file without extracting it and the service may not parse at all. And that's just a simple package-service interaction example. You could spend a week noodling on how to do that fairly precisely, then there's a hundred or a thousand other interactions to get correct. You're being told not to do the thing, so there's fundamentally a black-box there, and you need to figure out how much you're going to cheat, how much you're going to try to crack open the black box into some kind of sandbox so you can figure out its internals, and how much you're going to offload onto the user. Not actually an easy problem at all.
• wheybags 5y ago
  I've made a further extension of this principle for a video game release script, by adding a --unsafe flag. The script was responsible for building the game on a bunch of platforms, uploading it to various storefronts and announcing the release on twitter, reddit etc. There was a --dry-run flag too, that just prints what it would do, but sometimes you want to test part of the script without doing anything public. There were various flags like --no-upload and --no-twitter etc, but it was easy to forget them, so I added in a top level --unsafe flag. At every point where we do something public, at the deepest level of the call stack just before doing it, I check for unsafe, and if its not present, throw an exception. It worked pretty well, and definitely saved my ass a few times, especially during refactoring.
• aranchelk 5y ago
  Dry-run can be implemented succinctly in Python with a function decorator wrapping those functions that should run conditionally. Nice aspects of this technique: Inside the decorator you have access to both the function’s name and its arguments so you can print descriptive messages to the user e.g. “In dry run, would have run function ‘delete’ on ‘/foo/bar’” You can use the name of the decorator as de facto documentation, e.g. “@destructive”
• kevincox 5y ago
  Their example of pushing it into the type system is nice but requires a language with a fairly powerful type system. Once simple way to do this in basically any statically typed language is just requiring a "Run Token" for any bit of code that does something "for real". This method works well if you want the output to be similar or identical for dry run and actual run. struct RunToken; fn do_the_thing(_: RunToken, ...); The general approach works for just about any statically typed languages (although null is the enemy). You can make it a bit more tedious to generate that RunToken so that a `do_the_think(RunToken, ...)` looks more natural. But the idea is simply that if the only place you create a RunToken is when parsing the --no-dry-run flag you can't accidentally do_the_thing when you move the code out of the `if is_dry_run` block without noticing. Of course this is a simple way to get fairly reliable dry run. I do agree that having seperate plan and apply steps is approach is the best when you can do it.
  • AgentME 5y ago
    This is almost like one step toward an object-capability system. In an object-capability language, the main function would receive capability objects, like a reference to a module for accessing the filesystem, and functions could only use those capabilities if they were passed a reference to them. You could write your main function so it takes the filesystem module, creates a wrapper around it that only exposes read-only functionality, passes that read-only wrapper around to most functions that build up some kind of plan, and then the main function uses the raw unrestricted filesystem module to execute the plan at the end depending on the dry-run flag. You can then be easily sure that code doesn't do any writes to the filesystem if the unrestricted filesystem module isn't passed to it.
• swyx 5y ago
  all CLIs that do substantial work should have dry runs. more examples I've collected: - Angular CLI https://malcoded.com/posts/angular-fundamentals-cli/#--dry-run-try-things-out https://malcoded.com/posts/angular-fundamentals-cli/#--dry-r... - AWS CLI https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-help.html https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-h... - Rspec https://relishapp.com/rspec/rspec-core/v/3-8/docs/command-line/dry-run-option https://relishapp.com/rspec/rspec-core/v/3-8/docs/command-li... - Serverless framework https://forum.serverless.com/t/dry-run-with-serverless-framework-stack/8155/3 https://forum.serverless.com/t/dry-run-with-serverless-frame... at netlify we didnt do a dry run but Netlify Dev is close: https://news.ycombinator.com/item?id=19615546 https://news.ycombinator.com/item?id=19615546 I'm requesting one at Gatsby: https://github.com/gatsbyjs/gatsby/discussions/16384 https://github.com/gatsbyjs/gatsby/discussions/16384
• slver 5y ago
  Dry run is OK, but if we want to look at it closer, it occurs in tools with mixed query + command responsibilities. For example rm PATTERN Is really a shortcut for something like (pseudo code): find -name PATTERN | rm If things were always factored for command/query seggregation you'd not need dry-run, you'd simply just run the query.
  • rwl 5y ago
    This is a good point, but notice that what makes the segregation easy in your example is that the query and the command have a shared way to refer to what the command will operate on: the filename. Consider a conceptually very similar case: instead of finding and deleting files in a filesystem, think about finding and deleting lines matching a pattern within a file. Then the query is something like grep...but what do you put for the command on the other side of the pipe? Of course, you can tell grep to output line numbers and use a command that operates on line numbers, or similar. The point is, in order to achieve this kind of segregation, you need some common way of naming operands on both sides of the divide. And naming things is hard, so segregating things this way is hard, and thus there's a lot of tools with mixed query/command responsibilities.
    • slver 5y ago
      PowerShell solved this problem by outputting easy to map objects instead of just plain text. Unfortunately they made it a bit more verbose than it could've been, and the industry didn’t recognize the potential, so it remained a niche product. I still love PowerShell.
      • GlennS 5y ago
        I love PowerShell too, but I think the real barrier to adoption is that it's just too slow.
        • slver 5y ago
          The concept of object output isn't slow. PS is mostly slow because it's based on .NET and they didn't bother optimizing it.
• da39a3ee 5y ago
  This is obvious but a good way to implement dry run in some settings is to obtain a read-only connection to the relevant datastore / roll back a transaction without committing, etc.
• xupybd 5y ago
  Gresearch puts out some interesting articles, and they work in F#. I wish I were in the UK, I'd love to try and get a job there.
  • dkarp 5y ago
    They’re an interesting organization. See this past submission: https://news.ycombinator.com/item?id=18499712 https://news.ycombinator.com/item?id=18499712
  • Smaug123 5y ago
    I believe (though can't swear to it) that we do provide immigration/relocation assistance and money, at least for quant researchers and possibly for all employees. If that would be enough to get you interested, feel free to drop me a line at patrick.stevens@gresearch.co.uk and I can find out actual details rather than half-remembered possibly-false snippets from years ago.
    • xupybd 5y ago
      Sadly money is not the issue proximity to family is. I'll just hope one day they open an NZ or south east Asia branch.
• JustSomeNobody 5y ago
  I use --dry-run with rsync almost every time I use rsync. I don't really use rsync enough to remember if I should use a trailing slash or not...
• blt 5y ago
  Abstracting a bit, this is an application of the principle "represent important intermediate results as explicit data structures". It shows up in many places: testing, extensibility, logging, and parallelization are all easier to implement when the result is represented explicitly instead of implicitly in the program's call stack. Has anyone written about this idea? It seems to have some interesting interaction with laziness.
• deckard1 5y ago
  If you're doing a lot of modifications to a filesystem or a database, one pattern I've used and liked in the past is to have your code simply dump out the commands to run. Usually to stdout and you save it as a script. That way you don't need to worry about issues with bugs in the dry run logic and you have a record (a script) of what you actually ran. Even better if you can extend this output script so that it's idempotent and fails on the first failed command.
• nerpderp82 5y ago
  Even better is to invert the flag, that default is to dump the plan and have to pass an `--actually-do-it` to have it make the changes.
• ayoisaiah 5y ago
  This resonates with me. I recently published a cross-platform tool [1] for bulk renaming files with dry run as the default. In this mode, it shows the changes about to be made and any possible problems that may occur (such as overwriting existing files). To actually carry out the changes a flag must be used. I felt it was a good design decision since the effects of bulk renaming can be substantial. Most other similar tools have it backwards with their dry-run mode relegated to a secondary action. [1]: https://github.com/ayoisaiah/f2 https://github.com/ayoisaiah/f2
• voakbasda 5y ago
  Simulation is a great thing. I use this wrapper, even in my most trivial shell scripts: pretend=${scriptname_pretend:-false} run() { echo "$@"; $pretend || "$@"; } Sure, this doesn't give copy/paste re-usable output, but it's really just a quick and dirty sanity check when things start getting complicated.
• gm 5y ago
  What do you guys think of "dry run by default" where you have to turn dry run OFF via an argument? I've done this on some of my utilities just because screwing up the arguments would result in much grief (lost data).
• gorgoiler 5y ago
  I don’t even trust dry run for some things. Yesterday I wrote a cleanup tool. In fact it’s two tools: one that outputs a parseable list of things to clean up... # snapshotX # keep snapshotY # delete ...and another that consumes the list and does stuff. It’s the only way to be sure.
• FounderBurr 5y ago
  One lesson from making tools over the years having dry-run options is to always very explicitly state what you mean by dry-run. The most common use is as a write/modify gate, but many tools have uses where it may need to avoid certain classes of reads as well (network, db, etc).
• rlonn 5y ago
  I've found that while having a dry-run code path is always super useful, having a --dry-run flag is an anti-pattern. I prefer to make dry run the default, with all state-changing actions being in the form: if (clear_db_flag) { if (verbosity_level > 0) print("DROP DATABASE customers;") if (execute_flag) exec_sql("DROP DATABASE customers;") } I.e. instead of a --dry-run flag to make the script harmless, I have an --execute flag that weaponizes it. This means that while developing the tool I'm more likely to run in dry-run mode (i.e. without the --execute flag) and so test the dry-run path more than I perhaps would have. It also means I'm not very likely to delete the production database by accident, through running my script in what I intended to be dry-run mode but actually ending up deleting something I really didn't want to delete. I think it's a nice methodology, to always have a print statement first, then perform an action that the print statement informed the user about. Easy to remember to add the print statements and more likely that you keep the dry-run code path up to date.
• jiggawatts 5y ago
  In the PowerShell world: "-WhatIf" is your friend. To implement this in your own scripts is not difficult at all. The template snippet below automatically wires up "-Confirm" and "-WhatIf" support: [CmdletBinding(ConfirmImpact='High',SupportsShouldProcess=$true)] PARAM( [parameter(ValueFromPipeline=$true,Mandatory=$true)] $TheThingToProcess ) PROCESS { if ( $PSCmdlet.ShouldProcess( $TheThingToProcess, 'Do scary thing' )) { # Scary step } } That even wires up the full "[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend" logic, so you don't have to track boolean variables all over the place. It also passes through that preference to commands invoked witin the script, so things like "Format-Volume" will do nothing if you invoke the outer script with "-WhatIf".