5 ms·

Please also consider a --dry-run option that gives a preview of what actions would be taken without actually making any changes. This is really helpful for lea

Please also consider a --dry-run option that gives a preview of what actions would be taken without actually making any changes. This is really helpful for learning a tool and making sure you got complex options and fileglobs correct before committing to potentially irreversible changes.

samstave 3y ago
dry-run should be a pipable | function for any command; > do_thing.py | dry-run -- Think of it as "explain your work, step by step" as one would prompt... also, its a food-for-thought you muppets. --- @jasonjmcghee ; ( $ ) . ( $ ) great justice.
- deleted 3y ago
  [deleted]
- jasonjmcghee 3y ago
  How could that work? The script on the left executes first, piping its output to another command won’t prevent that
  - saghm 3y ago
    I think there are ways to detect if stdout is a pipe and operate differently (e.g. by using different defaults for coloring output), but I'm not sure if there are ways to detect what the other side of the pipe is, much less what `dry-run` would actually be expected to be in this case
    - jasonjmcghee 3y ago
      Maybe I’m misunderstanding but sounds like you’re proposing the modifying the command on the left to detect whether it’s piping it’s output (curious how you do that btw sounds pretty cool / useful) But at that point you could just handle —dry-run directly
      - kergonath 3y ago
        > Maybe I’m misunderstanding but sounds like you’re proposing the modifying the command on the left to detect whether it’s piping it’s output (curious how you do that btw sounds pretty cool / useful) You can do that. Possibly not from all languages but for anything that can call functions in the c standard library, that’s what isatty() is for (among other uses). It takes a file descriptor and returns whether it is a terminal or not. If you do this with stdout, this tells you whether it goes to a terminal or whether it is redirected in a way. As the parent suspects, though, this won’t tell you anything about what is on the other side of the redirection.
        jasonjmcghee 3y ago
        Very cool function - thank you!
        epcoa 3y ago
        It also doesn’t tell you what is the terminus of a pipeline. Because often with isatty what you really want to know is if the pipeline ends with a tty. `ls` is screen formatted but `ls | less` is not without extra work.
    - kjs3 3y ago
      If it operated differently based on what it was outputting to, that kinda defeats the point of 'dry run', which is to see exactly what's going to happen based on what's on the other side of the pipe when I run it for real. "Did this blow up because it's a bad command, or because there's a bug in the 'only kinda real dry run' code?".
      - samstave 3y ago
        +++ - What if there wasa 'deep-pipe' '||' which would be based on a set env/docker/blah - which would launch an env and execute your '||'d code in it, and output some log/metrics/whatever?
    - gpderetta 3y ago
      on the other hand you could have 'dry-run <command>' that via .so interposition tricks could intercept and list all destructive changes done by an arbitrary <command>, as a form of sandboxing.
      - account42 3y ago
        Not generally because you can't e.g. know if a write to a socket is destructive or just a query for information needed to decide the next steps/output.
        gpderetta 3y ago
        True, I was only thinking of fs access!
    - sokoloff 3y ago
      I often pipe output to tee and would be pretty annoyed if that changed the behavior of the original command to not do anything because stdout was a pipe.
      - indymike 3y ago
        > I often pipe output to tee and would be pretty annoyed if that changed the behavior of the original command The output sent to tee is usually not the same as the output from the command to the terminal, so you are getting something different than most human users expect from original command... the reason is that terminal escape codes and other formatting for humans may need to be omitted from output to a pipe. You do this by asking the OS, "is this thing a terminal?". Python example "terminal" if sys.stdout.isatty() else "something else" C is very similar: if (isatty (1)) fprintf (stdout, "Terminal."); else fprintf (stdout, "Not Terminal."); (printf works, too).
        sokoloff 3y ago
        Sure; the output format changes, but the functionality doesn't. Even ls outputs a tabular format by default when it's on a terminal and a list one file/dir per line when it's not on a terminal (if it's piped to cat for example or why ls | wc -l correctly counts the entries). But the (essential) behavior of the command remains the same. ls still lists files/dirs... scp still copies files, etc.
        indymike 3y ago
        > But the (essential) behavior of the command remains the same. Of course. A command needs to do it's defined function. You'll find some programs that are quite a bit different when invoked from outside the terminal vs inside the terminal. Developers need to take into account both situations, which is really the point the original post.
        saghm 3y ago
        You can see this pretty easily with `ls` as well by using different options for the color. If you run `ls --color=auto`, you'll get colored output when running directly but black and white output if you pipe to `less`. However, if you pass `--color=always`, you'll get colored output when running directly and a bunch of garbage around some of the entries when piping to `less` because it doesn't interpret ANSI escape codes for color by default (although depending on what the output you're piping into `less` is, there are some workarounds like https://www.gnu.org/software/src-highlite/ https://www.gnu.org/software/src-highlite/)
  - int_19h 3y ago
    Here's a possibly more interesting take on this: instead of do-thing.py, imagine if there was thing.py, which processed all that complex CLI options, and as output produced a linear shell script, with each line invoking some atomic operation, and a comment explaining which options contributed to that particular line.
    - eichin 3y ago
      Yeah, that's a pattern I use for occasional sysadmin tools - the command itself generates the actual commands, I review them, then "<up arrow> | sh -xeu". (Yes, there's no guarantee that the output is the same, so I don't use the pattern when that's a risk; it's also rarely if ever used for things I expect other people to run, just bulk operations that are saving me repetition.)
- koolba 3y ago
  Having it be a shell function would work. It could create a copy-on-write file system or override the syscalls for file system access.
  - agos 3y ago
    not all the destructive actions are on file system
- jasonjmcghee 3y ago
  Wasn’t trying to be a prick- i was commenting “here’s what I’m seeing with your proposal and the presented obstacles, how would you overcome them?” One approach could be something like “set -x” after setting a confirmation with “trap” command. confirm_execution() { echo -n "Execute $BASH_COMMAND? [y/N] " read response if [[ $response != [yY] ]]; then echo "Skipped." return 1 fi } trap 'confirm_execution' DEBUG set -x user_command set +x But that’s a wrapper script
  - samstave 3y ago
    confirm_execution() { echo -n "Execute $BASH_COMMAND? [y/N] " read response if [[ $response != [yY] ]]; then echo "Skipped." return 1 fi } draw_heart() { cat << EOF <svg xmlns="http://www.w3.org/2000/svg http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="100" height="100"> <path fill="red" d="M12 21.35l-1.45-1.32C5.4 15.36 2 12.28 2 8.5 2 5.42 4.42 3 7.5 3c1.74 0 3.41.81 4.5 2.09C13.09 3.81 14.76 3 16.5 3 19.58 3 22 5.42 22 8.5c0 3.78-3.4 6.86-8.55 11.54L12 21.35z"/> </svg> EOF } trap 'confirm_execution' DEBUG set -x draw_heart set +x
- foobarqux 3y ago
  "try" is not a prefix (like watch, nice, etc) uses an overlayfs in order to be able to see and accept or reject changes to your filesystem from a command https://github.com/binpash/try https://github.com/binpash/try
epage 3y ago
Or if the command can't be undone and with significant side effects, be `--dry-run` by default and have an `--execute` flag
- BD103 3y ago
  Exactly. This is implemented by the Javascript code formatter Prettier [1] where you have to pass `--write` in order to overwrite an unformatted file. [1]: https://prettier.io https://prettier.io
bongodongobob 3y ago
Whatif is one of my favorite things about powershell. Don't know what I'd do without it. Has saved me many times from completely destroying things.
- ojintoad 3y ago
  Agree! I'll go further. PowerShell covers a lot of the concerns in the OP out of the box. It is extremely well thought out and is one of my favorite cli models to buy into.
  - bigstrat2003 3y ago
    Powershell is just plain a great shell. Blows all the *sh variants out of the water. I'd love for it to gain traction in Linux (so that, for example I could use it as my shell on my desktop) but I don't really see that happening.
    - partdavid 3y ago
      What are the things that are stopping you? I use MacOS on my personal machine and Linux for various shellboxes and I switched to Powershell years ago and haven't looked back. Occasionally I invoke bash as a language runtime for checking shell script stuff, the way I would any other language's REPL, but for a shell? Powershell is strictly better. The one actual problem with Powershell in this area is quoting for external commands. It's solvable in scripts by replacing certain things with double-quoted equivalents but not really interactively, and it is an occasional pain (though I still think it's overall less of a problem than quoting in general in bash and its ilk).
- snuxoll 3y ago
  WhatIf in Powershell also remains completely broken; not that it doesn't work, but it is not properly passed down the call stack like it's supposed to from cmdlets or functions written in PowerShell (as opposed to those loaded from .net assemblies).
  - 3y ago
danstewart_ 3y ago
I tend to go the opposite way and have the default behaviour not actually make any changes and require passing `--commit` to actually do something. I feel it’s safer for scripts that have irreversible (or difficult to reverse) actions.
- enriquto 3y ago
  Even safer: your program does never perform any actual deed. It just prints commands that you can run afterwards, using an external program, ideally a shell. This has the advantage of allowing the user to edit all the actions one by one. Instead of myprog # see what would happen myprog --commit # alright, do it You do myprog # see what would happen myprog | sh # alright, do it But if you want to change something: myprog > x vi x cat x | sh And if you just want to run everything in parallel: myprog | parallel -j 16
azemetre 3y ago
Do you have an example of a CLT that has a dry run flag? I am very confused on how to design one for some CLTs I'm making. If you call an API to retrieve data, how can that be a dry run? Are you suppose to give fake examples with fake output?
- scbrg 3y ago
  apt-get. It simply tells you what it's going to remove/install. Calling an API to retrieve data is not really the type of program that requires a dry-run flag. It's mainly useful for commands that change the state of something in ways that could potentially be destructive, unwanted and/or hard to revert.
- amake 3y ago
  The AWS CLI has --[no]-dry-run in many of its subcommands. https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-help.html https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-h...
- funcDropShadow 3y ago
  make has -n or --dry-run. But originally it was called -n and that has been used many cli tools.

samstave 3y ago

dry-run should be a pipable | function for any command; > do_thing.py | dry-run -- Think of it as "explain your work, step by step" as one would prompt... also, its a food-for-thought you muppets. --- @jasonjmcghee ; ( $ ) . ( $ ) great justice.

deleted 3y ago
[deleted]
jasonjmcghee 3y ago
How could that work? The script on the left executes first, piping its output to another command won’t prevent that
- saghm 3y ago
  I think there are ways to detect if stdout is a pipe and operate differently (e.g. by using different defaults for coloring output), but I'm not sure if there are ways to detect what the other side of the pipe is, much less what `dry-run` would actually be expected to be in this case
  - jasonjmcghee 3y ago
    Maybe I’m misunderstanding but sounds like you’re proposing the modifying the command on the left to detect whether it’s piping it’s output (curious how you do that btw sounds pretty cool / useful) But at that point you could just handle —dry-run directly
    - kergonath 3y ago
      > Maybe I’m misunderstanding but sounds like you’re proposing the modifying the command on the left to detect whether it’s piping it’s output (curious how you do that btw sounds pretty cool / useful) You can do that. Possibly not from all languages but for anything that can call functions in the c standard library, that’s what isatty() is for (among other uses). It takes a file descriptor and returns whether it is a terminal or not. If you do this with stdout, this tells you whether it goes to a terminal or whether it is redirected in a way. As the parent suspects, though, this won’t tell you anything about what is on the other side of the redirection.
      - jasonjmcghee 3y ago
        Very cool function - thank you!
      - epcoa 3y ago
        It also doesn’t tell you what is the terminus of a pipeline. Because often with isatty what you really want to know is if the pipeline ends with a tty. `ls` is screen formatted but `ls | less` is not without extra work.
  - kjs3 3y ago
    If it operated differently based on what it was outputting to, that kinda defeats the point of 'dry run', which is to see exactly what's going to happen based on what's on the other side of the pipe when I run it for real. "Did this blow up because it's a bad command, or because there's a bug in the 'only kinda real dry run' code?".
    - samstave 3y ago
      +++ - What if there wasa 'deep-pipe' '||' which would be based on a set env/docker/blah - which would launch an env and execute your '||'d code in it, and output some log/metrics/whatever?
  - gpderetta 3y ago
    on the other hand you could have 'dry-run <command>' that via .so interposition tricks could intercept and list all destructive changes done by an arbitrary <command>, as a form of sandboxing.
    - account42 3y ago
      Not generally because you can't e.g. know if a write to a socket is destructive or just a query for information needed to decide the next steps/output.
      - gpderetta 3y ago
        True, I was only thinking of fs access!
  - sokoloff 3y ago
    I often pipe output to tee and would be pretty annoyed if that changed the behavior of the original command to not do anything because stdout was a pipe.
    - indymike 3y ago
      > I often pipe output to tee and would be pretty annoyed if that changed the behavior of the original command The output sent to tee is usually not the same as the output from the command to the terminal, so you are getting something different than most human users expect from original command... the reason is that terminal escape codes and other formatting for humans may need to be omitted from output to a pipe. You do this by asking the OS, "is this thing a terminal?". Python example "terminal" if sys.stdout.isatty() else "something else" C is very similar: if (isatty (1)) fprintf (stdout, "Terminal."); else fprintf (stdout, "Not Terminal."); (printf works, too).
      - sokoloff 3y ago
        Sure; the output format changes, but the functionality doesn't. Even ls outputs a tabular format by default when it's on a terminal and a list one file/dir per line when it's not on a terminal (if it's piped to cat for example or why ls | wc -l correctly counts the entries). But the (essential) behavior of the command remains the same. ls still lists files/dirs... scp still copies files, etc.
        indymike 3y ago
        > But the (essential) behavior of the command remains the same. Of course. A command needs to do it's defined function. You'll find some programs that are quite a bit different when invoked from outside the terminal vs inside the terminal. Developers need to take into account both situations, which is really the point the original post.
      - saghm 3y ago
        You can see this pretty easily with `ls` as well by using different options for the color. If you run `ls --color=auto`, you'll get colored output when running directly but black and white output if you pipe to `less`. However, if you pass `--color=always`, you'll get colored output when running directly and a bunch of garbage around some of the entries when piping to `less` because it doesn't interpret ANSI escape codes for color by default (although depending on what the output you're piping into `less` is, there are some workarounds like https://www.gnu.org/software/src-highlite/ https://www.gnu.org/software/src-highlite/)
- int_19h 3y ago
  Here's a possibly more interesting take on this: instead of do-thing.py, imagine if there was thing.py, which processed all that complex CLI options, and as output produced a linear shell script, with each line invoking some atomic operation, and a comment explaining which options contributed to that particular line.
  - eichin 3y ago
    Yeah, that's a pattern I use for occasional sysadmin tools - the command itself generates the actual commands, I review them, then "<up arrow> | sh -xeu". (Yes, there's no guarantee that the output is the same, so I don't use the pattern when that's a risk; it's also rarely if ever used for things I expect other people to run, just bulk operations that are saving me repetition.)
koolba 3y ago
Having it be a shell function would work. It could create a copy-on-write file system or override the syscalls for file system access.
- agos 3y ago
  not all the destructive actions are on file system
jasonjmcghee 3y ago
Wasn’t trying to be a prick- i was commenting “here’s what I’m seeing with your proposal and the presented obstacles, how would you overcome them?” One approach could be something like “set -x” after setting a confirmation with “trap” command. confirm_execution() { echo -n "Execute $BASH_COMMAND? [y/N] " read response if [[ $response != [yY] ]]; then echo "Skipped." return 1 fi } trap 'confirm_execution' DEBUG set -x user_command set +x But that’s a wrapper script
- samstave 3y ago
  confirm_execution() { echo -n "Execute $BASH_COMMAND? [y/N] " read response if [[ $response != [yY] ]]; then echo "Skipped." return 1 fi } draw_heart() { cat << EOF <svg xmlns="http://www.w3.org/2000/svg http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="100" height="100"> <path fill="red" d="M12 21.35l-1.45-1.32C5.4 15.36 2 12.28 2 8.5 2 5.42 4.42 3 7.5 3c1.74 0 3.41.81 4.5 2.09C13.09 3.81 14.76 3 16.5 3 19.58 3 22 5.42 22 8.5c0 3.78-3.4 6.86-8.55 11.54L12 21.35z"/> </svg> EOF } trap 'confirm_execution' DEBUG set -x draw_heart set +x
foobarqux 3y ago
"try" is not a prefix (like watch, nice, etc) uses an overlayfs in order to be able to see and accept or reject changes to your filesystem from a command https://github.com/binpash/try https://github.com/binpash/try

epage 3y ago

Or if the command can't be undone and with significant side effects, be `--dry-run` by default and have an `--execute` flag

BD103 3y ago
Exactly. This is implemented by the Javascript code formatter Prettier [1] where you have to pass `--write` in order to overwrite an unformatted file. [1]: https://prettier.io https://prettier.io

bongodongobob 3y ago

Whatif is one of my favorite things about powershell. Don't know what I'd do without it. Has saved me many times from completely destroying things.

ojintoad 3y ago
Agree! I'll go further. PowerShell covers a lot of the concerns in the OP out of the box. It is extremely well thought out and is one of my favorite cli models to buy into.
- bigstrat2003 3y ago
  Powershell is just plain a great shell. Blows all the *sh variants out of the water. I'd love for it to gain traction in Linux (so that, for example I could use it as my shell on my desktop) but I don't really see that happening.
  - partdavid 3y ago
    What are the things that are stopping you? I use MacOS on my personal machine and Linux for various shellboxes and I switched to Powershell years ago and haven't looked back. Occasionally I invoke bash as a language runtime for checking shell script stuff, the way I would any other language's REPL, but for a shell? Powershell is strictly better. The one actual problem with Powershell in this area is quoting for external commands. It's solvable in scripts by replacing certain things with double-quoted equivalents but not really interactively, and it is an occasional pain (though I still think it's overall less of a problem than quoting in general in bash and its ilk).
snuxoll 3y ago
WhatIf in Powershell also remains completely broken; not that it doesn't work, but it is not properly passed down the call stack like it's supposed to from cmdlets or functions written in PowerShell (as opposed to those loaded from .net assemblies).
- 3y ago

danstewart_ 3y ago

I tend to go the opposite way and have the default behaviour not actually make any changes and require passing `--commit` to actually do something. I feel it’s safer for scripts that have irreversible (or difficult to reverse) actions.

enriquto 3y ago
Even safer: your program does never perform any actual deed. It just prints commands that you can run afterwards, using an external program, ideally a shell. This has the advantage of allowing the user to edit all the actions one by one. Instead of myprog # see what would happen myprog --commit # alright, do it You do myprog # see what would happen myprog | sh # alright, do it But if you want to change something: myprog > x vi x cat x | sh And if you just want to run everything in parallel: myprog | parallel -j 16

azemetre 3y ago

Do you have an example of a CLT that has a dry run flag? I am very confused on how to design one for some CLTs I'm making. If you call an API to retrieve data, how can that be a dry run? Are you suppose to give fake examples with fake output?

scbrg 3y ago
apt-get. It simply tells you what it's going to remove/install. Calling an API to retrieve data is not really the type of program that requires a dry-run flag. It's mainly useful for commands that change the state of something in ways that could potentially be destructive, unwanted and/or hard to revert.
amake 3y ago
The AWS CLI has --[no]-dry-run in many of its subcommands. https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-help.html https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-h...
funcDropShadow 3y ago
make has -n or --dry-run. But originally it was called -n and that has been used many cli tools.