7 ms·

It seems like most of the scariness could go away if only you had to press a key combo each time you wanted to run the command / render the preview

jolmg 8y ago
Yes. Like Enter? People might still like the fact that the cursor doesn't need to move from wherever it's editing the command to run it, and can still continue to edit there.
- akavel 8y ago
  That's the main direction I currently seem to be coalescing on based on feedback I'm getting. That said, one person on lobste.rs suggested trying to play with Linux capabilities, to make the filesystem immutable... now that would be a seriously fancy trick if it works!
  - rcthompson 8y ago
    Consider that the filesystem isn't necessarily the only thing you can mess up by accident. It's much more unlikely, but you could accidentally trigger a bunch of unwanted network requests (e.g. piping into `xargs curl` or something), among other possible things with external effects.
    - akavel 8y ago
      Maybe cutting off network access could be done too? That would reduce some potentially useful features, but it could be a parameter to enable it (hey, we're talking Linux, never enough parameters! ;). That is, assuming that a syscalls/capabilities trick like that is possible at all. (Anyone can say more here? Some kernel hackers? This is HN, ain't it?)
      - rcthompson 8y ago
        Yeah, I think almost any ability to produce an effect other than printing to stdout and stderr should probably be restricted by default, with a flag to enable it. If people adopt this and frequently use specific capabilities, they can add a shell alias to add enable their preferred set of capabilities.
        rcthompson 8y ago
        Or, one possible alternative would be to always prevent any side-effects other than printing output, and then display a warning saying something like "This command tried to access the network/modify the filesystem/be naughty in some other way. To allow this and re-run the command, press Control+Shift+Enter"
      - em-bee 8y ago
        capabilities should certainly work. for portability, setting a non-existant proxy should also help. most commandline tools should honor shell proxy variables and fail the request. greetings, eMBee.
  - jolmg 8y ago
    Too fancy in my opinion. Personally, I prefer KISS solutions. Having an immutable filesystem in up would be unexpected, and I'm sure there's other destructive things that can be done that don't involve the filesystem. As odd as it may seem, I may also want to be able to do filesystem destructive things in up, why shouldn't I be able too? EDIT: As an example, I may exploratively be playing with find options to match different sets of files in a directory hierarchy. Once I see I've matched the files I want, I may want to add a `-delete` to the end of the find conditions I specified to delete them all. That seems like a useful use of up, but making the filesystem immutable would disallow it.
    - akavel 8y ago
      Hm. I think in my ideal vision, you'd then quit up, and somehow magically have the pipeline ready at your fingertips on the shell prompt, so that you can just add the `-delete` there in a "regular" fashion, and press Enter. Not quite sure how this could be done with bash easily. I think the upN.sh file is not a bad approximation however. (Someone on reddit (I think) seemed to mention that zsh seems to have some functionality which could allow building something like that over up.) Interestingly, this seems to me to support the idea that limiting syscalls/capabilities could be a good approach (if at all possible, that is). Potentially maybe also cutting off network access? But even if I add some safety options, even if I make them the defaults, I'll try really hard to keep an option available for people who want to play with it raw, with no safety belts. edit: I totally get you as far as KISS goes; that's why I built and shared the tool as is in the first place ;)
      - jolmg 8y ago
        > Interestingly, this seems to me to support the idea that limiting syscalls/capabilities could be a good approach (if at all possible, that is). Potentially maybe also cutting off network access? Yeah, I still think it's not a good approach. rcthompson gave an example with network access, but that doesn't mean that bad things can only happen with filesystems or network access. You can also kill processes, shutdown the computer, and many more things. I don't think you can guess what mistakes the user might do, or which were really mistakes or were intentional. Also, if you can avoid special permissions or capabilities for your core functionality, then that's better. People should be conservative in giving special permissions to programs, and I can't think it makes sense to give "up" the ability to make a filesystem immutable even if it's in an isolated namespace.
  - twanvl 8y ago
    Another solution is to only allow commands from a whitelist to be run. If the user types something not on the list you say "rm is not on the list of allowed commands, press enter to run and add it".
    - ElijahLynn 8y ago
      ++ And rather a blacklist, since a blacklist would be much smaller than a potentially an infinite amount of programs to manage for a whitelist.
      - chrisshroba 8y ago
        I think the whitelist makes more sense, since you can never say for sure that you've added every potentially harmful program to the blacklist. If it's easy to add things to the whitelist, as the parent comment proposed, I think the idea would work very well, especially considering most people have a relatively small number of commands they pipe other commands into. Btw, if you want to see all the commands you've ever piped data into, sorted by frequency, you can use this command (which I put together using `up`!) history | grep -o '| \w\w*' | sort | uniq -c | sort -n | nl
  - deelowe 8y ago
    I think from a design perspective, it's better to build your sanity checks and safeguards into the UI instead of some fancy abstraction that's running in the background. Throw in some options to make it user configurable and you have a pretty slick solution that behaves in a way that'll feel familiar to gurus. I think requiring a keypress by default to run any commands and maybe even throwing up a warning for a list of known dangerous ones (perhaps user configurable) would work well. A really fancy solution would be to do this stuff via some sort of linting.
    - akavel 8y ago
      I don't see how linting could work, I'm afraid. There are obscure options in zip, git, grep, and what else, not to mention full-blown interpreters like awk and perl, that can edit files via a myriad of syntaxes, and then there's the halting problem. Similarly with whitelist/blacklist, I'm not currently really convinced it could help much; take `xargs rm`, vs. some hyphotetical `foobar -rm` command + params (where maybe -r = recursive, -m = monitor?) Keypress, on the other hand, sounds to me like it could be a reasonable default compromise... still with an option to switch to "fully accelerated, fast mowing" mode at any point, if one so desires...
      - deelowe 8y ago
        Agreed. As another commenter said KISS.
      - jolmg 8y ago
        > `foobar -rm` command + params (where maybe -r = recursive, -m = monitor?) An example of such a foobar with those exact options would be `inotifywait`. :)
  - Spakman 8y ago
    Another thing you might want to investigate (for Linux, not sure about cross-platform alternatives) is OverlayFS[1], which allows you to create union filesystems where some layers are read-only and the top layer is read-write. This can allow destructive operations like rm without actually removing anything from the lower (read-only) layers, using whiteouts. I'd have replied to this on lobste.rs but don't have an account. I'm not a heavy poster but I'd love an invite if anyone has one going :) 1 - https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt https://www.kernel.org/doc/Documentation/filesystems/overlay...
    - akavel 8y ago
      I need your email for an invitation; if you can send me your address via my own email (see my profile), or via keybase, I'll send you an invite. I don't see your email advertised in your profile here on HN, so my hands are tied for now :)
      - Spakman 8y ago
        Thank you very much! I hadn't realised my email address isn't visible here. Maybe I need to move it into the "about" section. In case you aren't aware, your email address isn't visible in your profile either. I've emailed the GMail address that I found on your website :)
  - curun1r 8y ago
    Can I say that I hope you don't make this tool too Linux-specific. It seems to run pretty reasonably on Mac and would probably also work on BSD variants. Since I use all three (Mac, BSD, Linux) on a semi-regular basis, I'm pretty unlikely to integrate a tool into my Unix workflows if it doesn't work reliably on most unixes, so if you can look into cross-platform ways to accomplish things, that would widen the audience of people who could get value from the tool. For example, one way to address the rm problem would be to give it a "safe mode" that drops user permissions and runs as nobody. You might also create a blacklist of common, potentially-dangerous unix commands that you don't execute. Both of those are examples of how to address the need without resorting to Linux-only tricks.
    - akavel 8y ago
      How can I do the "drop user permissions" thing? I haven't seen such a keyword mentioned by anyone yet, sounds interesting. Does this not require some kind of syscall, however, thus not really being cross-platform anyway?
      - laumars 8y ago
        ‘nobody’ is an only UNIX thing. It’s just a user and group (like root, $USER, etc) except ‘nobody’ is meant to have no permissions to any folders nor files. In practice it’s only as good as your users ability to avoid the temptation of ‘chmod 777’ (for example) but it seems a good place to start. As an aside, I had the same idea to do a tool like this as well. Except mine would have been built into the $SHELL I’m writing (as that has a heavy focus on being more IDE-like than traditional shells). I scraped the plan for pipe previews precisely because of the dangers we’re discussing. However your tool makes a lot more sense because at least that is manually invoked whereas my original plan was to have that feature automatic and baked into the shell - which is an order of magnitude more dangerous. So I’m glad someone else has ran with this idea
        akavel 8y ago
        :) I've seen somebody mention the idea of applying this to a shell, but only after pressing some special key (e.g. Ctrl-Enter). Would probably make it un-dangerous enough? This seems to be what people want of up too, anyway. As to the "nobody", from what I'm reading, it seems you'd first have to be root, to be able to switch to "nobody"... so this doesn't really seem to be useful to me in this case... :/
- taeric 8y ago
  Eshell can be setup to do this. Is neat for small commands. I turned it off, though.
  - akavel 8y ago
    Can you elaborate a bit more? Which functionality are you referring to? And why did you disable it? Really curious! (Author of up here) I get it that by eshell you mean a shell in Emacs? (That's at least what I'm getting as a first result from google)
    - taeric 8y ago
      Indeed, I meant the emacs shell. https://www.masteringemacs.org/article/complete-guide-mastering-eshell https://www.masteringemacs.org/article/complete-guide-master... has a good overview in the section regarding the Plan 9 Smart Shell. I ultimately just didn't find I used it much. If i am iterating, org-mode with shell sources fits a bit more natural for me. Edit: and to be clear, I meant specifically to have the output below the command with the command still in edit mode. Was not trying to say it is the same as up.

jolmg 8y ago

Yes. Like Enter? People might still like the fact that the cursor doesn't need to move from wherever it's editing the command to run it, and can still continue to edit there.

akavel 8y ago
That's the main direction I currently seem to be coalescing on based on feedback I'm getting. That said, one person on lobste.rs suggested trying to play with Linux capabilities, to make the filesystem immutable... now that would be a seriously fancy trick if it works!
- rcthompson 8y ago
  Consider that the filesystem isn't necessarily the only thing you can mess up by accident. It's much more unlikely, but you could accidentally trigger a bunch of unwanted network requests (e.g. piping into `xargs curl` or something), among other possible things with external effects.
  - akavel 8y ago
    Maybe cutting off network access could be done too? That would reduce some potentially useful features, but it could be a parameter to enable it (hey, we're talking Linux, never enough parameters! ;). That is, assuming that a syscalls/capabilities trick like that is possible at all. (Anyone can say more here? Some kernel hackers? This is HN, ain't it?)
    - rcthompson 8y ago
      Yeah, I think almost any ability to produce an effect other than printing to stdout and stderr should probably be restricted by default, with a flag to enable it. If people adopt this and frequently use specific capabilities, they can add a shell alias to add enable their preferred set of capabilities.
      - rcthompson 8y ago
        Or, one possible alternative would be to always prevent any side-effects other than printing output, and then display a warning saying something like "This command tried to access the network/modify the filesystem/be naughty in some other way. To allow this and re-run the command, press Control+Shift+Enter"
    - em-bee 8y ago
      capabilities should certainly work. for portability, setting a non-existant proxy should also help. most commandline tools should honor shell proxy variables and fail the request. greetings, eMBee.
- jolmg 8y ago
  Too fancy in my opinion. Personally, I prefer KISS solutions. Having an immutable filesystem in up would be unexpected, and I'm sure there's other destructive things that can be done that don't involve the filesystem. As odd as it may seem, I may also want to be able to do filesystem destructive things in up, why shouldn't I be able too? EDIT: As an example, I may exploratively be playing with find options to match different sets of files in a directory hierarchy. Once I see I've matched the files I want, I may want to add a `-delete` to the end of the find conditions I specified to delete them all. That seems like a useful use of up, but making the filesystem immutable would disallow it.
  - akavel 8y ago
    Hm. I think in my ideal vision, you'd then quit up, and somehow magically have the pipeline ready at your fingertips on the shell prompt, so that you can just add the `-delete` there in a "regular" fashion, and press Enter. Not quite sure how this could be done with bash easily. I think the upN.sh file is not a bad approximation however. (Someone on reddit (I think) seemed to mention that zsh seems to have some functionality which could allow building something like that over up.) Interestingly, this seems to me to support the idea that limiting syscalls/capabilities could be a good approach (if at all possible, that is). Potentially maybe also cutting off network access? But even if I add some safety options, even if I make them the defaults, I'll try really hard to keep an option available for people who want to play with it raw, with no safety belts. edit: I totally get you as far as KISS goes; that's why I built and shared the tool as is in the first place ;)
    - jolmg 8y ago
      > Interestingly, this seems to me to support the idea that limiting syscalls/capabilities could be a good approach (if at all possible, that is). Potentially maybe also cutting off network access? Yeah, I still think it's not a good approach. rcthompson gave an example with network access, but that doesn't mean that bad things can only happen with filesystems or network access. You can also kill processes, shutdown the computer, and many more things. I don't think you can guess what mistakes the user might do, or which were really mistakes or were intentional. Also, if you can avoid special permissions or capabilities for your core functionality, then that's better. People should be conservative in giving special permissions to programs, and I can't think it makes sense to give "up" the ability to make a filesystem immutable even if it's in an isolated namespace.
- twanvl 8y ago
  Another solution is to only allow commands from a whitelist to be run. If the user types something not on the list you say "rm is not on the list of allowed commands, press enter to run and add it".
  - ElijahLynn 8y ago
    ++ And rather a blacklist, since a blacklist would be much smaller than a potentially an infinite amount of programs to manage for a whitelist.
    - chrisshroba 8y ago
      I think the whitelist makes more sense, since you can never say for sure that you've added every potentially harmful program to the blacklist. If it's easy to add things to the whitelist, as the parent comment proposed, I think the idea would work very well, especially considering most people have a relatively small number of commands they pipe other commands into. Btw, if you want to see all the commands you've ever piped data into, sorted by frequency, you can use this command (which I put together using `up`!) history | grep -o '| \w\w*' | sort | uniq -c | sort -n | nl
- deelowe 8y ago
  I think from a design perspective, it's better to build your sanity checks and safeguards into the UI instead of some fancy abstraction that's running in the background. Throw in some options to make it user configurable and you have a pretty slick solution that behaves in a way that'll feel familiar to gurus. I think requiring a keypress by default to run any commands and maybe even throwing up a warning for a list of known dangerous ones (perhaps user configurable) would work well. A really fancy solution would be to do this stuff via some sort of linting.
  - akavel 8y ago
    I don't see how linting could work, I'm afraid. There are obscure options in zip, git, grep, and what else, not to mention full-blown interpreters like awk and perl, that can edit files via a myriad of syntaxes, and then there's the halting problem. Similarly with whitelist/blacklist, I'm not currently really convinced it could help much; take `xargs rm`, vs. some hyphotetical `foobar -rm` command + params (where maybe -r = recursive, -m = monitor?) Keypress, on the other hand, sounds to me like it could be a reasonable default compromise... still with an option to switch to "fully accelerated, fast mowing" mode at any point, if one so desires...
    - deelowe 8y ago
      Agreed. As another commenter said KISS.
    - jolmg 8y ago
      > `foobar -rm` command + params (where maybe -r = recursive, -m = monitor?) An example of such a foobar with those exact options would be `inotifywait`. :)
- Spakman 8y ago
  Another thing you might want to investigate (for Linux, not sure about cross-platform alternatives) is OverlayFS[1], which allows you to create union filesystems where some layers are read-only and the top layer is read-write. This can allow destructive operations like rm without actually removing anything from the lower (read-only) layers, using whiteouts. I'd have replied to this on lobste.rs but don't have an account. I'm not a heavy poster but I'd love an invite if anyone has one going :) 1 - https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt https://www.kernel.org/doc/Documentation/filesystems/overlay...
  - akavel 8y ago
    I need your email for an invitation; if you can send me your address via my own email (see my profile), or via keybase, I'll send you an invite. I don't see your email advertised in your profile here on HN, so my hands are tied for now :)
    - Spakman 8y ago
      Thank you very much! I hadn't realised my email address isn't visible here. Maybe I need to move it into the "about" section. In case you aren't aware, your email address isn't visible in your profile either. I've emailed the GMail address that I found on your website :)
- curun1r 8y ago
  Can I say that I hope you don't make this tool too Linux-specific. It seems to run pretty reasonably on Mac and would probably also work on BSD variants. Since I use all three (Mac, BSD, Linux) on a semi-regular basis, I'm pretty unlikely to integrate a tool into my Unix workflows if it doesn't work reliably on most unixes, so if you can look into cross-platform ways to accomplish things, that would widen the audience of people who could get value from the tool. For example, one way to address the rm problem would be to give it a "safe mode" that drops user permissions and runs as nobody. You might also create a blacklist of common, potentially-dangerous unix commands that you don't execute. Both of those are examples of how to address the need without resorting to Linux-only tricks.
  - akavel 8y ago
    How can I do the "drop user permissions" thing? I haven't seen such a keyword mentioned by anyone yet, sounds interesting. Does this not require some kind of syscall, however, thus not really being cross-platform anyway?
    - laumars 8y ago
      ‘nobody’ is an only UNIX thing. It’s just a user and group (like root, $USER, etc) except ‘nobody’ is meant to have no permissions to any folders nor files. In practice it’s only as good as your users ability to avoid the temptation of ‘chmod 777’ (for example) but it seems a good place to start. As an aside, I had the same idea to do a tool like this as well. Except mine would have been built into the $SHELL I’m writing (as that has a heavy focus on being more IDE-like than traditional shells). I scraped the plan for pipe previews precisely because of the dangers we’re discussing. However your tool makes a lot more sense because at least that is manually invoked whereas my original plan was to have that feature automatic and baked into the shell - which is an order of magnitude more dangerous. So I’m glad someone else has ran with this idea
      - akavel 8y ago
        :) I've seen somebody mention the idea of applying this to a shell, but only after pressing some special key (e.g. Ctrl-Enter). Would probably make it un-dangerous enough? This seems to be what people want of up too, anyway. As to the "nobody", from what I'm reading, it seems you'd first have to be root, to be able to switch to "nobody"... so this doesn't really seem to be useful to me in this case... :/
taeric 8y ago
Eshell can be setup to do this. Is neat for small commands. I turned it off, though.
- akavel 8y ago
  Can you elaborate a bit more? Which functionality are you referring to? And why did you disable it? Really curious! (Author of up here) I get it that by eshell you mean a shell in Emacs? (That's at least what I'm getting as a first result from google)
  - taeric 8y ago
    Indeed, I meant the emacs shell. https://www.masteringemacs.org/article/complete-guide-mastering-eshell https://www.masteringemacs.org/article/complete-guide-master... has a good overview in the section regarding the Plan 9 Smart Shell. I ultimately just didn't find I used it much. If i am iterating, org-mode with shell sources fits a bit more natural for me. Edit: and to be clear, I meant specifically to have the output below the command with the command still in edit mode. Was not trying to say it is the same as up.