5 ms·

Sd: My Script Directory

128 points by shadytrees 6y ago

ggm 6y ago
Most instances of the organising principle I find here are amazing. I think "I would never have done it that way" followed almost immediately by "but damn: its good" for comparison: I have ~/bin and I don't distinguish between compiled and scripted personal commands.
- calvinmorrison 6y ago
  - having it in PATH is important. - most of us are programmers, spend a few minutes writing scripts! c@host:~$ ls ~/bin | wc- l 97
- garrepi 6y ago
  +1. Really love the organization of this
- dheera 6y ago
  I just have ~/bin/ and ~/bin/scripts/ and both are in my $PATH so I get autocompletion automatically. I usually don't create "--help" options for personal scripts. Most of my personal scripts are mostly designed to be extremely specific and be so damn obvious how to use them because the --help's of other commands are too complicated. Some examples of my scripts in ~/bin/scripts/: ffmpeg-extract-sound ffmpeg-jpegs-to-video ffmpeg-video-to-jpegs ffmpeg-ogv-to-mp4 convert-png-rgb24-to-grey8 iptables-ignore-rst-packets jabra-stop-changing-volume-goddamnit apt-remove-arm64-shit-that-nvidias-tx2-installer-installed-onto-my-x86-system ipv6-on ipv6-off ros-install-this etc.
  - lars_francke 6y ago
    Asking for a friend: What does your "jabra-stop-changing-volume-goddamnit" script look like?
  - smichel17 6y ago
    help-script(){ "${EDITOR:?vi}" "$(which "$1")"; } I don't actually use that but the point stands -- If the script is simple enough, you don't need a --help; comments in the source work just as well.
    - OMGWTF 6y ago
      zsh has `= expansion`: > If a word begins with an unquoted `=' and the EQUALS option is set, the remainder of the word is taken as the name of a command. If a command exists by that name, the word is replaced by the full pathname of the command. To edit the source of `myscript` I can then use vim =myscript
      - ianthehenry 6y ago
        That's really neat! I'd never heard of that. I think you just saved my future self many many keystrokes.
mootzville 6y ago
I made a ~/bin folder to do something similar, so I don't have to mess with any of the PATH stuff, and can just call it like anything else found in a /bin dir. EDIT: Oh, and autocomplete works with it by default. Just 'mkdir ~/bin' and you're good to go.
- dpflug 6y ago
  They cover that in the article, actually. They had hundreds of files in ~/bin and had forgotten the name of one of them. sd was created to handle that case.
hzhou321 6y ago
While the organization seems nice, the organization itself may have a cognition overhead. I find myself often abandon an organization scheme after I spending time perfecting it. Lately I am mostly using aliases to manage my personal scripts and commands. What the aliases does may change from time to time, but the aliases themselves can stabilize. Those aliases that failed to stabilize or can't fit into top of my head probably are not worth keeping anyway. I delete them every once a while.
grantseltzer 6y ago
If you want an actual project for this I use this one: https://github.com/knqyf263/pet https://github.com/knqyf263/pet
qbasic_forever 6y ago
'personal monorepo' is what I like to consider this kind of organization. I have templated starter projects, direnv configs, etc. all in a hierarchy of folders in my home.
DangitBobby 6y ago
I have my setup so that you can create a shell environment specific to the project associated with the working directory. For example, I can invoke edit-env -ds dump-data.sh and it will open a script for editing which will only be available to run in the project's directory. It would be callable as the bash function `dump-data`. I've got lots of useful little things like that now that I wouldn't necessarily want to be stored in the project's VC but that I find useful.
stevekemp 6y ago
I used to have very full ~/bin, and ~/$(hostname), directories. In the end I pared them back and started bundling things together in one binary. The end result is very similar to this approach, I run "sysbox blah", or "sysbox help", and use integrated subcommands. Very helpful and makes deployment easy by having only a single binary: https://github.com/skx/sysbox https://github.com/skx/sysbox Not bash/shell, but similar and useful idea to experiment with.
siraben 6y ago
One issue I've had with personal shell scripts is that the dependencies are often implicit, or command line arguments change, leading to (sometimes silent and critical) behavior. With Nix you can write shell scripts with a nix-shell shebang[0], specify the dependencies and the rest of the script will run with the dependencies satisfied, for instance, this will execute GNU Hello regardless of whether it is already in the path or not #! /usr/bin/env nix-shell #! nix-shell --pure -i bash -p "hello" hello a more realistic example is[1] which generates[2], and if necessary, the Nixpkgs revision can be pinned to fix the dependencies in time. This approach also extends across languages and some of my scripts are written in Haskell which is interpreted! The upshot of all this is that you can write reproducible shell scripts as if you had every package in Nixpkgs available to you, and share it easily with others. [0] https://nix.dev/tutorials/ad-hoc-developer-environments.html#reproducible-executables https://nix.dev/tutorials/ad-hoc-developer-environments.html... [1] https://edef.eu/~qyliss/nix/lib/gen.sh https://edef.eu/~qyliss/nix/lib/gen.sh [2] https://edef.eu/~qyliss/nix/lib/ https://edef.eu/~qyliss/nix/lib/
- ianthehenry 6y ago
  I learned about this very recently[1] and it seems like an amazing use case that I wish got more play in the marketing of Nix. I have a lot of dumb "productivity" scripts that I wrote at work (Linux) that I can't run at home (macOS) because of differences in the behavior of `find` or `date` whatever. I plan on porting them to use `nixpkgs.findutils` etc. and never having to worry about this again. [1]: https://ianthehenry.com/posts/how-to-learn-nix/command-reference/ https://ianthehenry.com/posts/how-to-learn-nix/command-refer...
  - rchoudhury 6y ago
    Oh my goodness, your blog series looks incredible. I'm in sort of the same boat as you--I've been barely treading water with NixOS for the last several years and have recently resolved to learn how it works well enough to get all the benefits. Thanks for putting this together; looking forward to diving into it!
  - siraben 6y ago
legacynl 6y ago
This is really similar to something I've recently created: https://github.com/simonmeulenbeek/Eezy https://github.com/simonmeulenbeek/Eezy . Although in my project's case it's scoped to the specific PWD you're in. I really like using the folder structure to have 'subcommands' (i.e. 'sd blog publish' ). Very neat!

128 points by shadytrees 6y ago

ggm 6y ago

Most instances of the organising principle I find here are amazing. I think "I would never have done it that way" followed almost immediately by "but damn: its good" for comparison: I have ~/bin and I don't distinguish between compiled and scripted personal commands.

calvinmorrison 6y ago
- having it in PATH is important. - most of us are programmers, spend a few minutes writing scripts! c@host:~$ ls ~/bin | wc- l 97
garrepi 6y ago
+1. Really love the organization of this
dheera 6y ago
I just have ~/bin/ and ~/bin/scripts/ and both are in my $PATH so I get autocompletion automatically. I usually don't create "--help" options for personal scripts. Most of my personal scripts are mostly designed to be extremely specific and be so damn obvious how to use them because the --help's of other commands are too complicated. Some examples of my scripts in ~/bin/scripts/: ffmpeg-extract-sound ffmpeg-jpegs-to-video ffmpeg-video-to-jpegs ffmpeg-ogv-to-mp4 convert-png-rgb24-to-grey8 iptables-ignore-rst-packets jabra-stop-changing-volume-goddamnit apt-remove-arm64-shit-that-nvidias-tx2-installer-installed-onto-my-x86-system ipv6-on ipv6-off ros-install-this etc.
- lars_francke 6y ago
  Asking for a friend: What does your "jabra-stop-changing-volume-goddamnit" script look like?
- smichel17 6y ago
  help-script(){ "${EDITOR:?vi}" "$(which "$1")"; } I don't actually use that but the point stands -- If the script is simple enough, you don't need a --help; comments in the source work just as well.
  - OMGWTF 6y ago
    zsh has `= expansion`: > If a word begins with an unquoted `=' and the EQUALS option is set, the remainder of the word is taken as the name of a command. If a command exists by that name, the word is replaced by the full pathname of the command. To edit the source of `myscript` I can then use vim =myscript
    - ianthehenry 6y ago
      That's really neat! I'd never heard of that. I think you just saved my future self many many keystrokes.

mootzville 6y ago

I made a ~/bin folder to do something similar, so I don't have to mess with any of the PATH stuff, and can just call it like anything else found in a /bin dir. EDIT: Oh, and autocomplete works with it by default. Just 'mkdir ~/bin' and you're good to go.

dpflug 6y ago
They cover that in the article, actually. They had hundreds of files in ~/bin and had forgotten the name of one of them. sd was created to handle that case.

hzhou321 6y ago

While the organization seems nice, the organization itself may have a cognition overhead. I find myself often abandon an organization scheme after I spending time perfecting it. Lately I am mostly using aliases to manage my personal scripts and commands. What the aliases does may change from time to time, but the aliases themselves can stabilize. Those aliases that failed to stabilize or can't fit into top of my head probably are not worth keeping anyway. I delete them every once a while.

grantseltzer 6y ago

If you want an actual project for this I use this one: https://github.com/knqyf263/pet https://github.com/knqyf263/pet

qbasic_forever 6y ago

'personal monorepo' is what I like to consider this kind of organization. I have templated starter projects, direnv configs, etc. all in a hierarchy of folders in my home.

DangitBobby 6y ago

I have my setup so that you can create a shell environment specific to the project associated with the working directory. For example, I can invoke edit-env -ds dump-data.sh and it will open a script for editing which will only be available to run in the project's directory. It would be callable as the bash function `dump-data`. I've got lots of useful little things like that now that I wouldn't necessarily want to be stored in the project's VC but that I find useful.

stevekemp 6y ago

I used to have very full ~/bin, and ~/$(hostname), directories. In the end I pared them back and started bundling things together in one binary. The end result is very similar to this approach, I run "sysbox blah", or "sysbox help", and use integrated subcommands. Very helpful and makes deployment easy by having only a single binary: https://github.com/skx/sysbox https://github.com/skx/sysbox Not bash/shell, but similar and useful idea to experiment with.

siraben 6y ago

One issue I've had with personal shell scripts is that the dependencies are often implicit, or command line arguments change, leading to (sometimes silent and critical) behavior. With Nix you can write shell scripts with a nix-shell shebang[0], specify the dependencies and the rest of the script will run with the dependencies satisfied, for instance, this will execute GNU Hello regardless of whether it is already in the path or not #! /usr/bin/env nix-shell #! nix-shell --pure -i bash -p "hello" hello a more realistic example is[1] which generates[2], and if necessary, the Nixpkgs revision can be pinned to fix the dependencies in time. This approach also extends across languages and some of my scripts are written in Haskell which is interpreted! The upshot of all this is that you can write reproducible shell scripts as if you had every package in Nixpkgs available to you, and share it easily with others. [0] https://nix.dev/tutorials/ad-hoc-developer-environments.html#reproducible-executables https://nix.dev/tutorials/ad-hoc-developer-environments.html... [1] https://edef.eu/~qyliss/nix/lib/gen.sh https://edef.eu/~qyliss/nix/lib/gen.sh [2] https://edef.eu/~qyliss/nix/lib/ https://edef.eu/~qyliss/nix/lib/

ianthehenry 6y ago
I learned about this very recently[1] and it seems like an amazing use case that I wish got more play in the marketing of Nix. I have a lot of dumb "productivity" scripts that I wrote at work (Linux) that I can't run at home (macOS) because of differences in the behavior of `find` or `date` whatever. I plan on porting them to use `nixpkgs.findutils` etc. and never having to worry about this again. [1]: https://ianthehenry.com/posts/how-to-learn-nix/command-reference/ https://ianthehenry.com/posts/how-to-learn-nix/command-refer...
- rchoudhury 6y ago
  Oh my goodness, your blog series looks incredible. I'm in sort of the same boat as you--I've been barely treading water with NixOS for the last several years and have recently resolved to learn how it works well enough to get all the benefits. Thanks for putting this together; looking forward to diving into it!
- siraben 6y ago

legacynl 6y ago

This is really similar to something I've recently created: https://github.com/simonmeulenbeek/Eezy https://github.com/simonmeulenbeek/Eezy . Although in my project's case it's scoped to the specific PWD you're in. I really like using the folder structure to have 'subcommands' (i.e. 'sd blog publish' ). Very neat!