7 ms·

Writing simple tab-completions for Bash and Zsh

271 points by lihaoyi 1y ago

lihaoyi 1y ago
I wrote this, hope everyone finds it as interesting reading this as I did figuring this out for the first time!
- oezi 1y ago
  Thanks, for the interesting read.
- tetha 1y ago
  It's a good first dive into zsh completion. The whole thing is quite the large system to wrap ones head around it and I'm still somewhat struggling. But at work, I've been slowly adding auto completion to our ansible wrapper scripts, like explanations which playbooks to use when, smart `-l` completion based off a possibly selected playbook (so, if the playbook is postgres.yml, it doesn't suggest mariadb groups), tag autocompletion (with a few, admittedly, hardcoded explanations how these tags should be used) and such. It's somewhat of a friday-afternoon struggle project, but it's making the big ansible project pretty approachable to use.
  - imcritic 1y ago
    Could you share how you do it? Ansible playbooks are ran via command `ansible-playbook` command and it surely has its own tab auto completion script.
    - tetha 1y ago
      It kinda does, but I've added a few bells and whistles to it. Mind you, this is zsh code I haven't really cleaned up, so it's kinda messy and most likely buggy in edge cases. This needs to be in a directory in your FPATH. At the core, it uses _arguments to dissect the command line. This both suggests that "-t" exists, is called "tags" and later on sets a state variable to "tags" or "limits" if we're completing these arguments. #compdef corp-ansible-wrapper _arguments '-t[tags]:tags:->tags' '-l[limit]:limit:->limits' '-D[diffmode]' '-C[checkmode]' '::optional arg:_files' detect_playbook case "$state" in tags) ... ;; limits) ... ;; Given this, the limits autocompletion goes one step further. `detect_playbook` mainly goes through $words and looks for a singular argument looking like "*.yml" and sets that as $PLAYBOOK. Then, based on "$PLAYBOOK", it selects a filter-expression for the groups. This ensures that a `./wrapper mariadb.yml -l<TAB>` only sees mariadb-groups, and not postgres-groups. All of that is shoved into `_values`, and then the usual zsh completion works, so with something like `./wrapper mariadb.yml -l prj4<TAB>`, zsh tries to filter the values based on the word, so this finds stuff like `prj4`, `prj49`, `dc2_prj45`, and so on, but not `prj5`. detect_playbook # should be an array. case $PLAYBOOK in postgres.yml) FILTER="pg_\|postgres_\|pgbackrest_" ;; mariadb.yml) FILTER="db_\|mariadb_" ;; # quite a few more *) FILTER="" ;; esac # probably overly complex ANSIBLE_GROUPS_RAW=$( cat $ANSIBLE_INVENTORY/groups | grep -E "$FILTER" | sort | uniq | tr '\n' ',') IFS=',' read -r -A ANSIBLE_GROUPS <<< "$ANSIBLE_GROUPS_RAW" _values -s: 'groups' "${ANSIBLE_GROUPS[@]}" For the tags I'm working on a similar thing, but this contains enough ugly shell-script already. However, the key parts there are: - You can run `ansible-playbook "$WORD" --list-tags` to get all tags a playbook references in the current inventory. - One can give `_values` descriptions. If `_values` sees 'foo[bar]' as an option, it will show the user something like: "foo: bar" and only auto-completes to foo. - This means, we can give standard or well-established tags short descriptions in a directory or an array or whatever, and instead of offering just "postgres_client_tls_certs" as possible auto-completions for `-t`, we can give the user a prompt like "postgres_client_tls_certs: Ensures the postgres cluster has valid and up-to-date TLS certificates for mutual TLS with applications". It took a bit of time to understand all of this. But now documenting a tag in a place that people actually look at is very easy and straightforward.
      - imcritic 1y ago
        Oh, I thought you didn't hardcore those as if conditions, but rather parsed inventory files to get suggestions for -t and parsed tasks in playbook's roles to get suggestions for -l. Still cool though, thanks for sharing!
- bbkane 1y ago
  Thanks for sharing! I hope to incorporate your bash completion ideas into my CLIs (I've already got zsh completions). Instead of sourcing the zsh completion script on every startup, you can install it into somewhere on $fpath and zsh will "compile" and cache the completions. This can really speed up shell startup time, but of course is harder to set up. Users have to understand $fpath to put the completions there. I distribute my CLIs via Homebrew which can install completions automatically.
oezi 1y ago
Isn't there a standard flag which programs can implement to avoid writing this bash script? Ideally this could all be part of a library such as argparse for typical cases, right?
- vcdimension 1y ago
  In zsh you can use the _gnu_generic function for simple completion of commands with a --help flag. Just put a line like this somewhere in your startup file: compdef _gnu_generic <CMD>
  - cb321 1y ago
    _gnu_generic is fantastic. I use it all the time. If your CLI toolkit emits colorized help, but skips said colorization if NO_COLOR[1] is set then you can prefix _gnu_generic with NO_COLOR=1 as in https://github.com/c-blake/cligen/wiki/Zsh-completion-for-cligen-commands-(with-colored-help-output) https://github.com/c-blake/cligen/wiki/Zsh-completion-for-cl... for the cligen Nim CLI toolkit. A similar thing in Bash is `complete -F _longopt YourCmd`, but these will not work with "multi-commands" that have "sub-commands" as the article of this thread covers. Truth is, as non-standard as GNU long opts already are, how subcommands work is even more non-standard (are there even global options? Or between each subcommand? Is it how Python does it? Or Go? Or some one specific library or ..?) [^1]: https://no-color.org/ https://no-color.org/
- mnahkies 1y ago
  I've wondered this as well - it would sure be nice if there was a standard --completion or something that common argument parsing libraries could automatically implement for us (much like they often implement automatic help text)
- duckerude 1y ago
  Rust has the clap_complete package for its most popular arg parsing library: https://crates.io/crates/clap_complete https://crates.io/crates/clap_complete ripgrep exposes its (bespoke) shell completion and man page generation through a --generate option: rg --generate=man, rg --generate=complete-bash, etcetera. In xh (clap-based) we provide the same but AFAIK we're the only one to copy that interface. Symfony (for PHP) provides some kind of runtime completion generation but I don't know the details.
bravesoul2 1y ago
Do many people do bash on osx or zsh on Linux, and would this make much of a difference?
- homebrewer 1y ago
  I don't know about "use" — luckily, there's no opt-out telemetry — but enough of "enthusiast distribution" users who have also opted in (very biased sample) have explicitly installed zsh (not necessarily run it) https://pkgstats.archlinux.de/compare/packages#packages=bash,fish,zsh https://pkgstats.archlinux.de/compare/packages#packages=bash... OTOH, it's only 4-7% on Debian (also opt-in): https://qa.debian.org/popcon.php?package=zsh https://qa.debian.org/popcon.php?package=zsh
  - bravesoul2 1y ago
    I use zsh at work and bash at home. I am such an unsophisticated user that I haven't noticed a real difference! Other than I can install ohmyzsh on zsh.
    - bbkane 1y ago
      I repeat the different variations of the same command so often that I get a large quality of life improvement by making that easier- that's why I take the trouble to install zsh.
  - yonatan8070 1y ago
    Why does this type of statistic require an opt-in solution? Can't the Arch mirrors just tally requests for each package without identifying information like IP adddress?
    - lpln3452 1y ago
      Maybe they can’t ask for info on every mirror.
- mdaniel 1y ago
vcdimension 1y ago
Here's another tutorial for creating zsh completers using the built-in functions: https://github.com/vapniks/zsh-completions/blob/master/zsh-completions-howto.org https://github.com/vapniks/zsh-completions/blob/master/zsh-c...
wiseowise 1y ago
Shell syntax is the exact reason why we've needed LLMs in the first place.
- camdroidw 1y ago
  You mean Unix shell syntax. Powershell has got this absolutely right ,and only this (which is probably still a 50% of what a shell is)
  - pastage 1y ago
    Can you expand on why it is good I have never read any good evangelisation for it. I can not stand PowerShell the syntax is backward, least important information first. It is also very slow for me but I have understood that it is a pebkac issue. So I am open to be corrected.
    - vips7L 1y ago
      I think they’re talking about powershells programming syntax which is much more sane than any Unix shell. There’s way less footguns and everything is typed. You’re not dealing with raw strings for everything. If you don’t like the Verb-Noun nonsense I’d encourage you to look at the default aliases as they make everything a lot less verbose. For example Where-Object is just “where” or getchild-item is default aliased to ls and gci. Id encourage you to look at NuShell as well since it is mostly the same philosophy as PowerShell.
    - camdroidw 1y ago
      On phone: Bad for your reasons (slow etc), good for siblings (objects). Also, understandable flags that auto-shorten. Eg: Descriptive-Command-Name -DescriptiveOption 32 Can be typed as: d-c-n <tab> -des 32 --- Interactively it's still far behind zsh sadly due to startup time but when I'm doing something thoughtful I prefer it. If you're not entering directories and doing small stuff, use pwsh. And for all shell scripting
    - fainpul 1y ago
      By "got this right", the parent poster probably meant tab completion. You get that for free when writing a script or function, as well as basic syntax documentation. Both can be improved gradually by adding hints / comments in your script. If you are bothered by the verbose `Verb-Noun` structure of commands, those are mostly used in scripts (and you never type them fully, you use tab-completion of course). On the commandline you use aliases for many of them. For example listing files and directories created in the last 10 days: ls | ? CreationTime -ge (Get-Date).AddDays(-10) | ft name, size All the long words are tab completed (CreationTime, Get-Date, AddDays). The fully verbose syntax you might write in a script looks a bit uglier: Get-ChildItem | Where-Object CreationTime -ge (Get-Date).AddDays(-10) | Format-Table Name, Size Also PowerShell is NOT case-sensitive, even though many people seem to think so.
homebrewer 1y ago
With fish, if the program you're interested in hasn't betrayed the decades-old tradition of shipping man pages, it's often as simple as running `fish_update_completions`. It parses all man pages on your system and generates completion files for you. By default, they go into ~/.cache/fish/generated_completions/* If the man page was written poorly/is missing, you can always write your own completion (and hopefully send it upstream). fish uses such a simple format that I don't think there's any need for tutorials save the official doc: https://fishshell.com/docs/current/completions.html https://fishshell.com/docs/current/completions.html For example, here's an excerpt from curl complete --command curl --short-option 'L' --long-option 'location' --description 'Follow redirects' complete --command curl --short-option 'O' --long-option 'remote-name' --description 'Write output to file named as remote file'
- btreecat 1y ago
  When I screen share, people don't realize I'm not using zsh and dozen plugins. It's just fish and it's beautiful out of the box.
- kekebo 1y ago
  Thank you for the comment. https://github.com/umlx5h/zsh-manpage-completion-generator https://github.com/umlx5h/zsh-manpage-completion-generator appears to adapt this to ZSH. Have yet to try through
- cb321 1y ago
  For those programs that have betrayed shipping man pages, instead say relying only on a --help system, do you happen to know if the fish shell has an analogue to Zsh `_gnu_generic` and Bash `complete -F _longopt`? If not, do you have any insight into why not/what it would take to make that happen?
  - nikita2206 1y ago
    The OP mentions them in the last part of their comment, there is `complete …` commands for registering completions
camdroidw 1y ago
Why doesn't someone (not me) just build a basic DSL and a transpiler that does this?
- cb321 1y ago
  The answer to your question is that command-lines have a much larger diversity of syntax (even to get help!) than most people realize. Folks have their 30..60 commands they run frequently and don't run into many or conveniently forget/neglect older ones like `gcc` or `tar` or `dd`. Many people (not saying you specifically) do not even realize that double-dash long options are a GNU extension never standardized or that Python toolkits typically allow --my-opt for --my-option abbreviations, just to name a couple of the dozen variations (space or '=', or ':' or '/' or any of the above or etc., etc.). There are probably hundreds if not thousands of syntax possibilities, but people often act like there is only one. As an example of diversity estimation that you can try at home, a couple of times I have run every single command in my command search PATH with --help </dev/null >/tmp/help.$c 2>&1 . Caution - be careful if you do this! Have backups/checksums of everything important and run as an unprivileged user. I always have to kill off several processes that just hang doing something or otherwise manually intervene. Anyway, this alone suggests data collection of help text is not a trivial problem. Beyond data collection, many commands did not/do not use CLI toolkits at all. Their commands may have even less regular syntax. Freeform help makes it harder to produce a regular help syntax to convert into the interpreter needed by a completion system. That said, as elsethread commented for some toolkits the Zsh _gnu_generic works great! It essentially IS the "automagic" system you might want, just for a highly restricted circumstance. Any CLI toolkit itself does have the data, by necessity. So, if the CLI framework supports the 2 or 3 common shells there is no need for a translator exactly. You just need a code generator. There is a stab at an auto-generation framework from said data for the Nim CLI toolkit, cligen, over at: https://github.com/c-blake/cligen/blob/master/util/complgen.nim https://github.com/c-blake/cligen/blob/master/util/complgen.... but it only works for Zsh right now. Anyway, I don't think perfect should be the enemy of the good or anything like that, but you seemed to ask an earnest "why" question and these are some of the complexities.
  - camdroidw 1y ago
derriz 1y ago
I feel that the ergonomics of bash completion took a hit as the configurations got “smarter” and “helpfully” started blocking file or directory name completion if it thinks it wouldn’t be appropriate to have a file name at the current cursor position. Instead of blocking, the default should always be to fall back to filename completion. Sometimes I’m close to disabling/uninstalling all completion scripts out of irritation as decades of muscle memory are frustrated by this behavior. It’s like that bad/annoying UX with text fields where the UI is constantly fighting against you in order prevent you from producing “illegal” intermediate input - e.g. let me paste the clipboard here goddammit - I know what I’m doing - I’ll correct it.
- cryptoz 1y ago
  I have come to absolutely despise web form inputs with front end email validators that are broken. Input field hints to type your email, so you start typing. As soon as you type the first letter it goes red and says “error!!! Invalid email!” Unbelievably frustrating.
  - pastage 1y ago
    As a guy who is vicously guarding our company email valdation I can tell you that it is a rite of passage for new frontend hires to mess that up.
- compressedgas 1y ago
  As it works as desired after running: complete -r; there is something broken about the bash-completion script.
- IshKebab 1y ago
  I agree that is annoying. It's waaay less confusing to complete a filename and then get an error from the actual program than it is for just ...nothing to happen so you get confused and have to `ls` to check if the file actually exists and it does and so you think tab completion is broken for some reason and you copy & paste the filename and then finally you get the error that explains what is going on. It should at least print a message like "file foo.exe exists but it isn't executable".
vbezhenar 1y ago
Here's zsh snippet I've came up with for my own simple functions. I'm using it as a base for other completions. In this example, function `set-java-home zulu-21` sets JAVA_HOME to `~/apps/java/zulu-21`. Here's `_set-java-home`: #compdef set-java-home local -a versions=(~/apps/java/*(:t)) _describe 'version' versions So basically almost a one-liner (but couldn't do it really one-liner, unfortunately).
medv 1y ago
JSON fields autocomplete right in bash/zsh: https://fx.wtf/install#autocomplete https://fx.wtf/install#autocomplete
- imcritic 1y ago
  Thanks for linking this! This is a lightweight solution, compared to ijq (interactive jq), but it still may come in handy. https://github.com/gpanders/ijq https://github.com/gpanders/ijq
xenophonf 1y ago
I wish tcsh would get more love.
- esafak 1y ago
  Why, it's a dinosaur? Have you tried nushell, murex, oil shell or xonsh?
  - xenophonf 1y ago
    I don't know why you're being downvoted. You're correct, and as the other commenter mentioned, csh programming is evil in the eyes of the LORD. Starting last year, I've been maintaining a bash profile that implements (mostly) the same setup as my tcsh configuration. I suppose I ought to make the final leap into something more modern. It would be nice not to jump through hoops to redirect stderr somewhere different than stdout or [insert your favorite of the myriad examples of csh/tcsh weirdness here]! But I feel irrationally sad about it. I cut my teeth on Unix systems where csh was the default shell—namely, NeXTstep and sunOS 4—and it feels a little like saying the final goodbyes to a dead friend. Silly of me, I realize, but there it is.
- chasil 1y ago
  There is a famous paper on the perils of scripting in the csh. It is unfortunate that Bill Joy was not able to write a formal grammar or parser for his language. It was certainly a missed opportunity, and tcsh cannot fix the design. That being said, csh advocates definitely influenced everything in the Bourne/POSIX family.
  - xenophonf 1y ago
    It's a classic Usenet post that deserves linking at every opportunity! https://harmful.cat-v.org/software/csh https://harmful.cat-v.org/software/csh
harimurti 1y ago
[dead]
sebtron 1y ago
Basic completion in ksh is as easy as defining an array. From https://man.openbsd.org/ksh https://man.openbsd.org/ksh : Custom completions may be configured by creating an array named ‘complete_command’, optionally suffixed with an argument number to complete only for a single argument. So defining an array named ‘complete_kill’ provides possible completions for any argument to the kill(1) command, but ‘complete_kill_1’ only completes the first argument. For example, the following command makes ksh offer a selection of signal names for the first argument to kill(1): set -A complete_kill_1 -- -9 -HUP -INFO -KILL -TERM
- chasil 1y ago
  Is this in the Korn & Bolsy ksh88 book? Or is this ksh93 syntax that oksh back ported?
  - sebtron 1y ago
    I don't know actually. Are there currently used version of ksh that are not derived from ksh93?
    - chasil 1y ago
      Very much so - pdksh was a bsd-licensed clone that is the ancestor of mksh (Android's system shell) and OpenBSD's oksh, which you previously mentioned. David Korn thanked the pdksh authors and maintainers for making it available in the years when true ksh was closed (ksh88) or open but licensed with awkward terms (ksh93 for several years). David Korn interview (I asked one of the questions): https://m.slashdot.org/story/16351 https://m.slashdot.org/story/16351 "First of all pdksh is a ksh88 clone; and I might add a better clone than the MKS Korn Shell... "I don't know the pdksh development team but I would like to thank them for the service they have done in making a version of ksh available while ksh was proprietary. I have noticed remarkable improvements in pdksh in its ability to mimic ksh88 functionality. I don't know what plans the pdksh development team has now that ksh93 is available in open source form, but I certainly would help them try to maintain compatibility if they do continue pdksh distribution. Otherwise, I would hope that they would pick up the ksh93 source and help support and enhance it."
gosub100 1y ago
I did something similar to this for tab-completing server names for use with ssh. I went a step further and allowed pattern matching based on the server being qa/prod and on locale. so for instance you could type `ssh co prod <tab>` and it would tab-complete / suggest any servers that were production and located in the Denver datacenter (co is the state abbrev for Colorado, for non-US readers). Unfortunately my work doesn't allow me to share code, but essentially I remapped ssh to a bash script that maintains an environment variable containing the args (you must do this because each <tab> press is an independent invocation of the script. Then you run into persistence problems, so I added a call to compute elapsed seconds so that it flushes the state variable after a 10s timeout). The bash script then forwards the args to a python script that reads a JSON file and figures out which params (such as 'co' or 'qa') map to which hostnames. It also matches against partial hostnames, so when you see this after tab qa-server-co1 qa-server-co2 pr-server-co3 you only need to add '3' to the list of args to narrow it down to 1 match, then hit <enter> to ssh to that host.
paradox460 1y ago
I've started using jdx's usage[1] for my clis. It integrates neatly into clap, and can be used stand alone in scripts. It can generate completions, argparse, manpages, and more I'm still on the fence if replacing the argparse blocks in my fish scripts is worth the hassle, but against things like old school optparse, it's far better [1]: https://usage.jdx.dev/ https://usage.jdx.dev/
- FujiApple 1y ago
  I’ve recently been building a similar tool [1] which defines a specification for CLIs, though the goals are slightly different to the tool you mention I think. I just added support for fish as it happens. [1] https://github.com/fujiapple852/claptrap https://github.com/fujiapple852/claptrap
  - paradox460 1y ago
    Nice. I'll have to add that to my tool belt, although I think usage will still sit front and center haha
vismit2000 1y ago
Since auto text completion is the primary task of an LLM, I have seen GitHub copilot do that very well on both bash and zsh within vscode terminal!
VagabundoP 1y ago
zsh standard tab completion drives me batty. But I come from bash where 99% of the time I just want filename completion.

271 points by lihaoyi 1y ago

lihaoyi 1y ago

I wrote this, hope everyone finds it as interesting reading this as I did figuring this out for the first time!

oezi 1y ago
Thanks, for the interesting read.
tetha 1y ago
It's a good first dive into zsh completion. The whole thing is quite the large system to wrap ones head around it and I'm still somewhat struggling. But at work, I've been slowly adding auto completion to our ansible wrapper scripts, like explanations which playbooks to use when, smart `-l` completion based off a possibly selected playbook (so, if the playbook is postgres.yml, it doesn't suggest mariadb groups), tag autocompletion (with a few, admittedly, hardcoded explanations how these tags should be used) and such. It's somewhat of a friday-afternoon struggle project, but it's making the big ansible project pretty approachable to use.
- imcritic 1y ago
  Could you share how you do it? Ansible playbooks are ran via command `ansible-playbook` command and it surely has its own tab auto completion script.
  - tetha 1y ago
    It kinda does, but I've added a few bells and whistles to it. Mind you, this is zsh code I haven't really cleaned up, so it's kinda messy and most likely buggy in edge cases. This needs to be in a directory in your FPATH. At the core, it uses _arguments to dissect the command line. This both suggests that "-t" exists, is called "tags" and later on sets a state variable to "tags" or "limits" if we're completing these arguments. #compdef corp-ansible-wrapper _arguments '-t[tags]:tags:->tags' '-l[limit]:limit:->limits' '-D[diffmode]' '-C[checkmode]' '::optional arg:_files' detect_playbook case "$state" in tags) ... ;; limits) ... ;; Given this, the limits autocompletion goes one step further. `detect_playbook` mainly goes through $words and looks for a singular argument looking like "*.yml" and sets that as $PLAYBOOK. Then, based on "$PLAYBOOK", it selects a filter-expression for the groups. This ensures that a `./wrapper mariadb.yml -l<TAB>` only sees mariadb-groups, and not postgres-groups. All of that is shoved into `_values`, and then the usual zsh completion works, so with something like `./wrapper mariadb.yml -l prj4<TAB>`, zsh tries to filter the values based on the word, so this finds stuff like `prj4`, `prj49`, `dc2_prj45`, and so on, but not `prj5`. detect_playbook # should be an array. case $PLAYBOOK in postgres.yml) FILTER="pg_\|postgres_\|pgbackrest_" ;; mariadb.yml) FILTER="db_\|mariadb_" ;; # quite a few more *) FILTER="" ;; esac # probably overly complex ANSIBLE_GROUPS_RAW=$( cat $ANSIBLE_INVENTORY/groups | grep -E "$FILTER" | sort | uniq | tr '\n' ',') IFS=',' read -r -A ANSIBLE_GROUPS <<< "$ANSIBLE_GROUPS_RAW" _values -s: 'groups' "${ANSIBLE_GROUPS[@]}" For the tags I'm working on a similar thing, but this contains enough ugly shell-script already. However, the key parts there are: - You can run `ansible-playbook "$WORD" --list-tags` to get all tags a playbook references in the current inventory. - One can give `_values` descriptions. If `_values` sees 'foo[bar]' as an option, it will show the user something like: "foo: bar" and only auto-completes to foo. - This means, we can give standard or well-established tags short descriptions in a directory or an array or whatever, and instead of offering just "postgres_client_tls_certs" as possible auto-completions for `-t`, we can give the user a prompt like "postgres_client_tls_certs: Ensures the postgres cluster has valid and up-to-date TLS certificates for mutual TLS with applications". It took a bit of time to understand all of this. But now documenting a tag in a place that people actually look at is very easy and straightforward.
    - imcritic 1y ago
      Oh, I thought you didn't hardcore those as if conditions, but rather parsed inventory files to get suggestions for -t and parsed tasks in playbook's roles to get suggestions for -l. Still cool though, thanks for sharing!
bbkane 1y ago
Thanks for sharing! I hope to incorporate your bash completion ideas into my CLIs (I've already got zsh completions). Instead of sourcing the zsh completion script on every startup, you can install it into somewhere on $fpath and zsh will "compile" and cache the completions. This can really speed up shell startup time, but of course is harder to set up. Users have to understand $fpath to put the completions there. I distribute my CLIs via Homebrew which can install completions automatically.

oezi 1y ago

Isn't there a standard flag which programs can implement to avoid writing this bash script? Ideally this could all be part of a library such as argparse for typical cases, right?

vcdimension 1y ago
In zsh you can use the _gnu_generic function for simple completion of commands with a --help flag. Just put a line like this somewhere in your startup file: compdef _gnu_generic <CMD>
- cb321 1y ago
  _gnu_generic is fantastic. I use it all the time. If your CLI toolkit emits colorized help, but skips said colorization if NO_COLOR[1] is set then you can prefix _gnu_generic with NO_COLOR=1 as in https://github.com/c-blake/cligen/wiki/Zsh-completion-for-cligen-commands-(with-colored-help-output) https://github.com/c-blake/cligen/wiki/Zsh-completion-for-cl... for the cligen Nim CLI toolkit. A similar thing in Bash is `complete -F _longopt YourCmd`, but these will not work with "multi-commands" that have "sub-commands" as the article of this thread covers. Truth is, as non-standard as GNU long opts already are, how subcommands work is even more non-standard (are there even global options? Or between each subcommand? Is it how Python does it? Or Go? Or some one specific library or ..?) [^1]: https://no-color.org/ https://no-color.org/
mnahkies 1y ago
I've wondered this as well - it would sure be nice if there was a standard --completion or something that common argument parsing libraries could automatically implement for us (much like they often implement automatic help text)
duckerude 1y ago
Rust has the clap_complete package for its most popular arg parsing library: https://crates.io/crates/clap_complete https://crates.io/crates/clap_complete ripgrep exposes its (bespoke) shell completion and man page generation through a --generate option: rg --generate=man, rg --generate=complete-bash, etcetera. In xh (clap-based) we provide the same but AFAIK we're the only one to copy that interface. Symfony (for PHP) provides some kind of runtime completion generation but I don't know the details.

bravesoul2 1y ago

Do many people do bash on osx or zsh on Linux, and would this make much of a difference?

homebrewer 1y ago
I don't know about "use" — luckily, there's no opt-out telemetry — but enough of "enthusiast distribution" users who have also opted in (very biased sample) have explicitly installed zsh (not necessarily run it) https://pkgstats.archlinux.de/compare/packages#packages=bash,fish,zsh https://pkgstats.archlinux.de/compare/packages#packages=bash... OTOH, it's only 4-7% on Debian (also opt-in): https://qa.debian.org/popcon.php?package=zsh https://qa.debian.org/popcon.php?package=zsh
- bravesoul2 1y ago
  I use zsh at work and bash at home. I am such an unsophisticated user that I haven't noticed a real difference! Other than I can install ohmyzsh on zsh.
  - bbkane 1y ago
    I repeat the different variations of the same command so often that I get a large quality of life improvement by making that easier- that's why I take the trouble to install zsh.
- yonatan8070 1y ago
  Why does this type of statistic require an opt-in solution? Can't the Arch mirrors just tally requests for each package without identifying information like IP adddress?
  - lpln3452 1y ago
    Maybe they can’t ask for info on every mirror.
mdaniel 1y ago

vcdimension 1y ago

Here's another tutorial for creating zsh completers using the built-in functions: https://github.com/vapniks/zsh-completions/blob/master/zsh-completions-howto.org https://github.com/vapniks/zsh-completions/blob/master/zsh-c...

wiseowise 1y ago

Shell syntax is the exact reason why we've needed LLMs in the first place.

camdroidw 1y ago
You mean Unix shell syntax. Powershell has got this absolutely right ,and only this (which is probably still a 50% of what a shell is)
- pastage 1y ago
  Can you expand on why it is good I have never read any good evangelisation for it. I can not stand PowerShell the syntax is backward, least important information first. It is also very slow for me but I have understood that it is a pebkac issue. So I am open to be corrected.
  - vips7L 1y ago
    I think they’re talking about powershells programming syntax which is much more sane than any Unix shell. There’s way less footguns and everything is typed. You’re not dealing with raw strings for everything. If you don’t like the Verb-Noun nonsense I’d encourage you to look at the default aliases as they make everything a lot less verbose. For example Where-Object is just “where” or getchild-item is default aliased to ls and gci. Id encourage you to look at NuShell as well since it is mostly the same philosophy as PowerShell.
  - camdroidw 1y ago
    On phone: Bad for your reasons (slow etc), good for siblings (objects). Also, understandable flags that auto-shorten. Eg: Descriptive-Command-Name -DescriptiveOption 32 Can be typed as: d-c-n <tab> -des 32 --- Interactively it's still far behind zsh sadly due to startup time but when I'm doing something thoughtful I prefer it. If you're not entering directories and doing small stuff, use pwsh. And for all shell scripting
  - fainpul 1y ago
    By "got this right", the parent poster probably meant tab completion. You get that for free when writing a script or function, as well as basic syntax documentation. Both can be improved gradually by adding hints / comments in your script. If you are bothered by the verbose `Verb-Noun` structure of commands, those are mostly used in scripts (and you never type them fully, you use tab-completion of course). On the commandline you use aliases for many of them. For example listing files and directories created in the last 10 days: ls | ? CreationTime -ge (Get-Date).AddDays(-10) | ft name, size All the long words are tab completed (CreationTime, Get-Date, AddDays). The fully verbose syntax you might write in a script looks a bit uglier: Get-ChildItem | Where-Object CreationTime -ge (Get-Date).AddDays(-10) | Format-Table Name, Size Also PowerShell is NOT case-sensitive, even though many people seem to think so.

homebrewer 1y ago

With fish, if the program you're interested in hasn't betrayed the decades-old tradition of shipping man pages, it's often as simple as running `fish_update_completions`. It parses all man pages on your system and generates completion files for you. By default, they go into ~/.cache/fish/generated_completions/* If the man page was written poorly/is missing, you can always write your own completion (and hopefully send it upstream). fish uses such a simple format that I don't think there's any need for tutorials save the official doc: https://fishshell.com/docs/current/completions.html https://fishshell.com/docs/current/completions.html For example, here's an excerpt from curl complete --command curl --short-option 'L' --long-option 'location' --description 'Follow redirects' complete --command curl --short-option 'O' --long-option 'remote-name' --description 'Write output to file named as remote file'

btreecat 1y ago
When I screen share, people don't realize I'm not using zsh and dozen plugins. It's just fish and it's beautiful out of the box.
kekebo 1y ago
Thank you for the comment. https://github.com/umlx5h/zsh-manpage-completion-generator https://github.com/umlx5h/zsh-manpage-completion-generator appears to adapt this to ZSH. Have yet to try through
cb321 1y ago
For those programs that have betrayed shipping man pages, instead say relying only on a --help system, do you happen to know if the fish shell has an analogue to Zsh `_gnu_generic` and Bash `complete -F _longopt`? If not, do you have any insight into why not/what it would take to make that happen?
- nikita2206 1y ago
  The OP mentions them in the last part of their comment, there is `complete …` commands for registering completions

camdroidw 1y ago

Why doesn't someone (not me) just build a basic DSL and a transpiler that does this?

cb321 1y ago
The answer to your question is that command-lines have a much larger diversity of syntax (even to get help!) than most people realize. Folks have their 30..60 commands they run frequently and don't run into many or conveniently forget/neglect older ones like `gcc` or `tar` or `dd`. Many people (not saying you specifically) do not even realize that double-dash long options are a GNU extension never standardized or that Python toolkits typically allow --my-opt for --my-option abbreviations, just to name a couple of the dozen variations (space or '=', or ':' or '/' or any of the above or etc., etc.). There are probably hundreds if not thousands of syntax possibilities, but people often act like there is only one. As an example of diversity estimation that you can try at home, a couple of times I have run every single command in my command search PATH with --help </dev/null >/tmp/help.$c 2>&1 . Caution - be careful if you do this! Have backups/checksums of everything important and run as an unprivileged user. I always have to kill off several processes that just hang doing something or otherwise manually intervene. Anyway, this alone suggests data collection of help text is not a trivial problem. Beyond data collection, many commands did not/do not use CLI toolkits at all. Their commands may have even less regular syntax. Freeform help makes it harder to produce a regular help syntax to convert into the interpreter needed by a completion system. That said, as elsethread commented for some toolkits the Zsh _gnu_generic works great! It essentially IS the "automagic" system you might want, just for a highly restricted circumstance. Any CLI toolkit itself does have the data, by necessity. So, if the CLI framework supports the 2 or 3 common shells there is no need for a translator exactly. You just need a code generator. There is a stab at an auto-generation framework from said data for the Nim CLI toolkit, cligen, over at: https://github.com/c-blake/cligen/blob/master/util/complgen.nim https://github.com/c-blake/cligen/blob/master/util/complgen.... but it only works for Zsh right now. Anyway, I don't think perfect should be the enemy of the good or anything like that, but you seemed to ask an earnest "why" question and these are some of the complexities.
- camdroidw 1y ago

derriz 1y ago

I feel that the ergonomics of bash completion took a hit as the configurations got “smarter” and “helpfully” started blocking file or directory name completion if it thinks it wouldn’t be appropriate to have a file name at the current cursor position. Instead of blocking, the default should always be to fall back to filename completion. Sometimes I’m close to disabling/uninstalling all completion scripts out of irritation as decades of muscle memory are frustrated by this behavior. It’s like that bad/annoying UX with text fields where the UI is constantly fighting against you in order prevent you from producing “illegal” intermediate input - e.g. let me paste the clipboard here goddammit - I know what I’m doing - I’ll correct it.

cryptoz 1y ago
I have come to absolutely despise web form inputs with front end email validators that are broken. Input field hints to type your email, so you start typing. As soon as you type the first letter it goes red and says “error!!! Invalid email!” Unbelievably frustrating.
- pastage 1y ago
  As a guy who is vicously guarding our company email valdation I can tell you that it is a rite of passage for new frontend hires to mess that up.
compressedgas 1y ago
As it works as desired after running: complete -r; there is something broken about the bash-completion script.
IshKebab 1y ago
I agree that is annoying. It's waaay less confusing to complete a filename and then get an error from the actual program than it is for just ...nothing to happen so you get confused and have to `ls` to check if the file actually exists and it does and so you think tab completion is broken for some reason and you copy & paste the filename and then finally you get the error that explains what is going on. It should at least print a message like "file foo.exe exists but it isn't executable".

vbezhenar 1y ago

Here's zsh snippet I've came up with for my own simple functions. I'm using it as a base for other completions. In this example, function `set-java-home zulu-21` sets JAVA_HOME to `~/apps/java/zulu-21`. Here's `_set-java-home`: #compdef set-java-home local -a versions=(~/apps/java/*(:t)) _describe 'version' versions So basically almost a one-liner (but couldn't do it really one-liner, unfortunately).

medv 1y ago

JSON fields autocomplete right in bash/zsh: https://fx.wtf/install#autocomplete https://fx.wtf/install#autocomplete

imcritic 1y ago
Thanks for linking this! This is a lightweight solution, compared to ijq (interactive jq), but it still may come in handy. https://github.com/gpanders/ijq https://github.com/gpanders/ijq

xenophonf 1y ago

I wish tcsh would get more love.

esafak 1y ago
Why, it's a dinosaur? Have you tried nushell, murex, oil shell or xonsh?
- xenophonf 1y ago
  I don't know why you're being downvoted. You're correct, and as the other commenter mentioned, csh programming is evil in the eyes of the LORD. Starting last year, I've been maintaining a bash profile that implements (mostly) the same setup as my tcsh configuration. I suppose I ought to make the final leap into something more modern. It would be nice not to jump through hoops to redirect stderr somewhere different than stdout or [insert your favorite of the myriad examples of csh/tcsh weirdness here]! But I feel irrationally sad about it. I cut my teeth on Unix systems where csh was the default shell—namely, NeXTstep and sunOS 4—and it feels a little like saying the final goodbyes to a dead friend. Silly of me, I realize, but there it is.
chasil 1y ago
There is a famous paper on the perils of scripting in the csh. It is unfortunate that Bill Joy was not able to write a formal grammar or parser for his language. It was certainly a missed opportunity, and tcsh cannot fix the design. That being said, csh advocates definitely influenced everything in the Bourne/POSIX family.
- xenophonf 1y ago
  It's a classic Usenet post that deserves linking at every opportunity! https://harmful.cat-v.org/software/csh https://harmful.cat-v.org/software/csh

harimurti 1y ago

[dead]

sebtron 1y ago

Basic completion in ksh is as easy as defining an array. From https://man.openbsd.org/ksh https://man.openbsd.org/ksh : Custom completions may be configured by creating an array named ‘complete_command’, optionally suffixed with an argument number to complete only for a single argument. So defining an array named ‘complete_kill’ provides possible completions for any argument to the kill(1) command, but ‘complete_kill_1’ only completes the first argument. For example, the following command makes ksh offer a selection of signal names for the first argument to kill(1): set -A complete_kill_1 -- -9 -HUP -INFO -KILL -TERM

chasil 1y ago
Is this in the Korn & Bolsy ksh88 book? Or is this ksh93 syntax that oksh back ported?
- sebtron 1y ago
  I don't know actually. Are there currently used version of ksh that are not derived from ksh93?
  - chasil 1y ago
    Very much so - pdksh was a bsd-licensed clone that is the ancestor of mksh (Android's system shell) and OpenBSD's oksh, which you previously mentioned. David Korn thanked the pdksh authors and maintainers for making it available in the years when true ksh was closed (ksh88) or open but licensed with awkward terms (ksh93 for several years). David Korn interview (I asked one of the questions): https://m.slashdot.org/story/16351 https://m.slashdot.org/story/16351 "First of all pdksh is a ksh88 clone; and I might add a better clone than the MKS Korn Shell... "I don't know the pdksh development team but I would like to thank them for the service they have done in making a version of ksh available while ksh was proprietary. I have noticed remarkable improvements in pdksh in its ability to mimic ksh88 functionality. I don't know what plans the pdksh development team has now that ksh93 is available in open source form, but I certainly would help them try to maintain compatibility if they do continue pdksh distribution. Otherwise, I would hope that they would pick up the ksh93 source and help support and enhance it."

gosub100 1y ago

I did something similar to this for tab-completing server names for use with ssh. I went a step further and allowed pattern matching based on the server being qa/prod and on locale. so for instance you could type `ssh co prod <tab>` and it would tab-complete / suggest any servers that were production and located in the Denver datacenter (co is the state abbrev for Colorado, for non-US readers). Unfortunately my work doesn't allow me to share code, but essentially I remapped ssh to a bash script that maintains an environment variable containing the args (you must do this because each <tab> press is an independent invocation of the script. Then you run into persistence problems, so I added a call to compute elapsed seconds so that it flushes the state variable after a 10s timeout). The bash script then forwards the args to a python script that reads a JSON file and figures out which params (such as 'co' or 'qa') map to which hostnames. It also matches against partial hostnames, so when you see this after tab qa-server-co1 qa-server-co2 pr-server-co3 you only need to add '3' to the list of args to narrow it down to 1 match, then hit <enter> to ssh to that host.

paradox460 1y ago

I've started using jdx's usage[1] for my clis. It integrates neatly into clap, and can be used stand alone in scripts. It can generate completions, argparse, manpages, and more I'm still on the fence if replacing the argparse blocks in my fish scripts is worth the hassle, but against things like old school optparse, it's far better [1]: https://usage.jdx.dev/ https://usage.jdx.dev/

FujiApple 1y ago
I’ve recently been building a similar tool [1] which defines a specification for CLIs, though the goals are slightly different to the tool you mention I think. I just added support for fish as it happens. [1] https://github.com/fujiapple852/claptrap https://github.com/fujiapple852/claptrap
- paradox460 1y ago
  Nice. I'll have to add that to my tool belt, although I think usage will still sit front and center haha

vismit2000 1y ago

Since auto text completion is the primary task of an LLM, I have seen GitHub copilot do that very well on both bash and zsh within vscode terminal!

VagabundoP 1y ago

zsh standard tab completion drives me batty. But I come from bash where 99% of the time I just want filename completion.