4 ms·

My issue with bash completion is that it requires a completion script (i.e executing `complete`) for each command you want it to complete. The shell cannot auto

by vunie 7y ago

My issue with bash completion is that it requires a completion script (i.e executing `complete`) for each command you want it to complete. The shell cannot automatically deduce appropriate completions when possible. This problem is not specific to bash. Fish and other shells can't automatically complete commands either. There is simply no standard way to detect what type of auto-completion a command supports. I know this comment is a not about the OP, but I've wasted so much time dealing with this that I feel compelled to post this here. I wish shell developers would form some sort of consortium (like XDG) where they can agree on standard solutions to issues like this. No one benefits from having developers waste time manually writing and testing completion scripts for each project (times the number of shells they want to support). Side note: I remember that there was a project that modified bash internals to detect automatic completion support. It did this by scanning for the presence of a magic string in the first N bytes of a file (or in some ELF section of a binary). If that magic string was present, bash would automatically generate a completion set by passing `--_complete` to the command. I think this is a simple and elegant solution to this problem and one I hope shell developers would consider.

shakna 7y ago
> The shell cannot automatically deduce appropriate completions when possible. I don't think this even _can_ be possible for a large number of programs. For some small number of programs it _might_ be possible, if somehow (ignoring how for now) you were exposed some standard interfaces like getopt or argparse and didn't have to care about argument ordering. But a ton of other programs, with extremely complicated interfaces, simply wouldn't be possible. And these include commonly used programs. Auto-completing awk would require a full language parser at the least, for example. Another, ffmpeg, has a dizzying array of options, and the order of those options can completely change the intent of the command, and what options are allowed to follow without being ignored silently. Even if we had perfect interface detection, we could only probably generate a subset of auto-completion options because that particular problem might not _always_ be solveable.
- vunie 7y ago
  You raise a good point about solvable completion, and one that I agree with. However, automatic completion support doesn't have to cover every use case. It only needs to be good enough. Complex applications can fall back to providing their own completion scripts. One point I'm not clear on is why you think such a scheme wouldn't be feasible for a large number of commands. Completion wouldn't be invoked until the fist argument (the command's name) is complete just like how bash doesn't scan the system's completion directory until it knows what command is being invoked. And as for completion for the command's arguments, the command is only ran once to generate the list of completion candidates. Note that there are projects that already do exactly this like [1] . [1]:https://github.com/kislyuk/argcomplete https://github.com/kislyuk/argcomplete
  - shakna 7y ago
    > One point I'm not clear on is why you think such a scheme wouldn't be feasible for a large number of commands. Completion wouldn't be invoked until the fist argument (the command's name) is complete just like how bash doesn't scan the system's completion directory until it knows what command is being invoked. Simply because some command pipelines are extremely complicated. For example, with ffmpeg, some flags can disable earlier flags, or re-enable them. Completion isn't a simple left-to-right thing. There's other programs that can be similar.
    - vunie 7y ago
      > Completion isn't a simple left-to-right thing. Sorry, I still don't understand why that's relavent. It's probably my fault for not being clear enough. Please be patient with me as I try to further elaborate: If the command is in charge of both generating completions and also for parsing arguments, wouldn't it be aware of all inner-dependencies between flags? Wouldn't such awareness make it possible to generate candidates based on previous flags? I'm not familiar with ffmpeg, but let's assume that there was another similar program called ggmpeg. Let's assume that it accepts the flag '--input-format' and then either the flag '--repeat-video-range' or '--repeat-image-frames', depending on whether the format is an video or an image. That is, only the following flag sequences are valid: ggmpeg --input-format=video/webm --repeat-video-range=.. ... or ggmpeg --input-format=image/png --repeat-image-frames=.. ... Let's also assume that ggmpeg signals to the shell that it is responsible for generating its own completions by passing the flag --_completion to it. Now, if the user types the following command ggmpeg --input-format=image/png --repea[CURSOR] and then presses tab, thereby invoking the shell's auto completion routine, which in turn ultimately executes "ggmpeg" "--_completion" "--input-format=image/png" "--repea" wouldn't ggmpeg have all the necessary information to only suggest '--repeat-image-frames' and not '--repeat-video-range'?
greggman2 7y ago
I'm only guessing this is what PowerShell was supposed to enable. I've never used it but I gather programs are supposed to offer enough metadata to do this?
hnlmorg 7y ago
Some shells do parse manages and use other tricks to auto-determine completes. Though they're not POSIX compliant shells so you'd have a to retrain a little there. I'm interested to know more about that ELF / --_complete trick though. Do you have any more details? (when DDging the only result I can find on the topic is your comment)
- vunie 7y ago
  > (when DDging the only result I can find on the topic is your comment) You weren't kidding at all. After spending more time than I'd care to admit searching for the library, I couldn't find traces of it anywhere on google, ddg, or bing. I relented and digged whatever information I could find in my history. Revisiting these projects, it's painfully obvious that I have jumbled a few projects together. Sincere apologies for the unintended goose chase. This project [1] is the one that introduced the `--_completion` flag. It registers itself as a default completion function, but blindly and dangerously tries to run any command it gets with this flag. (This project is completely de-indexed from search engines and I'm not sure why). Like [1], project [2] registers itself as a default completion function, but checks the first 1024 bytes for the magic string "PYTHON_ARGCOMPLETE_OK". It seems to be relying on argparse for completion. I couldn't manage to find the project that was scanning elf sections. I'm not entirely sure if it was relying on elf sections specifically or if the fact that it was scanning elf sections was only an implementation detail. [1]: https://github.com/dbarnett/python-selfcompletion https://github.com/dbarnett/python-selfcompletion [2]: https://github.com/kislyuk/argcomplete https://github.com/kislyuk/argcomplete
  - hnlmorg 7y ago
    Awesome. I really appreciate you taking the time to research that for me. Some great material for me to read through. Thank you

by vunie 7y ago

shakna 7y ago

> The shell cannot automatically deduce appropriate completions when possible. I don't think this even _can_ be possible for a large number of programs. For some small number of programs it _might_ be possible, if somehow (ignoring how for now) you were exposed some standard interfaces like getopt or argparse and didn't have to care about argument ordering. But a ton of other programs, with extremely complicated interfaces, simply wouldn't be possible. And these include commonly used programs. Auto-completing awk would require a full language parser at the least, for example. Another, ffmpeg, has a dizzying array of options, and the order of those options can completely change the intent of the command, and what options are allowed to follow without being ignored silently. Even if we had perfect interface detection, we could only probably generate a subset of auto-completion options because that particular problem might not _always_ be solveable.

vunie 7y ago
You raise a good point about solvable completion, and one that I agree with. However, automatic completion support doesn't have to cover every use case. It only needs to be good enough. Complex applications can fall back to providing their own completion scripts. One point I'm not clear on is why you think such a scheme wouldn't be feasible for a large number of commands. Completion wouldn't be invoked until the fist argument (the command's name) is complete just like how bash doesn't scan the system's completion directory until it knows what command is being invoked. And as for completion for the command's arguments, the command is only ran once to generate the list of completion candidates. Note that there are projects that already do exactly this like [1] . [1]:https://github.com/kislyuk/argcomplete https://github.com/kislyuk/argcomplete
- shakna 7y ago
  > One point I'm not clear on is why you think such a scheme wouldn't be feasible for a large number of commands. Completion wouldn't be invoked until the fist argument (the command's name) is complete just like how bash doesn't scan the system's completion directory until it knows what command is being invoked. Simply because some command pipelines are extremely complicated. For example, with ffmpeg, some flags can disable earlier flags, or re-enable them. Completion isn't a simple left-to-right thing. There's other programs that can be similar.
  - vunie 7y ago
    > Completion isn't a simple left-to-right thing. Sorry, I still don't understand why that's relavent. It's probably my fault for not being clear enough. Please be patient with me as I try to further elaborate: If the command is in charge of both generating completions and also for parsing arguments, wouldn't it be aware of all inner-dependencies between flags? Wouldn't such awareness make it possible to generate candidates based on previous flags? I'm not familiar with ffmpeg, but let's assume that there was another similar program called ggmpeg. Let's assume that it accepts the flag '--input-format' and then either the flag '--repeat-video-range' or '--repeat-image-frames', depending on whether the format is an video or an image. That is, only the following flag sequences are valid: ggmpeg --input-format=video/webm --repeat-video-range=.. ... or ggmpeg --input-format=image/png --repeat-image-frames=.. ... Let's also assume that ggmpeg signals to the shell that it is responsible for generating its own completions by passing the flag --_completion to it. Now, if the user types the following command ggmpeg --input-format=image/png --repea[CURSOR] and then presses tab, thereby invoking the shell's auto completion routine, which in turn ultimately executes "ggmpeg" "--_completion" "--input-format=image/png" "--repea" wouldn't ggmpeg have all the necessary information to only suggest '--repeat-image-frames' and not '--repeat-video-range'?

greggman2 7y ago

I'm only guessing this is what PowerShell was supposed to enable. I've never used it but I gather programs are supposed to offer enough metadata to do this?

hnlmorg 7y ago

Some shells do parse manages and use other tricks to auto-determine completes. Though they're not POSIX compliant shells so you'd have a to retrain a little there. I'm interested to know more about that ELF / --_complete trick though. Do you have any more details? (when DDging the only result I can find on the topic is your comment)

vunie 7y ago
> (when DDging the only result I can find on the topic is your comment) You weren't kidding at all. After spending more time than I'd care to admit searching for the library, I couldn't find traces of it anywhere on google, ddg, or bing. I relented and digged whatever information I could find in my history. Revisiting these projects, it's painfully obvious that I have jumbled a few projects together. Sincere apologies for the unintended goose chase. This project [1] is the one that introduced the `--_completion` flag. It registers itself as a default completion function, but blindly and dangerously tries to run any command it gets with this flag. (This project is completely de-indexed from search engines and I'm not sure why). Like [1], project [2] registers itself as a default completion function, but checks the first 1024 bytes for the magic string "PYTHON_ARGCOMPLETE_OK". It seems to be relying on argparse for completion. I couldn't manage to find the project that was scanning elf sections. I'm not entirely sure if it was relying on elf sections specifically or if the fact that it was scanning elf sections was only an implementation detail. [1]: https://github.com/dbarnett/python-selfcompletion https://github.com/dbarnett/python-selfcompletion [2]: https://github.com/kislyuk/argcomplete https://github.com/kislyuk/argcomplete
- hnlmorg 7y ago
  Awesome. I really appreciate you taking the time to research that for me. Some great material for me to read through. Thank you