3 ms·

That's an interesting invariant that I've been thinking about: hitting TAB should never generate something that would be an invalid command (or more pedanticall

by chubot 8y ago

That's an interesting invariant that I've been thinking about: hitting TAB should never generate something that would be an invalid command (or more pedantically, generate a command prefix that would have a syntax error once the user fills in the rest of the command). You would THINK that shell tab completion already works that way. But it doesn't in bash, and not even in fish or zsh. Not only does the completion logic duplicates the tool's own parsing logic, it doesn't attempt to do it faithfully. (1) First, there's version skew in all the repos for bash, fish, and zsh. (2) Leaving that aside, the parsing models they use aren't rich enough to express some command lines. "find" is a good example. They will let you do something like find -type f -dept<TAB> -- completes -depth, but it's not valid here This is because find is split up into options and the expression language. There are probably some simpler and more compelling examples I can come up with. Another one is completing flags in this situation: grep -- --c<TAB> After --, there are only args, not flags, so it's misleading to complete. ---- Being able to spam TAB is what I'm going for with Oil. That's not true of bash. One annoyance the (y/n) prompt that bash gives you when there are thousands of completions (e.g. hitting TAB on an empty line). I have some upcoming blog posts along these lines, which I mention here: http://www.oilshell.org/blog/2019/02/05.html http://www.oilshell.org/blog/2019/02/05.html

arketyp 8y ago
Glad to hear someone is thinking about this. Right, hitting tab should never generate something invalid. That's the shell being sound. Conversely, the system should also be able to determine whether what's being typed could possibly become a valid command or not. That's the shell being complete.
- chubot 8y ago
  Yes, I think it basically reduces to the problem of using the tool's ACTUAL parser. There are enough variations in flag syntax that it's not possible to be 100% correct without running arbitrary code. And if you're going to do that, you might as well run the code that's already written (with some caveats). That could be solved by my proposal for a "Shellac protocol", akin to Microsoft's Language Server Protocol (e.g. for VSCode): http://www.oilshell.org/blog/2018/12/05.html http://www.oilshell.org/blog/2018/12/05.html Summary: 1. The protocol was mainly motivated by the problem of bootstrapping a corpus of shell completions. Every shell has to "boil the ocean" itself, i.e. provide completions for many popular commands. git's completion is thousands of lines in each of bash, zsh, and fish. Upstream only maintains bash. 2. What I noticed in implementing completion for Oil is that bash fundamentally CONFLATES two problems: a) The problem of completing the shell language itself, e.g. ${H -> ${HOSTNAME, or ~a -> ~andy b) The problem of completing command line flag syntax. This is a huge mistake from the POV of implementation complexity. And obviously if you want shell-agnostic auto-completions, (b) needs to be separate from (a), which is shell-specific. Luckily the authors of bash-completion tried to separate these two problems a bit, and I was able to salvage most code. Since I wrote that post, and implemented a compatible completion API in OSH, it's become a lot clearer to me how the protocol should work. 3) The invariant you mention would be a third benefit of having a more principled completion API that can run arbitrary code (in arbitrary languages). Now, I'm not saying that EVERY command needs to use this. There are some engineering problems with this approach too. But I think you should have the POSSIBILITY of doing something correct if you want. Heuristics may be OK in the simple cases. But overall what I discovered is that the system is a huge mess. The bash-completion project tries to parse bash in bash, and unsurprisingly does a poor job. My upcoming blog posts will explain this.
the_duke 8y ago
Bash/ZSH can have arbitary completion functions/scripts that receive the full input. A lot of the completions are auto-generated from man files, which don't contain information like "argument x is only available if argument y is present", etc. (at least in a structured way) It's "just" a matter of implementing completions manually to enable what you are asking for.

by chubot 8y ago

arketyp 8y ago

Glad to hear someone is thinking about this. Right, hitting tab should never generate something invalid. That's the shell being sound. Conversely, the system should also be able to determine whether what's being typed could possibly become a valid command or not. That's the shell being complete.

chubot 8y ago
Yes, I think it basically reduces to the problem of using the tool's ACTUAL parser. There are enough variations in flag syntax that it's not possible to be 100% correct without running arbitrary code. And if you're going to do that, you might as well run the code that's already written (with some caveats). That could be solved by my proposal for a "Shellac protocol", akin to Microsoft's Language Server Protocol (e.g. for VSCode): http://www.oilshell.org/blog/2018/12/05.html http://www.oilshell.org/blog/2018/12/05.html Summary: 1. The protocol was mainly motivated by the problem of bootstrapping a corpus of shell completions. Every shell has to "boil the ocean" itself, i.e. provide completions for many popular commands. git's completion is thousands of lines in each of bash, zsh, and fish. Upstream only maintains bash. 2. What I noticed in implementing completion for Oil is that bash fundamentally CONFLATES two problems: a) The problem of completing the shell language itself, e.g. ${H -> ${HOSTNAME, or ~a -> ~andy b) The problem of completing command line flag syntax. This is a huge mistake from the POV of implementation complexity. And obviously if you want shell-agnostic auto-completions, (b) needs to be separate from (a), which is shell-specific. Luckily the authors of bash-completion tried to separate these two problems a bit, and I was able to salvage most code. Since I wrote that post, and implemented a compatible completion API in OSH, it's become a lot clearer to me how the protocol should work. 3) The invariant you mention would be a third benefit of having a more principled completion API that can run arbitrary code (in arbitrary languages). Now, I'm not saying that EVERY command needs to use this. There are some engineering problems with this approach too. But I think you should have the POSSIBILITY of doing something correct if you want. Heuristics may be OK in the simple cases. But overall what I discovered is that the system is a huge mess. The bash-completion project tries to parse bash in bash, and unsurprisingly does a poor job. My upcoming blog posts will explain this.

the_duke 8y ago

Bash/ZSH can have arbitary completion functions/scripts that receive the full input. A lot of the completions are auto-generated from man files, which don't contain information like "argument x is only available if argument y is present", etc. (at least in a structured way) It's "just" a matter of implementing completions manually to enable what you are asking for.