Yes, so bash is unique in that there is a two level structure. Tokens turn are parsed into words. And then words are used as "tokens" for the "command sublang

Yes, so bash is unique in that there is a two level structure. Tokens turn are parsed into words. And then words are used as "tokens" for the "command sublanguage", the grammar for which is given in POSIX. I mention that in a few places -- there are 4 sublanguages, arith/bool/word/command, but POSIX only covers the command sublanguage. http://www.oilshell.org/blog/2016/10/26.html http://www.oilshell.org/blog/2016/10/26.html This is one of the reasons parser generators as "frameworks" don't work for shell. I have a diagram in mind for the parser architecture which I think will be illuminating. The command language uses the words as "tokens", but they're also mutually recursive, because an entire subprogram can fit inside a word/string, like "prefix $(my-func() { echo hi; }; my-func) suffix" That is defining a function and calling it inside a string.