3 ms·

The POSIX shell (and its precursors and derivatives) are not LR-parsed languages, and they suffer from ambiguities and inconsistencies. The extent of the distru

by chasil 3y ago

The POSIX shell (and its precursors and derivatives) are not LR-parsed languages, and they suffer from ambiguities and inconsistencies. The extent of the distrust is clear in the video and slides below. I don't know as much about the descent of the COMMAND.COM of MS-DOS with the influence of OS/2 into NT CMD.EXE, but I can say with certainty that it was no better. We needed something clean and maintainable, and we didn't get it. https://archive.fosdem.org/2018/schedule/event/code_parsing_posix_s_hell/ https://archive.fosdem.org/2018/schedule/event/code_parsing_...

skissane 3y ago
The problem the slides are raising is that the POSIX shell's grammar is ad hoc: despite the POSIX standard containing an LR yacc grammar, it is annotated with English-language rules which violate the LR formalism. But not being LR-parsed in particular seems like a red herring – if it had been designed to be cleanly LL-parsed, I don't think anyone would be complaining.
- chasil 3y ago
  > The problem the slides are raising is that the POSIX shell's grammar is ad hoc... ...and that's the point. A shell language with no inconsistencies or ambiguities is what we needed, and we didn't get it. A language with a formal yacc grammar would have helped us get it. Neither the POSIX shell nor CMD.EXE are ever going to meet such a requirement. I don't know if DCL was clean enough to have a yacc grammar. If it was, then it would have been a better choice. I don't know why you are defending such a precept when the weaknesses are so very well known. EDIT: A yacc grammar might be preferred as yacc itself is part of POSIX.2, assuring many implementations. You can find lex and yacc here: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ However, had any formal grammar been used, it would have been far better than what we have now.
  - skissane 3y ago
    There are languages with perfectly clean grammars which can't be parsed by yacc because they aren't LALR(1). I agree that a command processor should have a grammar that can be expressed in a well-known formalism, and its parser generated by a parser generator. I agree that both POSIX shell and CMD.EXE are flawed because that isn't true. What I'm disagreeing with, is that it is important that the grammar formalism be LALR(1) in particular, and that the parser generator be yacc in particular. Suppose I have a Packrat parser generator. [0] And my command processor has a nice clean PEG grammar. [1] And I use the Packrat parser generator to generate the parser of my command processor. That grammar quite possibly isn't LALR(1), and hence yacc in particular won't be able to generate a parser for it. But what's the problem with that? If it is a problem at all, it is a very different problem than the problem that CMD.EXE and POSIX shell have [0] e.g. https://github.com/arithy/packcc https://github.com/arithy/packcc [1] https://en.wikipedia.org/wiki/Parsing_expression_grammar https://en.wikipedia.org/wiki/Parsing_expression_grammar

by chasil 3y ago

skissane 3y ago

The problem the slides are raising is that the POSIX shell's grammar is ad hoc: despite the POSIX standard containing an LR yacc grammar, it is annotated with English-language rules which violate the LR formalism. But not being LR-parsed in particular seems like a red herring – if it had been designed to be cleanly LL-parsed, I don't think anyone would be complaining.

chasil 3y ago
> The problem the slides are raising is that the POSIX shell's grammar is ad hoc... ...and that's the point. A shell language with no inconsistencies or ambiguities is what we needed, and we didn't get it. A language with a formal yacc grammar would have helped us get it. Neither the POSIX shell nor CMD.EXE are ever going to meet such a requirement. I don't know if DCL was clean enough to have a yacc grammar. If it was, then it would have been a better choice. I don't know why you are defending such a precept when the weaknesses are so very well known. EDIT: A yacc grammar might be preferred as yacc itself is part of POSIX.2, assuring many implementations. You can find lex and yacc here: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ However, had any formal grammar been used, it would have been far better than what we have now.
- skissane 3y ago
  There are languages with perfectly clean grammars which can't be parsed by yacc because they aren't LALR(1). I agree that a command processor should have a grammar that can be expressed in a well-known formalism, and its parser generated by a parser generator. I agree that both POSIX shell and CMD.EXE are flawed because that isn't true. What I'm disagreeing with, is that it is important that the grammar formalism be LALR(1) in particular, and that the parser generator be yacc in particular. Suppose I have a Packrat parser generator. [0] And my command processor has a nice clean PEG grammar. [1] And I use the Packrat parser generator to generate the parser of my command processor. That grammar quite possibly isn't LALR(1), and hence yacc in particular won't be able to generate a parser for it. But what's the problem with that? If it is a problem at all, it is a very different problem than the problem that CMD.EXE and POSIX shell have [0] e.g. https://github.com/arithy/packcc https://github.com/arithy/packcc [1] https://en.wikipedia.org/wiki/Parsing_expression_grammar https://en.wikipedia.org/wiki/Parsing_expression_grammar