15 ms·

Really I would love to know how parse context sensitive stuff like typedef which will have "switched" syntax for some tokens. Would like to know things like "ho

by stevefan1999 9mo ago

Really I would love to know how parse context sensitive stuff like typedef which will have "switched" syntax for some tokens. Would like to know things like "hoisting" in C++, where you can you the class and struct after the code inside the function too, but I just find it hard to describe them in rigorous formal language and grammar. Hacky solution for PEG such as adding a context stack requires careful management of the entry/exit point, but the more fundamental problem is that you still can't "switch" syntax, or you have to add all possible syntax combination depending on the numbers of such stacks. I believe persistent data structure and transactional data structure would help but I just couldn't find a formalism for that.

remexre 9mo ago
https://en.wikipedia.org/wiki/Lexer_hack https://en.wikipedia.org/wiki/Lexer_hack Make your parser call back into your lexer, so it can pass state to it; make the set of type names available to it.
torginus 9mo ago
C/C++ has one of the worst-designed syntaxes, its such a shame that entire families of the most popular languages ended up copying the same mistakes. I know it's no solace to you, but Rust and Go don't even have this problem Afaik, and it's avoidable by careful consideration.
- suspended_state 9mo ago
  I don't really know what you mean by "worst-designed syntax". Do you mean that the design process was bad, or that the result is bad?
  - torginus 9mo ago
    I meant exactly what the parent-comment pointed out - that C can't be parsed without a symbol table. Like the example on wikipedia: A * B; Which either represents a multiplication or a pointer of type A* to B, depending what the symbol table looks like. That means parsing C is impossible without these hacks, and you need to basically parse the whole file to build up this information. A lot text editors which only support grammars can't parse this properly, there are a ton of buggy C parsers in the wild. The issues that led to this were completely avoidable, and many languages like Pascal (which was more or less its contemporary), Go or Rust did avoid them. They don't lead to the language being more powerful or expressive. Calling it the 'worst' might be hyperbole, but given how influential C-style syntax has become, and how much C code is out there, these issue have affected a ton of other languages downstream.
    - suspended_state 9mo ago
      So you were criticizing the C language syntax, without considering the context which it was designed in. Just to give this context a little bit more substance, Pascal was designed to work on a mainframe which could address up to 4MB of RAM, with a typical setup of around 1MB (it's actually not the real amounts: the CDC-6600 the values are 128Kwords, but it had 60 bits word). These machine were beasts designed for scientific computation. The first C compiler was implemented on a PDP-11, which could handle up to 64KB of RAM, and had 16bits words. I assume that these constraints had a heavy influence on how each language was designed and implemented. Note that I wasn't aware of all these details before writing this comment, I had to check. See: http://pascal.hansotten.com/niklaus-wirth/zurich-pascal-compilers/ http://pascal.hansotten.com/niklaus-wirth/zurich-pascal-comp... Regarding the C compiler, it is likely that the first version was written in assembly language, which was later "translated" to C. An early version of the compiler can be found there: https://github.com/theunafraid/first_c_compiler https://github.com/theunafraid/first_c_compiler and does look like assembly hand converted to (early) C.
luksenburg 9mo ago
Another possible solution is the usage of functional parsers (e.g.: [0]) and making use of some form of the ‘do’ notation. Each step makes its result available to all subsequent parsers. [0] https://hackage.haskell.org/package/parsec https://hackage.haskell.org/package/parsec

remexre 9mo ago

https://en.wikipedia.org/wiki/Lexer_hack https://en.wikipedia.org/wiki/Lexer_hack Make your parser call back into your lexer, so it can pass state to it; make the set of type names available to it.

torginus 9mo ago

C/C++ has one of the worst-designed syntaxes, its such a shame that entire families of the most popular languages ended up copying the same mistakes. I know it's no solace to you, but Rust and Go don't even have this problem Afaik, and it's avoidable by careful consideration.

suspended_state 9mo ago
I don't really know what you mean by "worst-designed syntax". Do you mean that the design process was bad, or that the result is bad?
- torginus 9mo ago
  I meant exactly what the parent-comment pointed out - that C can't be parsed without a symbol table. Like the example on wikipedia: A * B; Which either represents a multiplication or a pointer of type A* to B, depending what the symbol table looks like. That means parsing C is impossible without these hacks, and you need to basically parse the whole file to build up this information. A lot text editors which only support grammars can't parse this properly, there are a ton of buggy C parsers in the wild. The issues that led to this were completely avoidable, and many languages like Pascal (which was more or less its contemporary), Go or Rust did avoid them. They don't lead to the language being more powerful or expressive. Calling it the 'worst' might be hyperbole, but given how influential C-style syntax has become, and how much C code is out there, these issue have affected a ton of other languages downstream.
  - suspended_state 9mo ago
    So you were criticizing the C language syntax, without considering the context which it was designed in. Just to give this context a little bit more substance, Pascal was designed to work on a mainframe which could address up to 4MB of RAM, with a typical setup of around 1MB (it's actually not the real amounts: the CDC-6600 the values are 128Kwords, but it had 60 bits word). These machine were beasts designed for scientific computation. The first C compiler was implemented on a PDP-11, which could handle up to 64KB of RAM, and had 16bits words. I assume that these constraints had a heavy influence on how each language was designed and implemented. Note that I wasn't aware of all these details before writing this comment, I had to check. See: http://pascal.hansotten.com/niklaus-wirth/zurich-pascal-compilers/ http://pascal.hansotten.com/niklaus-wirth/zurich-pascal-comp... Regarding the C compiler, it is likely that the first version was written in assembly language, which was later "translated" to C. An early version of the compiler can be found there: https://github.com/theunafraid/first_c_compiler https://github.com/theunafraid/first_c_compiler and does look like assembly hand converted to (early) C.

luksenburg 9mo ago

Another possible solution is the usage of functional parsers (e.g.: [0]) and making use of some form of the ‘do’ notation. Each step makes its result available to all subsequent parsers. [0] https://hackage.haskell.org/package/parsec https://hackage.haskell.org/package/parsec