11 ms·

I wonder how this compares to boost spirit.

by Foomf 5y ago

xyzzy_plugh 5y ago
Well for one it's about a million times more readable than goddamn Spirit. It still abuses operator overloading so it's just as terrible. Use a real tool instead of hacks like this. For fun? Do whatever you want. If I ever come across something like this in production it's going to be removed immediately. Seriously, what's so wrong with lex + yacc, or antlr or parsec or ragel or any of the others?
- ncmncm 5y ago
  Lex, yacc, antlr, parsec, ragel, and the others are all hacks. Necessary hacks, when the language is inadequate to the job. But C++ is no longer inadequate. Lex and yacc depend on global variables, so you get one (1) parser per program, unless you further massage their output.
  - klodolph 5y ago
    Flex and Bison depend on global variables by default, but you can eliminate the global variables if you like (e.g. %option reentrant) and change the prefix if you like (%option prefix="mylexer_"). Bison has similar options. I'm sure you could only have one lexer back at some point in the distant past, but that particular issue has been solvable for a long time by now.
    - EuAndreh 5y ago
      lex and yacc depend on global variables, flex and bison, which are one the implementations of lex and yacc, don't.
      - klodolph 5y ago
        Is there a reason why you'd be stuck on lex / yacc and unable to use flex / bison? The main reason I can think of is that you are in a legacy code base which already uses lex / yacc.
        EuAndreh 5y ago
        lex and yacc are part of POSIX, while flex and bison aren't.
        klodolph 5y ago
        Sure, and C++ isn't part of POSIX. So if you must stick to strictly POSIX utilities, you cannot use C++.
        ncmncm 5y ago
        C++ and Posix are both ISO Standards.
        klodolph 5y ago
        Are you constrained to only use software which is standardized by ISO? I'm just trying to figure out what you're saying here.
        ncmncm 5y ago
        I am not so constrained, but some are. But in general, for anybody deploying software widely, any time you can avoid a necessary build dependency on another tool, you come out ahead.
        klodolph 5y ago
        Could you tell me more about who is constrained this way? I'm curious about what kind of organization has these kinds of constraints. I have some friends who work in aerospace or at defense contractors and these constraints sound a little extreme even by their standards.
        ncmncm 5y ago
        They would know better than I do.
  - pcwalton 5y ago
    Is C++ adequate, when compile time and maintainability are taken into account?
    - ncmncm 5y ago
      Compile time seems like an odd thing for you to mention here. I don't know if your favorite language is similarly adequate, but I know it compiles way more slowly. But generally the time to analyze a constexpr initializer expression, even of even a pretty substantial grammar, will generally be pretty negligible, particularly given that it need not appear in a header file.
    - pjmlp 5y ago
      Microsoft would say so, specially with all the module's talks. I guess all those logos on ISO C++ meetings would state similar assertions. Some of them might be playing with Rust, but when it comes down to shipping SDKs, it is all about C++.
  - kazinator 5y ago
    This is false; GNU Bison introduced re-entrant parsing probably about a decade and half ago at least, in combination with Flex, and Berkeley Yacc implemented a compatible implementation. I use this in TXR Lisp. There are situations in which a Lisp macro may be expanded in the context of the parser. The macro can use the parser (e.g. call (read "(1 2 3)"). This works.
- dataflow 5y ago
  > Seriously, what's so wrong with lex + yacc, or antlr or parsec or ragel or any of the others? Not sure ANTLR is as fast (doesn't it use recursive descent? which is slower algorithmically) but even ignoring performance, it's basically the cost of adding a new tool to your toolchain. If you already have 20 other tools, maybe the 21st isn't a big deal, but if you have 2, the 3rd adds some complexity. And in general it's kinda nice to have fewer things to glue together. (I'm not claiming "therefore this approach trumps what you listed", I'm just explaining some of its advantages. Obviously it has disadvantages too.)
  - jjice 5y ago
    ANTLR is LL, but not sure how that stacks up against LR in yacc in terms of speed.
- jcelerier 5y ago
  that's nice, you and me will be in a cycle of immediately replacing each other's code. Seriously, lex+yacc / flex+bison absolutely suck, good luck when you have to put them as part of a complex cross-compilation build pipeline or run on windows. Also, have fun when you have multiple grammars in the same program and you don't want "colorful" ODR issues due to LTO, etc... they are entirely unfit for the job. In contrast Spirit just works for me after spending an hour reading the tutorial (my main problem was the sad issue common with expression templates and auto). It also does not require to link anything, just -I/path/to/boost no matter the platform and tada, you get a fairly performant parser.
  - klodolph 5y ago
    Flex / Bison are manageable if you spend a bit of extra time reading the manual. I admit that there are often better options out there these days, I'm just saying that some of these complaints about Flex / Bison may be a bit outdated. Multiple grammars are fine, you just instruct Flex and Bison to use static or add prefixes. No worries about ODR or LTO. If your build system can't handle generated code, that's an indictment of your build system. I know that's a bit harsh, but there are plenty of build systems which can handle generated code and cross-compile just fine. Flex and Bison work fine on Windows.
    - jcelerier 5y ago
      It's good that prefixes are now possible, but it's still something additional to know (I have never seen them in the wild and I've seen a lot of yacc in various projects), while most C++ programmers don't have to search inside some manual to know that two parsers must not have the same name : while with yacc / bison, you generally only discover it when it crashes at run-time. Also http://gnuwin32.sourceforge.net/packages/bison.htm http://gnuwin32.sourceforge.net/packages/bison.htm is not what "work fine on Windows" mean (hell, it does not even have the prefix thing you mentioned in the other comment). If there's anything ressembling a ./configure / some step that requires bash in its build pipeline then it's a complete no-go and given the GNU history of bison I doubt it'd use CMake. Though even if all of this was solved, it'd still not be as convenient as Spirit.
- pjmlp 5y ago
  Most of them have poor debugging tools, one has to try to guess what the piles of generated code actually do, hence why they are traditionally only used to bootstrap a compiler, and eventually the parser gets hand written.
  - gpderetta 5y ago
    I understand the biggest issue with automatically generated parsers is error handling, i.e. pleasant and precise error messages to the user and recovering from partial parse failures (for example for tooling like completion).
- junon 5y ago
  > lex + yacc They're abysmal to use, have poor interfaces and poor error handling facilities. > antlr At least when I tried it years ago, it was an unapproachable mess of java tooling and editors. > parsec Is a parser combinator. This is different than a generator. > ragel Not heard about this.
  - amir734jj 5y ago
    Try Antlr 4. It's more approachable.
peter-winter 5y ago
I'm the author, let me just answer this one. Spirit produces recursive descent parser, ctpg produces LR(1). So the parse speed is just much better. And the set of possible language grammars is bigger. Plus (this is just my opinion) ctpg just looks more intuitive.

I wonder how this compares to boost spirit.

by Foomf 5y ago

I wonder how this compares to boost spirit.

Well for one it's about a million times more readable than goddamn Spirit. It still abuses operator overloading so it's just as terrible. Use a real tool instead of hacks like this. For fun? Do whatever you want. If I ever come across something like this in production it's going to be removed immediately. Seriously, what's so wrong with lex + yacc, or antlr or parsec or ragel or any of the others?

ncmncm 5y ago
Lex, yacc, antlr, parsec, ragel, and the others are all hacks. Necessary hacks, when the language is inadequate to the job. But C++ is no longer inadequate. Lex and yacc depend on global variables, so you get one (1) parser per program, unless you further massage their output.
- klodolph 5y ago
  Flex and Bison depend on global variables by default, but you can eliminate the global variables if you like (e.g. %option reentrant) and change the prefix if you like (%option prefix="mylexer_"). Bison has similar options. I'm sure you could only have one lexer back at some point in the distant past, but that particular issue has been solvable for a long time by now.
  - EuAndreh 5y ago
    lex and yacc depend on global variables, flex and bison, which are one the implementations of lex and yacc, don't.
    - klodolph 5y ago
      Is there a reason why you'd be stuck on lex / yacc and unable to use flex / bison? The main reason I can think of is that you are in a legacy code base which already uses lex / yacc.
      - EuAndreh 5y ago
        lex and yacc are part of POSIX, while flex and bison aren't.
        klodolph 5y ago
        Sure, and C++ isn't part of POSIX. So if you must stick to strictly POSIX utilities, you cannot use C++.
        ncmncm 5y ago
        C++ and Posix are both ISO Standards.
        klodolph 5y ago
        Are you constrained to only use software which is standardized by ISO? I'm just trying to figure out what you're saying here.
        ncmncm 5y ago
        I am not so constrained, but some are. But in general, for anybody deploying software widely, any time you can avoid a necessary build dependency on another tool, you come out ahead.
        klodolph 5y ago
        Could you tell me more about who is constrained this way? I'm curious about what kind of organization has these kinds of constraints. I have some friends who work in aerospace or at defense contractors and these constraints sound a little extreme even by their standards.
        ncmncm 5y ago
        They would know better than I do.
- pcwalton 5y ago
  Is C++ adequate, when compile time and maintainability are taken into account?
  - ncmncm 5y ago
    Compile time seems like an odd thing for you to mention here. I don't know if your favorite language is similarly adequate, but I know it compiles way more slowly. But generally the time to analyze a constexpr initializer expression, even of even a pretty substantial grammar, will generally be pretty negligible, particularly given that it need not appear in a header file.
  - pjmlp 5y ago
    Microsoft would say so, specially with all the module's talks. I guess all those logos on ISO C++ meetings would state similar assertions. Some of them might be playing with Rust, but when it comes down to shipping SDKs, it is all about C++.
- kazinator 5y ago
  This is false; GNU Bison introduced re-entrant parsing probably about a decade and half ago at least, in combination with Flex, and Berkeley Yacc implemented a compatible implementation. I use this in TXR Lisp. There are situations in which a Lisp macro may be expanded in the context of the parser. The macro can use the parser (e.g. call (read "(1 2 3)"). This works.
dataflow 5y ago
> Seriously, what's so wrong with lex + yacc, or antlr or parsec or ragel or any of the others? Not sure ANTLR is as fast (doesn't it use recursive descent? which is slower algorithmically) but even ignoring performance, it's basically the cost of adding a new tool to your toolchain. If you already have 20 other tools, maybe the 21st isn't a big deal, but if you have 2, the 3rd adds some complexity. And in general it's kinda nice to have fewer things to glue together. (I'm not claiming "therefore this approach trumps what you listed", I'm just explaining some of its advantages. Obviously it has disadvantages too.)
- jjice 5y ago
  ANTLR is LL, but not sure how that stacks up against LR in yacc in terms of speed.
jcelerier 5y ago
that's nice, you and me will be in a cycle of immediately replacing each other's code. Seriously, lex+yacc / flex+bison absolutely suck, good luck when you have to put them as part of a complex cross-compilation build pipeline or run on windows. Also, have fun when you have multiple grammars in the same program and you don't want "colorful" ODR issues due to LTO, etc... they are entirely unfit for the job. In contrast Spirit just works for me after spending an hour reading the tutorial (my main problem was the sad issue common with expression templates and auto). It also does not require to link anything, just -I/path/to/boost no matter the platform and tada, you get a fairly performant parser.
- klodolph 5y ago
  Flex / Bison are manageable if you spend a bit of extra time reading the manual. I admit that there are often better options out there these days, I'm just saying that some of these complaints about Flex / Bison may be a bit outdated. Multiple grammars are fine, you just instruct Flex and Bison to use static or add prefixes. No worries about ODR or LTO. If your build system can't handle generated code, that's an indictment of your build system. I know that's a bit harsh, but there are plenty of build systems which can handle generated code and cross-compile just fine. Flex and Bison work fine on Windows.
  - jcelerier 5y ago
    It's good that prefixes are now possible, but it's still something additional to know (I have never seen them in the wild and I've seen a lot of yacc in various projects), while most C++ programmers don't have to search inside some manual to know that two parsers must not have the same name : while with yacc / bison, you generally only discover it when it crashes at run-time. Also http://gnuwin32.sourceforge.net/packages/bison.htm http://gnuwin32.sourceforge.net/packages/bison.htm is not what "work fine on Windows" mean (hell, it does not even have the prefix thing you mentioned in the other comment). If there's anything ressembling a ./configure / some step that requires bash in its build pipeline then it's a complete no-go and given the GNU history of bison I doubt it'd use CMake. Though even if all of this was solved, it'd still not be as convenient as Spirit.
pjmlp 5y ago
Most of them have poor debugging tools, one has to try to guess what the piles of generated code actually do, hence why they are traditionally only used to bootstrap a compiler, and eventually the parser gets hand written.
- gpderetta 5y ago
  I understand the biggest issue with automatically generated parsers is error handling, i.e. pleasant and precise error messages to the user and recovering from partial parse failures (for example for tooling like completion).
junon 5y ago
> lex + yacc They're abysmal to use, have poor interfaces and poor error handling facilities. > antlr At least when I tried it years ago, it was an unapproachable mess of java tooling and editors. > parsec Is a parser combinator. This is different than a generator. > ragel Not heard about this.
- amir734jj 5y ago
  Try Antlr 4. It's more approachable.

peter-winter 5y ago

I'm the author, let me just answer this one. Spirit produces recursive descent parser, ctpg produces LR(1). So the parse speed is just much better. And the set of possible language grammars is bigger. Plus (this is just my opinion) ctpg just looks more intuitive.