3 ms·

To code generation writers: I highly recommend not having source code in the specification file and instead generate an interface that is called by the generat

• wahern 4y ago
  1) You can put language-agnostic machines in a shared file and include them from the language-specific files. 2) Ragel machines are restartable--you can suspend and resume at any arbitrary point. You can also insert your code blocks just about anywhere as well, and even control transitions that way. You can do all of this without callbacks, and for the most part the only state you have to worry about saving and restoring is a single integer variable. No other code generator library I've encountered supports these features, and it's extremely useful for writing asynchronous machines. 2b) Relatedly, Ragel supports recursion using a user-provided stack for the state variables (basically int[]). So you can suspend and resume a machine at any depth, something that wouldn't be possible if recursion relied on native function calls (some languages support async/await or fibers, but both are significantly more troublesome than how Ragel operates, especially if you want to maintain multi-language machines.) 3) Ragel can generate pure goto-based machines, among other generation modes. Combined with inline user code blocks you can write extremely efficient machines which wouldn't be possible through function pointers or similar indirection. For these and other reasons, Ragel inline generation is key to its usefulness. Moreover, it's infinitely easier to work with than Yacc, Bison, etc, which have all the downsides of intermixed code with almost none of the benefits. If Ragel is too low-level (and it often is), there are of course other options. These days I use LPeg much more than Ragel. LPeg is similarly unique in the extent to which you can intermix user code as the parser progresses. In fact, though PEGs don't support content-sensitive grammars (e.g. TLV grammars), this is actually quite trivial with LPeg and I've written a complete X.509 certificate parser as a giant PEG, which upon matching returns a tree that is already properly transformed into an idiomatic Lua data structure--as opposed to a verbose tree of nodes reflecting the literal details of the grammar. But LPeg simply isn't useful for the things Ragel shines at, and vice-versa.
  • deterministic 4y ago
    I wrote a hierarchical state machine generator years ago that did all of that by simply calling an interface when entering/leaving/staying in a state (tick). Recursively calling the interface when pushing/popping a hierarchy of states. I used it to implement a commercial game. It worked great! It did everything I needed (and more) without having to mix source code in the spec files. I also implemented a code generator that I am using at work to generate C++/Java parsers from the same (BNF style) spec file. Again without adding source code to the spec files. The generated parsers are used in production and has been for years. The generated code is super clean. With a clear separation between the generated code and the code using it. It would be easy to generate (say) Typescript as well without touching the spec files. But of course there might be scenarios that you need that I have never needed where embedding source code is necessary. I just haven’t seen any situation where that is required in my 30+ years career. I always write my own code generators by the way. It’s great fun, it’s easy, and I can make it perfectly fit what I need. And it makes other developers think I am some sort of genius because they don’t know how easy it is :)