5 ms·
Has anyone done a comparison of memory footprints for parsers? Say, Antlr vs Bison vs ???. I used Antlr3 to create a NLP parser for entering a calendar event.
by musesum 11y ago
Has anyone done a comparison of memory footprints for parsers? Say, Antlr vs Bison vs ???.
I used Antlr3 to create a NLP parser for entering a calendar event. For iOS, the C runtime's parse tree source was many megabytes and difficult to debug directly. For Android, the Java runtime parse tree ran out of memory and had to be broken into smaller pieces. Haven't tried Antlr4.
Current workaround is to parse a document directly into a token tree. Instead of separate lever + parser, merged a tweaked BNF with regex. Memory stays rather small and can download a revised grammar without recompiling a new binary.
- Terr_ 11y agoAre you sure the issue was with the tool and not the language/grammar?
- musesum 11y agoWas probably a bit of both. Using a LL parser for NLP is an odd fit. The problem was that the n-gram parsers had a fairly large file footprint. So, paging from memory would be slow and consume battery. As for tool, the Antlr3 C runtime was very fiddly. On the forums, I saw one developer give up on porting their working Antlr2 C++ runtime to Antler's C runtime. This may have been due to C lacking exception handing, so had to implement backtracking another way. (Still, kudos to the volunteer who implemented the C runtime; in our case, it did make it into production.) For really simple NLP parsing, one hack is combine an Island parser with an LL parser. This is essentially what Siri used several years ago (have no idea what it uses now).