4 ms·
If you need to parse a file that is more complex than what regex can handle, what does one /generally/ do? I'm doing some analysis in R, and the data is genera
by wodenokoto 8y ago
If you need to parse a file that is more complex than what regex can handle, what does one /generally/ do?
I'm doing some analysis in R, and the data is generated by a program that more or less just spit things out.
Its values are comma separated, but if there are 2 values on one line, it is column 1 and 10. If the first character on a line is a comma, it is not a separator but a value. Commas inside quotations or escaped with backslashes are also values.
The list of special cases and oddities is pretty long. Right now I have a large switch statement that applies different search and replace on all sorts of conditions before feeding the CSV to read_csv.
Is this a case for owl? Can I use it within R or Python?
- jacoblambda 8y agoFrom what I can tell, your best bet would be to build the generated parser (which will generate some C code) and then build a small interface in C. From there I would just use R/Python's foreign function interface to call your C wrapper. Alternatively for python you could use a parse combinator like parsec. Depending on how complicated your file can get, using parsec could very likely be the better choice as it has first class support in python and seems powerful enough to handle your use case. There do seem to be a number of parse combinators in R but none of them seem as well established as parsec for python (which is based on the much more well known parsec for haskell). A quick look around seems to show that python will be your best bet for parsing as it has some decent tooling. Here is an article on some of the different parse generators/parse combinators in the space. https://tomassetti.me/parsing-in-python/ https://tomassetti.me/parsing-in-python/