4 ms·
A couple of years ago I worked on a project that involved parsing the very idiosyncratic output of a lot of different command-line commands. Not standard or sem
by paulrpotts 4y ago
A couple of years ago I worked on a project that involved parsing the very idiosyncratic output of a lot of different command-line commands. Not standard or semi-standardized commands like cat, ls, etc. - a vendor-written command line interface running on a microcontroller, for testing a board. Because apparently a number of different engineers worked on it, all the commands had "ad hoc" output formats, with very little commonality in the way the output was formatted, so there was not a lot of opportunity to write common handlers. I wound up using a lot of Python regular expressions that captured specific fields in the output, handling IIRC about forty different commands.
The best I could do was to break the regexes down across multiple lines and comment each line, and break the handlers down into a hierarchy of smaller functions. Each function had a number of sample strings right in the code, and each file of code for handling the variations on output of a single command also had a unit test right in the same file which fed all the samples to the regexes. So the handling of the 40 different commands could be unit-tested very quickly with a single pytest command line. This was a godsend to help keep my change things-test things-change things again "loop" very quick.
The alternative might have been to actually write parsers, but I don't think that would have been more readable and it certainly wouldn't have been as quick.
I don't think language designers should get rid of regexes and I don't think programmers should stop using them. I do think there is definitely room for languages that implement regexes with more syntax. People might argue that this is just needless syntactic sugar, but so is spelling out keywords like "else." We ought to be looking at prior art. The history of programming languages is a very deep trashpile containing a lot of gold that can be mined with a little effort. SNOBOL's syntax for pattern matching was maybe too wordy by modern standards but I'm sure a lot of younger programmers would be pretty surprised to find how early a lot of things like regexes were invented, and how inspirational it can be to look at old languages.