3 ms·

Yes indeed. If you are doing this kind of job, reach for a parser generator framework and fuzz your program. Also go read Parse Don’t Validate https://lexi-lam

Yes indeed. If you are doing this kind of job, reach for a parser generator framework and fuzz your program. Also go read Parse Don’t Validate https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

teeheelol 2y ago
Yep. Looking at how this whole thing is pasted together, there's probably a regex engine in one of those sys files somewhere that was doing the "parsing"...
lolinder 2y ago
> reach for a parser generator framework and fuzz your program I agree to the second but disagree on the first. Parser generator frameworks produce a lot of code that is hard to read and understand and they don't necessarily do a better job of error handling than you would. A hand-written recursive descent parser will usually be more legible, will clearly line up with the grammar that you're supposed to be parsing, and will be easier to add better error handling to. Once you're aware of the risks of a bad parser you're halfway there. Write a parser with proper parsing theory in mind and in a language that forces you to handle all cases. Then fuzz the program, turn bad inputs that turn up into permanent regression tests, and write your own tests with your knowledge of the inner workings of your parser in mind. This isn't like rolling your own crypto because the alternative isn't a battle-tested open source library, it's a framework that generates a brand new library that only you will use and maintain. If you're going to end up with a bespoke library anyway, you ought to understand it well.
- fire_lake 2y ago
  You place your trust in: 1. The battle testing of the generator (e.g. lex yacc is widely used) 2. The readability of the grammar file, which enables good review You don’t (often) read the generated code, only test it. Much like you wouldn’t read the binary generated by a compiler.

teeheelol 2y ago

Yep. Looking at how this whole thing is pasted together, there's probably a regex engine in one of those sys files somewhere that was doing the "parsing"...

lolinder 2y ago

> reach for a parser generator framework and fuzz your program I agree to the second but disagree on the first. Parser generator frameworks produce a lot of code that is hard to read and understand and they don't necessarily do a better job of error handling than you would. A hand-written recursive descent parser will usually be more legible, will clearly line up with the grammar that you're supposed to be parsing, and will be easier to add better error handling to. Once you're aware of the risks of a bad parser you're halfway there. Write a parser with proper parsing theory in mind and in a language that forces you to handle all cases. Then fuzz the program, turn bad inputs that turn up into permanent regression tests, and write your own tests with your knowledge of the inner workings of your parser in mind. This isn't like rolling your own crypto because the alternative isn't a battle-tested open source library, it's a framework that generates a brand new library that only you will use and maintain. If you're going to end up with a bespoke library anyway, you ought to understand it well.

fire_lake 2y ago
You place your trust in: 1. The battle testing of the generator (e.g. lex yacc is widely used) 2. The readability of the grammar file, which enables good review You don’t (often) read the generated code, only test it. Much like you wouldn’t read the binary generated by a compiler.