4 ms·

> I made one of these as a filter written in C using Lex. What could possibly go wrong? Having cut my CS teeth on both I must say both C and Lex are giant foo

> I made one of these as a filter written in C using Lex. What could possibly go wrong? Having cut my CS teeth on both I must say both C and Lex are giant foot-cannons if used to parse untrusted input.

kazinator 5y ago
A number of things could go wrong. You have to look at the actual program, rather than making assumptions about it based on the language it uses. Flex-generated code could be a risk. But in many years, I've not seen problems with Flex scanners. TXR uses a complicated Flex scanner. In almost twelve years of developing this program, I've never had to debug issues with Flex-generated code and have never seen valgrind report any problems against that code, so I'm confident in deploying a simple Flex-generated scanner program for untrusted data. Though Flex generates C, it isn't C. When you use Flex, you're not rolling your own scanner in C; you're relying on a widely used tool. There are parts of the Lex input file which are C, but in my program, all they do is return integer constants. Some of them also use BEGIN to switch to a different lex state. My Lex file is very low risk. I didn't customize the input buffering or anything. I also de-risked the rest of the program substantially by defining a token structure which is passed and returned by value. There is almost no pointer manipulation or arithmetic. There is only a small bit of manual memory management: I used strdup to copy the token text, subject to manual freeing. When a function returns a token to another, the caller becomes the owner; it calls deltok if it consumes the token (doesn't return it to anyone). The ownership protocol is very easy to verify. The program carefully accesses static arrays which represent the whitelist, and doesn't perform any arithmetic that could overflow. Now, on the other hand, delegating HTML cleaning to browsers: what could go wrong there? Unsafe HTML should probably not be served to browsers, period; unsafe data should not escape from the back end. If you can catch bad HTML at the back end, why wouldn't you?

kazinator 5y ago

A number of things could go wrong. You have to look at the actual program, rather than making assumptions about it based on the language it uses. Flex-generated code could be a risk. But in many years, I've not seen problems with Flex scanners. TXR uses a complicated Flex scanner. In almost twelve years of developing this program, I've never had to debug issues with Flex-generated code and have never seen valgrind report any problems against that code, so I'm confident in deploying a simple Flex-generated scanner program for untrusted data. Though Flex generates C, it isn't C. When you use Flex, you're not rolling your own scanner in C; you're relying on a widely used tool. There are parts of the Lex input file which are C, but in my program, all they do is return integer constants. Some of them also use BEGIN to switch to a different lex state. My Lex file is very low risk. I didn't customize the input buffering or anything. I also de-risked the rest of the program substantially by defining a token structure which is passed and returned by value. There is almost no pointer manipulation or arithmetic. There is only a small bit of manual memory management: I used strdup to copy the token text, subject to manual freeing. When a function returns a token to another, the caller becomes the owner; it calls deltok if it consumes the token (doesn't return it to anyone). The ownership protocol is very easy to verify. The program carefully accesses static arrays which represent the whitelist, and doesn't perform any arithmetic that could overflow. Now, on the other hand, delegating HTML cleaning to browsers: what could go wrong there? Unsafe HTML should probably not be served to browsers, period; unsafe data should not escape from the back end. If you can catch bad HTML at the back end, why wouldn't you?