9 ms·
This is great guys. It's surprising that you're able to get an 8x-20x speedup over email.parser with another pure python parser. (What does this say about email
by sigil 13y ago
This is great guys. It's surprising that you're able to get an 8x-20x speedup over email.parser with another pure python parser. (What does this say about email.parser, eh?)
Did you consider using a C extension and tools like lex or ragel for the rfc822 scanning phase? Not worth it?
- lifeisstillgood 13y agoI think it says that email.parser reads the whole message then parses. Flanker can just read the headers and ignore the message, so I guess with the right configuration and test data you get the 20x speed up. It's pretty unlikely that many well-used and years old OSS language libraries have orders of magnitude speed ups lying on the table, without there being a design decision in there that can be argued many different ways.
- sigil 13y agoBut Flanker's full-message parser is still 8x faster than email.parser's full-message parser, according to their benchmark. It's pretty unlikely that many well-used and years old OSS language libraries have orders of magnitude speed ups lying on the table... Normally I'd agree with you, but this is the python stdlib we're talking about.
- lifeisstillgood 13y agoI missed that. But 20 down to 8 is still a headline vs a benchmark, and benchmarks can be ... swayed based on who is benchmarking. But I think my point elsewhere stands - I hardly want to parse email these days, even though I consume more of it. Most of my mail is outsourced and it seems to be a trend. Maybe not a good one. I am not sure. I used to hate spamassassin though