3 ms·

It's the best of both worlds but doesn't shield you from the worst of one of the worlds. Untrusted input is still reaching code that has direct access to system

It's the best of both worlds but doesn't shield you from the worst of one of the worlds. Untrusted input is still reaching code that has direct access to system memory. Hopefully not, anyway. But probably. Still, it's the way to go if performance is key.

haberman 10y ago
Very true. Thankfully fuzzing tools are getting better all the time. LLVM's libfuzzer is great.
- coderdude 10y ago
  Absolutely it's a positive that this is an active area of research. One day these types of problems are going to be something future programmers joke about. Or don't even know existed.
- ericfrederich 10y ago
  I'd recommend to watch this: https://www.youtube.com/watch?v=y0hyqzR6hIY https://www.youtube.com/watch?v=y0hyqzR6hIY He goes into detail about how these types of libraries are particularly difficult or impossible to fuzz. He uses OpenSSL as an example but I would imagine an http library being similar.
  - haberman 10y ago
    > He goes into detail about how these types of libraries are particularly difficult or impossible to fuzz. I just watched the video on 2x and I don't think that's a fair summary. He seems positive on fuzzing in general and mentions that fuzzing found two extremely tricky bugs in libsndfile and flac. He does point out that there are some cases like OpenSSL that are particularly difficult to fuzz completely because they are encrypted and heavily stateful, creating transient keys on the fly and such. I don't think HTTP has this problem, for the most part. Cool to see a video of Erik de Castro Lopo though -- I've worked with that guy since the early 2000s when I was working on Audacity (which uses his excellend libsndfile internally -- or at least did at the time).
jgraham 10y ago
These days you would probably want to write the parser part in Rust, with a small amount of unsafe code to implement a C-compatible API that could then be called from Python, or wherever. I did this for some regexp-based log parsing code written in Python, and saw a considerable (2-3x) performance win. The main outstanding issue is that Rust isn't as easy to distribute as C to random end users (e.g. it likely requires the user to have rustc installed for |pip install| to work, which is unlikely and not always possible through standard package managers).
- edraferi 10y ago
  Doesn't Conda help with that problem, allowing you to precompile rust extensions just like C extensions?

haberman 10y ago

Very true. Thankfully fuzzing tools are getting better all the time. LLVM's libfuzzer is great.

coderdude 10y ago
Absolutely it's a positive that this is an active area of research. One day these types of problems are going to be something future programmers joke about. Or don't even know existed.
ericfrederich 10y ago
I'd recommend to watch this: https://www.youtube.com/watch?v=y0hyqzR6hIY https://www.youtube.com/watch?v=y0hyqzR6hIY He goes into detail about how these types of libraries are particularly difficult or impossible to fuzz. He uses OpenSSL as an example but I would imagine an http library being similar.
- haberman 10y ago
  > He goes into detail about how these types of libraries are particularly difficult or impossible to fuzz. I just watched the video on 2x and I don't think that's a fair summary. He seems positive on fuzzing in general and mentions that fuzzing found two extremely tricky bugs in libsndfile and flac. He does point out that there are some cases like OpenSSL that are particularly difficult to fuzz completely because they are encrypted and heavily stateful, creating transient keys on the fly and such. I don't think HTTP has this problem, for the most part. Cool to see a video of Erik de Castro Lopo though -- I've worked with that guy since the early 2000s when I was working on Audacity (which uses his excellend libsndfile internally -- or at least did at the time).

jgraham 10y ago

These days you would probably want to write the parser part in Rust, with a small amount of unsafe code to implement a C-compatible API that could then be called from Python, or wherever. I did this for some regexp-based log parsing code written in Python, and saw a considerable (2-3x) performance win. The main outstanding issue is that Rust isn't as easy to distribute as C to random end users (e.g. it likely requires the user to have rustc installed for |pip install| to work, which is unlikely and not always possible through standard package managers).

edraferi 10y ago
Doesn't Conda help with that problem, allowing you to precompile rust extensions just like C extensions?