4 ms·

> However, the performance bottleneck in aiohttp turned out to be its HTTP parser, which is so slow, that it matters very little how fast the underlying I/O lib

> However, the performance bottleneck in aiohttp turned out to be its HTTP parser, which is so slow, that it matters very little how fast the underlying I/O library is. This is exactly the same observation that motivated the Mongrel web server for Ruby, 10 years ago this year. "Mongrel is a small library that provides a very fast HTTP 1.1 server for Ruby web applications. [...] What makes Mongrel so fast is the careful use of an Ragel [C] extension to provide fast, accurate HTTP 1.1 protocol parsing. This makes the server scream without too many portability issues." -- https://github.com/mongrel/mongrel https://github.com/mongrel/mongrel And its successor Thin: "Thin is a Ruby web server that glues together 3 of the best Ruby libraries in web history: (1) the Mongrel parser, the root of Mongrel speed and security" -- http://code.macournoyer.com/thin/ http://code.macournoyer.com/thin/ In case anyone is still wondering, parsing in Ruby/Python/Lua is pretty slow compared to C/C++. That's why I personally have been really interested for a long time in writing parsers in C that can be used from higher level languages. That way you can get the best of both worlds.

coderdude 11y ago
It's the best of both worlds but doesn't shield you from the worst of one of the worlds. Untrusted input is still reaching code that has direct access to system memory. Hopefully not, anyway. But probably. Still, it's the way to go if performance is key.
- haberman 11y ago
  Very true. Thankfully fuzzing tools are getting better all the time. LLVM's libfuzzer is great.
  - coderdude 11y ago
    Absolutely it's a positive that this is an active area of research. One day these types of problems are going to be something future programmers joke about. Or don't even know existed.
  - ericfrederich 11y ago
    I'd recommend to watch this: https://www.youtube.com/watch?v=y0hyqzR6hIY https://www.youtube.com/watch?v=y0hyqzR6hIY He goes into detail about how these types of libraries are particularly difficult or impossible to fuzz. He uses OpenSSL as an example but I would imagine an http library being similar.
    - haberman 11y ago
      > He goes into detail about how these types of libraries are particularly difficult or impossible to fuzz. I just watched the video on 2x and I don't think that's a fair summary. He seems positive on fuzzing in general and mentions that fuzzing found two extremely tricky bugs in libsndfile and flac. He does point out that there are some cases like OpenSSL that are particularly difficult to fuzz completely because they are encrypted and heavily stateful, creating transient keys on the fly and such. I don't think HTTP has this problem, for the most part. Cool to see a video of Erik de Castro Lopo though -- I've worked with that guy since the early 2000s when I was working on Audacity (which uses his excellend libsndfile internally -- or at least did at the time).
- jgraham 11y ago
  These days you would probably want to write the parser part in Rust, with a small amount of unsafe code to implement a C-compatible API that could then be called from Python, or wherever. I did this for some regexp-based log parsing code written in Python, and saw a considerable (2-3x) performance win. The main outstanding issue is that Rust isn't as easy to distribute as C to random end users (e.g. it likely requires the user to have rustc installed for |pip install| to work, which is unlikely and not always possible through standard package managers).
  - edraferi 11y ago
    Doesn't Conda help with that problem, allowing you to precompile rust extensions just like C extensions?
mangeletti 11y ago
This is also what inspired the uwsgi[1] protocol in the uWSGI project[2]. 1. https://uwsgi-docs.readthedocs.io/en/latest/Protocol.html https://uwsgi-docs.readthedocs.io/en/latest/Protocol.html 2. https://uwsgi-docs.readthedocs.io/en/latest/ https://uwsgi-docs.readthedocs.io/en/latest/
sigil 11y ago
Check out http-parser for Python: https://pypi.python.org/pypi/http-parser https://pypi.python.org/pypi/http-parser This uses Dahl's original http_parser.c FSM. With a little work you can write a WSGI handler around it. Highly recommend.
cesarbs 11y ago
On a related note, we go through great lengths in the Kestrel HTTP server [0] (which also uses libuv) to have fast HTTP parsing. As an example, we attempt to read the method and the HTTP version as longs and compare them to pre-computed longs in order to have fast comparisons and reuse strings containing standard methods and versions (reducing memory allocation is the main driver of our optimizations) [1], so we don't have to allocate those strings on every request. We do a similar thing for headers [2]. We also manage a lot of our own memory, despite using a garbage collected language [3]. [0] https://github.com/aspnet/KestrelHttpServer https://github.com/aspnet/KestrelHttpServer [1] https://github.com/aspnet/KestrelHttpServer/blob/3a424f6abac6a019da4d00c32e98880c0f99e8d7/src/Microsoft.AspNetCore.Server.Kestrel/Infrastructure/MemoryPoolIteratorExtensions.cs#L295 https://github.com/aspnet/KestrelHttpServer/blob/3a424f6abac... [2] https://github.com/aspnet/KestrelHttpServer/blob/dev/src/Microsoft.AspNetCore.Server.Kestrel/Http/FrameHeaders.Generated.cs https://github.com/aspnet/KestrelHttpServer/blob/dev/src/Mic... [3] https://github.com/aspnet/KestrelHttpServer/blob/dev/src/Microsoft.AspNetCore.Server.Kestrel/Infrastructure/MemoryPool.cs https://github.com/aspnet/KestrelHttpServer/blob/dev/src/Mic...
- 1st1 11y ago
  Have you benchmarked it?
igravious 11y ago
How much of a parsing stack can you build atop of Ragel? I gave Ragel a good look over recently when checking out parsers for Ruby because Ragel has a Ruby binding. I chose A Ruby PEG† library (Parslet) because I decided that there was too much of a gap between the low level finite automata machinery that Ragel provides and the parsing generation that I was looking for. Was I wrong to decide that, I wonder. † On a related note, I'm finding it difficult finding (fast) GLR or GLL parser generators for Ruby
- haberman 11y ago
  I think you're right: the gap between Ragel and what you want in higher-level languages is too wide to make a stack out of it IMO. I'm working on what I consider a parsing stack. The key observation, in my opinion, is that you need a way of representing structured data in the high-level language that is both rich and efficient. I think Protocol Buffers can serve that role. Once you have the structured data representation, you can write parsers to read/write it. But you want the parsers to be independent of how you represent the structured data in any one language.
  - igravious 11y ago
    I've spent the last little while browsing through the upb and gazelle repos. I'm struggling to get a high-level overview of what you want to do. I'm curious though because what you're saying here on HN intrigues me.

coderdude 11y ago

It's the best of both worlds but doesn't shield you from the worst of one of the worlds. Untrusted input is still reaching code that has direct access to system memory. Hopefully not, anyway. But probably. Still, it's the way to go if performance is key.

haberman 11y ago
Very true. Thankfully fuzzing tools are getting better all the time. LLVM's libfuzzer is great.
- coderdude 11y ago
  Absolutely it's a positive that this is an active area of research. One day these types of problems are going to be something future programmers joke about. Or don't even know existed.
- ericfrederich 11y ago
  I'd recommend to watch this: https://www.youtube.com/watch?v=y0hyqzR6hIY https://www.youtube.com/watch?v=y0hyqzR6hIY He goes into detail about how these types of libraries are particularly difficult or impossible to fuzz. He uses OpenSSL as an example but I would imagine an http library being similar.
  - haberman 11y ago
    > He goes into detail about how these types of libraries are particularly difficult or impossible to fuzz. I just watched the video on 2x and I don't think that's a fair summary. He seems positive on fuzzing in general and mentions that fuzzing found two extremely tricky bugs in libsndfile and flac. He does point out that there are some cases like OpenSSL that are particularly difficult to fuzz completely because they are encrypted and heavily stateful, creating transient keys on the fly and such. I don't think HTTP has this problem, for the most part. Cool to see a video of Erik de Castro Lopo though -- I've worked with that guy since the early 2000s when I was working on Audacity (which uses his excellend libsndfile internally -- or at least did at the time).
jgraham 11y ago
These days you would probably want to write the parser part in Rust, with a small amount of unsafe code to implement a C-compatible API that could then be called from Python, or wherever. I did this for some regexp-based log parsing code written in Python, and saw a considerable (2-3x) performance win. The main outstanding issue is that Rust isn't as easy to distribute as C to random end users (e.g. it likely requires the user to have rustc installed for |pip install| to work, which is unlikely and not always possible through standard package managers).
- edraferi 11y ago
  Doesn't Conda help with that problem, allowing you to precompile rust extensions just like C extensions?

mangeletti 11y ago

This is also what inspired the uwsgi[1] protocol in the uWSGI project[2]. 1. https://uwsgi-docs.readthedocs.io/en/latest/Protocol.html https://uwsgi-docs.readthedocs.io/en/latest/Protocol.html 2. https://uwsgi-docs.readthedocs.io/en/latest/ https://uwsgi-docs.readthedocs.io/en/latest/

sigil 11y ago

Check out http-parser for Python: https://pypi.python.org/pypi/http-parser https://pypi.python.org/pypi/http-parser This uses Dahl's original http_parser.c FSM. With a little work you can write a WSGI handler around it. Highly recommend.

cesarbs 11y ago

On a related note, we go through great lengths in the Kestrel HTTP server [0] (which also uses libuv) to have fast HTTP parsing. As an example, we attempt to read the method and the HTTP version as longs and compare them to pre-computed longs in order to have fast comparisons and reuse strings containing standard methods and versions (reducing memory allocation is the main driver of our optimizations) [1], so we don't have to allocate those strings on every request. We do a similar thing for headers [2]. We also manage a lot of our own memory, despite using a garbage collected language [3]. [0] https://github.com/aspnet/KestrelHttpServer https://github.com/aspnet/KestrelHttpServer [1] https://github.com/aspnet/KestrelHttpServer/blob/3a424f6abac6a019da4d00c32e98880c0f99e8d7/src/Microsoft.AspNetCore.Server.Kestrel/Infrastructure/MemoryPoolIteratorExtensions.cs#L295 https://github.com/aspnet/KestrelHttpServer/blob/3a424f6abac... [2] https://github.com/aspnet/KestrelHttpServer/blob/dev/src/Microsoft.AspNetCore.Server.Kestrel/Http/FrameHeaders.Generated.cs https://github.com/aspnet/KestrelHttpServer/blob/dev/src/Mic... [3] https://github.com/aspnet/KestrelHttpServer/blob/dev/src/Microsoft.AspNetCore.Server.Kestrel/Infrastructure/MemoryPool.cs https://github.com/aspnet/KestrelHttpServer/blob/dev/src/Mic...

1st1 11y ago
Have you benchmarked it?

igravious 11y ago

How much of a parsing stack can you build atop of Ragel? I gave Ragel a good look over recently when checking out parsers for Ruby because Ragel has a Ruby binding. I chose A Ruby PEG† library (Parslet) because I decided that there was too much of a gap between the low level finite automata machinery that Ragel provides and the parsing generation that I was looking for. Was I wrong to decide that, I wonder. † On a related note, I'm finding it difficult finding (fast) GLR or GLL parser generators for Ruby

haberman 11y ago
I think you're right: the gap between Ragel and what you want in higher-level languages is too wide to make a stack out of it IMO. I'm working on what I consider a parsing stack. The key observation, in my opinion, is that you need a way of representing structured data in the high-level language that is both rich and efficient. I think Protocol Buffers can serve that role. Once you have the structured data representation, you can write parsers to read/write it. But you want the parsers to be independent of how you represent the structured data in any one language.
- igravious 11y ago
  I've spent the last little while browsing through the upb and gazelle repos. I'm struggling to get a high-level overview of what you want to do. I'm curious though because what you're saying here on HN intrigues me.