3 ms·

What are the performance implications of something like this (and other similar ones posted here) for a really light server (let’s say a small chat server). Wou

What are the performance implications of something like this (and other similar ones posted here) for a really light server (let’s say a small chat server). Would getting down closer to the metal matter much?

jeffbee 6y ago
The performance problems of HTTP servers are architectural, not usually about things like instructions per cycle. For example common topics are whether accepting connections will tend to starve serving established connections, whether events are distributed optimally among threads or CPUs, how the server behaves if a client sends one byte per frame, very slowly, or if the client receives the response very slowly, whether the process will run out of memory if a number of events happen suddenly, etc. A web server that's actually equipped to survive as a public service will be much, much longer than 2500 lines of code.
- kragen 6y ago
  While mostly I agree with your comment, I suspect both darkhttpd and httpdito (linked above) are actually equipped to survive as a public service. httpdito is about 700 lines of code (in assembly); it handles these issues as follows: 1. Accepting connections starving established connections: this is mostly a problem for event-driven servers. httpdito runs each connection in its own child process, has a compiled-in limit of 2048 concurrent connections. Whenever it has 2048 live children, it blocks in waitpid() until at least one of them dies. Modern Linux does a fair enough job of scheduling that this seems to be sufficient in practice to keep the parent from hogging the whole machine and preventing the children from making progress, even if most of your incoming connections just close without sending an HTTP request or any bytes at all. Darkhttpd might have a problem with this or not, I haven't tried, but it seems to rely on OS ulimits to limit its number of concurrent connections. My guess is that with the default ulimit -n 1024 darkhttpd won't have any problem with this at all. 2. Whether events are distributed optimally among threads or CPUs: the real issue here is not optimality but stability, and again, this is mostly an issue for event-driven servers (or servers made of thread pools, etc.) The essential property for production is usually not that you're using the machine to within 0.1% or 10% of its capacity, but that you're not wasting 99+% of its capacity uselessly thrashing—or, at the other extreme, responding so fairly that you provide some service to every client, but not enough to actually ever complete a whole request successfully. Modern Linux does a good enough job of this that httpdito's very stupid approach of forking a new child for each request works reasonably well. darkhttpd avoids the problem in a different way by not having threads and not using multiple CPUs, which is clearly not optimal but also totally avoids any related stability concerns. 3. How the server behaves if a client sends one byte per frame, very slowly: in overload conditions, which is when this matters "to survive as a public service", Linux's TCP stack does a reasonably efficient job of consolidating the bytes until a particular server child gets a chance to run. httpdito does a super stupid O(N²) thing in these cases where it looks for the CRLFCRLF at every position in the buffer after every received chunk (potentially a single frame), but this is ameliorated not only by Linux's TCP stack but also by the fact that the request buffer size is only 1024 bytes, so in the worst case we spend 4084 instructions on an incoming packet the last time we run this loop for a given client, and only half that (2042 instructions) on average. This is still fast enough to handle somewhere in excess of 4 million packets per core and probably faster than Linux's TCP stack. The flip side of that is that by sending data slowly in this way, or by not sending any data at all, a client can monopolize all 2048 concurrent connections, preventing other clients from connecting. This is a problem that cannot be solved in TCP/IP, only ameliorated. httpdito ameliorates it with a 32-second timeout, counted from when it accepts, so in the steady state it always accepts an average of at least 64 new connections per second, regardless of how fast or slowly the requests come in. I haven't tested darkhttpd under these conditions but it has a 30-second idle timeout and a 4000-byte max request length. It kind of looks like it does the same O(N²) thing, but I've only glanced at the code. I don't know how long it allows for the client to slowloris the initial requests in one byte at a time. 4. How the server behaves if the client receives the response very slowly: I think the above covers this. However, extremely large files or extremely slow connections will whack into the 32-second timeout. At 9600 baud, for example, responses of more than about 30 kilobytes will get truncated. There's an unavoidable tradeoff between handling slow connections successfully (which requires many memory buffers), handling fast connections efficiently (which requires large memory buffers), and withstanding DoS attacks (which requires, among other things, limiting memory use). 5. Whether the process will run out of memory if a number of events happen suddenly: httpdito does all of its within-process memory allocation at compile time, so this isn't a thing that can happen within a process. Its limit of 2048 children works out to about 16 mebibytes of RAM typically, so it's not a problem systemwide either, unless you're running it on a Rio Receiver or something. darkhttpd does allocate memory, but at first glance it seems to be limited to a couple of allocations of less than 4000 bytes per concurrent connection, so I don't see a problem here either. It's common for event-loop servers to handle overload events like these much more gracefully than multitasking servers; httpdito is kind of an outlier here because it uses about a thousandth the RAM of a normal multitasking server, and running on an OS whose task scheduling is less shitty than is typical, which allows it to get away with a lot. (The original slowloris README describes it as a DoS tool for threaded web servers.) By contrast, the usual overload problem event-loop systems have trouble with is where some event handler that usually takes a millisecond or a microsecond suddenly starts taking a second or a kilosecond, locking out everything else until it's done. Maybe you have an O(N³) algorithm in your HTTP content negotiation, but you never noticed because normal browsers send an Accept: header with five content types and so N³ means 125 iterations, and then some guy in Turkey with a screwed up browser configuration sends you an Accept: header with 1296 content-types and your whole server freezes for several seconds. Every three minutes. You could hypothetically make an event-loop system immune to this kind of thing by doing a rigorous WCET analysis on every single event handler, as if it were jet engine control firmware. But that's a huge pain in the ass for any program of more than about 1000 machine instructions, so I've never heard of anybody doing this with an event-loop-driven network server. I've been toying with ideas for solving this problem over the years, including approaches like BPF and software transactional memory. Some of my notes on this are at https://dercuano.github.io/topics/transactions.html https://dercuano.github.io/topics/transactions.html, while others are in Derctuo and Dernocua. That said, I haven't tested darkhttpd at all, and I'm not running any public services on httpdito. It wouldn't surprise me if either or both had some kind of stability bug under overload conditions. But it would surprise me very much if fixing those bugs required adding thousands of lines of code.
  - jart 6y ago
    I can see you've spent a lot of time thinking about things like slowloris. What do you think of the approach that redbean takes? https://justine.lol/redbean/index.html https://justine.lol/redbean/index.html It still needs more unit tests, but basically what it has is a meltdown mode so that when fork() starts to fail it sends SIGUSR2 to the process group to EINTR recv() calls so that lingering processes can exit gracefully. Its parser also isn't cubic, so even for a fragmented messages, it only needs to consider each character a single time: https://github.com/jart/cosmopolitan/blob/master/net/http/parsehttprequest.c https://github.com/jart/cosmopolitan/blob/master/net/http/pa...
  - kragen 6y ago
    Hmm, I was just looking at httpdito and I see that I was wrong about the timeout when it's sending: it disables the timeout once it starts sending the response, on the theory that sooner or later TCP will time out if you stop acknowledging. Well, it will, but it might take a lot longer than 32 seconds. And I don't know for sure but I suspect you can hold open a TCP connection forever by periodically resending the same ACK packet, which would mean you could totally DoS httpdito just by requesting some large web page and then never reading any of it, 2048 times. It would probably need to be big enough that it won't fit in the kernel's TCP send buffer in order to keep the child process from exiting. At least that "solves" the 9600 baud problem, I guess. I don't think that's the kind of thing that's likely to happen by accident, but it's probably an easy enough attack to mount intentionally.
kragen 6y ago
darkhttpd and things like `python -m SimpleHTTPServer` only support static content, so you can't run a chat server on them. A lot depends on your context. I haven't benchmarked darkhttpd, but it's probably significantly better than my httpdito (linked above), which can spew out 1.8 gigabits per second and at least 20000 requests per second on my 8-year-old 4-core 2.3-GHz amd64 laptop. An event-driven server like darkhttpd would be a much better basis for adding Comet functionality like chat—that's why, at KnowNow in 02000, we contracted Robert Thau to move our Comet functionality from Apache with Perl CGI (IIRC ≈64 concurrent connections and ≈64 chat messages per second on a 1GHz server with 1GiB of RAM) to his select()-driven thttpd (≈8192). This work, now mostly of archaeological interest, has been open-sourced as mod_pubsub, which also includes a compatible select()-driven Python server I wrote the next year. It was also able to handle thousands of messages per second and thousands of concurrent clients. There are at least four axes to your problem: - How many concurrent connections do you mean when you say "a small chat server"? This could be anywhere from 2 to 2048. - How many messages per second is it processing? This could be anywhere from 128 per connection (for something like Mumble) to 1/2048 per connection (if almost all clients are idle almost all the time). - How big are these messages on average? This could be anywhere from 64 bytes to 64 mebibytes. - What kind of hardware are you running it on? This could be anything from a 1MHz Commodore PET with an 8-bit 6502 and 16KiB of RAM to a 3.7GHz 12-core Ryzen 9 5900X with 256GiB of RAM. (I don't think anyone's written a chat server on Contiki, but it would be easy to do. It just wouldn't scale to very high loads.) So, the answer to your question may vary by a factor of about 2⁷³, 22 decimal orders of magnitude. Can you give more detail on what you're thinking about?
- deleted 6y ago
  [deleted]

jeffbee 6y ago

The performance problems of HTTP servers are architectural, not usually about things like instructions per cycle. For example common topics are whether accepting connections will tend to starve serving established connections, whether events are distributed optimally among threads or CPUs, how the server behaves if a client sends one byte per frame, very slowly, or if the client receives the response very slowly, whether the process will run out of memory if a number of events happen suddenly, etc. A web server that's actually equipped to survive as a public service will be much, much longer than 2500 lines of code.

kragen 6y ago
While mostly I agree with your comment, I suspect both darkhttpd and httpdito (linked above) are actually equipped to survive as a public service. httpdito is about 700 lines of code (in assembly); it handles these issues as follows: 1. Accepting connections starving established connections: this is mostly a problem for event-driven servers. httpdito runs each connection in its own child process, has a compiled-in limit of 2048 concurrent connections. Whenever it has 2048 live children, it blocks in waitpid() until at least one of them dies. Modern Linux does a fair enough job of scheduling that this seems to be sufficient in practice to keep the parent from hogging the whole machine and preventing the children from making progress, even if most of your incoming connections just close without sending an HTTP request or any bytes at all. Darkhttpd might have a problem with this or not, I haven't tried, but it seems to rely on OS ulimits to limit its number of concurrent connections. My guess is that with the default ulimit -n 1024 darkhttpd won't have any problem with this at all. 2. Whether events are distributed optimally among threads or CPUs: the real issue here is not optimality but stability, and again, this is mostly an issue for event-driven servers (or servers made of thread pools, etc.) The essential property for production is usually not that you're using the machine to within 0.1% or 10% of its capacity, but that you're not wasting 99+% of its capacity uselessly thrashing—or, at the other extreme, responding so fairly that you provide some service to every client, but not enough to actually ever complete a whole request successfully. Modern Linux does a good enough job of this that httpdito's very stupid approach of forking a new child for each request works reasonably well. darkhttpd avoids the problem in a different way by not having threads and not using multiple CPUs, which is clearly not optimal but also totally avoids any related stability concerns. 3. How the server behaves if a client sends one byte per frame, very slowly: in overload conditions, which is when this matters "to survive as a public service", Linux's TCP stack does a reasonably efficient job of consolidating the bytes until a particular server child gets a chance to run. httpdito does a super stupid O(N²) thing in these cases where it looks for the CRLFCRLF at every position in the buffer after every received chunk (potentially a single frame), but this is ameliorated not only by Linux's TCP stack but also by the fact that the request buffer size is only 1024 bytes, so in the worst case we spend 4084 instructions on an incoming packet the last time we run this loop for a given client, and only half that (2042 instructions) on average. This is still fast enough to handle somewhere in excess of 4 million packets per core and probably faster than Linux's TCP stack. The flip side of that is that by sending data slowly in this way, or by not sending any data at all, a client can monopolize all 2048 concurrent connections, preventing other clients from connecting. This is a problem that cannot be solved in TCP/IP, only ameliorated. httpdito ameliorates it with a 32-second timeout, counted from when it accepts, so in the steady state it always accepts an average of at least 64 new connections per second, regardless of how fast or slowly the requests come in. I haven't tested darkhttpd under these conditions but it has a 30-second idle timeout and a 4000-byte max request length. It kind of looks like it does the same O(N²) thing, but I've only glanced at the code. I don't know how long it allows for the client to slowloris the initial requests in one byte at a time. 4. How the server behaves if the client receives the response very slowly: I think the above covers this. However, extremely large files or extremely slow connections will whack into the 32-second timeout. At 9600 baud, for example, responses of more than about 30 kilobytes will get truncated. There's an unavoidable tradeoff between handling slow connections successfully (which requires many memory buffers), handling fast connections efficiently (which requires large memory buffers), and withstanding DoS attacks (which requires, among other things, limiting memory use). 5. Whether the process will run out of memory if a number of events happen suddenly: httpdito does all of its within-process memory allocation at compile time, so this isn't a thing that can happen within a process. Its limit of 2048 children works out to about 16 mebibytes of RAM typically, so it's not a problem systemwide either, unless you're running it on a Rio Receiver or something. darkhttpd does allocate memory, but at first glance it seems to be limited to a couple of allocations of less than 4000 bytes per concurrent connection, so I don't see a problem here either. It's common for event-loop servers to handle overload events like these much more gracefully than multitasking servers; httpdito is kind of an outlier here because it uses about a thousandth the RAM of a normal multitasking server, and running on an OS whose task scheduling is less shitty than is typical, which allows it to get away with a lot. (The original slowloris README describes it as a DoS tool for threaded web servers.) By contrast, the usual overload problem event-loop systems have trouble with is where some event handler that usually takes a millisecond or a microsecond suddenly starts taking a second or a kilosecond, locking out everything else until it's done. Maybe you have an O(N³) algorithm in your HTTP content negotiation, but you never noticed because normal browsers send an Accept: header with five content types and so N³ means 125 iterations, and then some guy in Turkey with a screwed up browser configuration sends you an Accept: header with 1296 content-types and your whole server freezes for several seconds. Every three minutes. You could hypothetically make an event-loop system immune to this kind of thing by doing a rigorous WCET analysis on every single event handler, as if it were jet engine control firmware. But that's a huge pain in the ass for any program of more than about 1000 machine instructions, so I've never heard of anybody doing this with an event-loop-driven network server. I've been toying with ideas for solving this problem over the years, including approaches like BPF and software transactional memory. Some of my notes on this are at https://dercuano.github.io/topics/transactions.html https://dercuano.github.io/topics/transactions.html, while others are in Derctuo and Dernocua. That said, I haven't tested darkhttpd at all, and I'm not running any public services on httpdito. It wouldn't surprise me if either or both had some kind of stability bug under overload conditions. But it would surprise me very much if fixing those bugs required adding thousands of lines of code.
- jart 6y ago
  I can see you've spent a lot of time thinking about things like slowloris. What do you think of the approach that redbean takes? https://justine.lol/redbean/index.html https://justine.lol/redbean/index.html It still needs more unit tests, but basically what it has is a meltdown mode so that when fork() starts to fail it sends SIGUSR2 to the process group to EINTR recv() calls so that lingering processes can exit gracefully. Its parser also isn't cubic, so even for a fragmented messages, it only needs to consider each character a single time: https://github.com/jart/cosmopolitan/blob/master/net/http/parsehttprequest.c https://github.com/jart/cosmopolitan/blob/master/net/http/pa...
- kragen 6y ago
  Hmm, I was just looking at httpdito and I see that I was wrong about the timeout when it's sending: it disables the timeout once it starts sending the response, on the theory that sooner or later TCP will time out if you stop acknowledging. Well, it will, but it might take a lot longer than 32 seconds. And I don't know for sure but I suspect you can hold open a TCP connection forever by periodically resending the same ACK packet, which would mean you could totally DoS httpdito just by requesting some large web page and then never reading any of it, 2048 times. It would probably need to be big enough that it won't fit in the kernel's TCP send buffer in order to keep the child process from exiting. At least that "solves" the 9600 baud problem, I guess. I don't think that's the kind of thing that's likely to happen by accident, but it's probably an easy enough attack to mount intentionally.

kragen 6y ago

darkhttpd and things like `python -m SimpleHTTPServer` only support static content, so you can't run a chat server on them. A lot depends on your context. I haven't benchmarked darkhttpd, but it's probably significantly better than my httpdito (linked above), which can spew out 1.8 gigabits per second and at least 20000 requests per second on my 8-year-old 4-core 2.3-GHz amd64 laptop. An event-driven server like darkhttpd would be a much better basis for adding Comet functionality like chat—that's why, at KnowNow in 02000, we contracted Robert Thau to move our Comet functionality from Apache with Perl CGI (IIRC ≈64 concurrent connections and ≈64 chat messages per second on a 1GHz server with 1GiB of RAM) to his select()-driven thttpd (≈8192). This work, now mostly of archaeological interest, has been open-sourced as mod_pubsub, which also includes a compatible select()-driven Python server I wrote the next year. It was also able to handle thousands of messages per second and thousands of concurrent clients. There are at least four axes to your problem: - How many concurrent connections do you mean when you say "a small chat server"? This could be anywhere from 2 to 2048. - How many messages per second is it processing? This could be anywhere from 128 per connection (for something like Mumble) to 1/2048 per connection (if almost all clients are idle almost all the time). - How big are these messages on average? This could be anywhere from 64 bytes to 64 mebibytes. - What kind of hardware are you running it on? This could be anything from a 1MHz Commodore PET with an 8-bit 6502 and 16KiB of RAM to a 3.7GHz 12-core Ryzen 9 5900X with 256GiB of RAM. (I don't think anyone's written a chat server on Contiki, but it would be easy to do. It just wouldn't scale to very high loads.) So, the answer to your question may vary by a factor of about 2⁷³, 22 decimal orders of magnitude. Can you give more detail on what you're thinking about?

deleted 6y ago
[deleted]