3 ms·

No it doesn't gets serialized just by using TCP, the multiplexed streams are still handled concurrently, better utilizing the waits that come from IO heavy work

No it doesn't gets serialized just by using TCP, the multiplexed streams are still handled concurrently, better utilizing the waits that come from IO heavy workloads. Doing the same on top of http 1.1 can work but it'd be just rebuilding http 2 with more overhead; or use multiple http 1.1 connections, which takes up more OS resources and still has more overhead for binary data due to being text — and encoding to base64 then gziping or using deflate to get somewhat down im size again won't make it faster or more efficient. But yeah on package loss there's head of line blocking, which http 3 solves by using UDP and handling the reliability on the higher level.

mgaunard 4y ago
TCP must send all data in order. To implement that, all data sent out gets written to a retransmit buffer (which may be dynamically sized, but only up to a maximum), and that data only gets removed once acked by the other end. Once the retransmit buffer is full, you are unable to write more data; your TCP implementation would then most likely suspend your process and wake it up once it can write again. Your thread is spending its time waiting for the other end to ack before it can send more data, delaying any other kind of work your thread has to do (such as serving other clients). The rate at which it can write doesn't magically increase by attempting to write multiple streams concurrently; quite the opposite. The rate is controlled by the other end, and what you should do if you implement a high-performance TCP service (HTTP or not) is simply disconnect clients that require you to ever block. Now you could say, "I don't care, I write the data asynchronously". Asynchronous writes are very bad and an anti-pattern, they should typically never be used because they mean you are just queueing data indefinitely until it is eventually written, wasting massive amounts of RAM (if not taking the machine down entirely). Queues in general are serious business and require a careful design to ensure the producer doesn't produce data faster than the consumer, with adequate back-pressure algorithms. It is better to keep track of when you can write and decide on how to handle not being able to write rather than block or queue like an idiot. Multiplexing with HTTP/2 logically makes the TCP stream which is logically a spsc queue into a pseudo-mpsc queue. Those are also way more difficult to reason about to guarantee fairness and back-pressure, they're pretty much a bottleneck by design. It's just a bad idea any way you look at it.
- tlamponi 4y ago
  Sorry, but did you let GPT write this response? As from the language and the amount of misinformation it contains it'd read as good fit - not meant to take as stab at you; just trying to fine tune my "detection heuristics" for such stuff. > TCP must send all data in order. To implement that, all data sent out gets written to a retransmit buffer (which may be dynamically sized, but only up to a maximum). To a send buffer, at least that's what most TCP implementation name it and with modern surprisingly high reliable networks it's mostly used for that. But yes, again (3rd time now), this is head-of-line blocking which HTTP/3 would solve (and which you also classified as being just useful for web browsers, whatever that even means). More important, yes while TCP is in-order and if the network is slow and/or drops a lot of packets you will notice that, BUT it can batch multiple HTTP requests into single TCP package and that's why an async concurrent or even parallel application can benefit from using it directly. > Your thread is spending its time waiting for the other end to ack before it can send more data, delaying any other kind of work your thread has to do (such as serving other clients). No it doesn't, as we are fully async on all layers with rust's zero-cost async model, e.g. using the async tokio rust crate, it can just continue to handle other requests. Also, what would change here by using HTTP/1.1? It has the same limitations as it also uses TCP, but additionally can't batch multiple requests into single TCP packets is text based (bad for binary data) and has additional down sites - e.g., slow on start up. > Now you could say, "I don't care, I write the data asynchronously". Asynchronous writes are very bad and an anti-pattern, they should typically never be used because they mean you are just queueing data indefinitely until it is eventually written, wasting massive amounts of RAM (if not taking the machine down entirely). Note that 1) we're talking mostly about reads here, and 2) async writes where never that bad and especially nowadays we're not in the spinning-rust-HDDs-only era anymore, so I'd recommend getting that rather dated and never quite right view updated. How would async writes even take down a machine? The kernel just uses the scheduler (or none for NVMes as they have massive amounts of queues and queue depths and can do that themselves) to reorder them in-flight and commit them, once there's to many, your write OPs will block (i.e., unsurprisingly the kernel doesn't allow every unprivileged process to just take down a machine by triggering some async writes) now with a sync model that means the thread is blocked, with async I can just do something else. > It's just a bad idea any way you look at it. Nope, not in our experience; and we heavily evaluated that back when adopting our current design. But happy to be actually proved wrong, just prototype the fully async multiplexing on top of HTTP/1.1 and let's see if it's faster. You can either check against our Proxmox Backup Server project directly; it has a benchmark that among other things checks the pure HTTP streaming bandwidth, but we also have a few (a bit dusted) example programs from the very early evaluation time: https://git.proxmox.com/?p=proxmox-backup.git;a=tree;f=examples;h=a98d687985ea21adb0d8ac9f6540825f401f2f7d;hb=HEAD https://git.proxmox.com/?p=proxmox-backup.git;a=tree;f=examp...

mgaunard 4y ago

TCP must send all data in order. To implement that, all data sent out gets written to a retransmit buffer (which may be dynamically sized, but only up to a maximum), and that data only gets removed once acked by the other end. Once the retransmit buffer is full, you are unable to write more data; your TCP implementation would then most likely suspend your process and wake it up once it can write again. Your thread is spending its time waiting for the other end to ack before it can send more data, delaying any other kind of work your thread has to do (such as serving other clients). The rate at which it can write doesn't magically increase by attempting to write multiple streams concurrently; quite the opposite. The rate is controlled by the other end, and what you should do if you implement a high-performance TCP service (HTTP or not) is simply disconnect clients that require you to ever block. Now you could say, "I don't care, I write the data asynchronously". Asynchronous writes are very bad and an anti-pattern, they should typically never be used because they mean you are just queueing data indefinitely until it is eventually written, wasting massive amounts of RAM (if not taking the machine down entirely). Queues in general are serious business and require a careful design to ensure the producer doesn't produce data faster than the consumer, with adequate back-pressure algorithms. It is better to keep track of when you can write and decide on how to handle not being able to write rather than block or queue like an idiot. Multiplexing with HTTP/2 logically makes the TCP stream which is logically a spsc queue into a pseudo-mpsc queue. Those are also way more difficult to reason about to guarantee fairness and back-pressure, they're pretty much a bottleneck by design. It's just a bad idea any way you look at it.

tlamponi 4y ago
Sorry, but did you let GPT write this response? As from the language and the amount of misinformation it contains it'd read as good fit - not meant to take as stab at you; just trying to fine tune my "detection heuristics" for such stuff. > TCP must send all data in order. To implement that, all data sent out gets written to a retransmit buffer (which may be dynamically sized, but only up to a maximum). To a send buffer, at least that's what most TCP implementation name it and with modern surprisingly high reliable networks it's mostly used for that. But yes, again (3rd time now), this is head-of-line blocking which HTTP/3 would solve (and which you also classified as being just useful for web browsers, whatever that even means). More important, yes while TCP is in-order and if the network is slow and/or drops a lot of packets you will notice that, BUT it can batch multiple HTTP requests into single TCP package and that's why an async concurrent or even parallel application can benefit from using it directly. > Your thread is spending its time waiting for the other end to ack before it can send more data, delaying any other kind of work your thread has to do (such as serving other clients). No it doesn't, as we are fully async on all layers with rust's zero-cost async model, e.g. using the async tokio rust crate, it can just continue to handle other requests. Also, what would change here by using HTTP/1.1? It has the same limitations as it also uses TCP, but additionally can't batch multiple requests into single TCP packets is text based (bad for binary data) and has additional down sites - e.g., slow on start up. > Now you could say, "I don't care, I write the data asynchronously". Asynchronous writes are very bad and an anti-pattern, they should typically never be used because they mean you are just queueing data indefinitely until it is eventually written, wasting massive amounts of RAM (if not taking the machine down entirely). Note that 1) we're talking mostly about reads here, and 2) async writes where never that bad and especially nowadays we're not in the spinning-rust-HDDs-only era anymore, so I'd recommend getting that rather dated and never quite right view updated. How would async writes even take down a machine? The kernel just uses the scheduler (or none for NVMes as they have massive amounts of queues and queue depths and can do that themselves) to reorder them in-flight and commit them, once there's to many, your write OPs will block (i.e., unsurprisingly the kernel doesn't allow every unprivileged process to just take down a machine by triggering some async writes) now with a sync model that means the thread is blocked, with async I can just do something else. > It's just a bad idea any way you look at it. Nope, not in our experience; and we heavily evaluated that back when adopting our current design. But happy to be actually proved wrong, just prototype the fully async multiplexing on top of HTTP/1.1 and let's see if it's faster. You can either check against our Proxmox Backup Server project directly; it has a benchmark that among other things checks the pure HTTP streaming bandwidth, but we also have a few (a bit dusted) example programs from the very early evaluation time: https://git.proxmox.com/?p=proxmox-backup.git;a=tree;f=examples;h=a98d687985ea21adb0d8ac9f6540825f401f2f7d;hb=HEAD https://git.proxmox.com/?p=proxmox-backup.git;a=tree;f=examp...