5 ms·

I mean... you do have to parse text formats. HTTP parsing may be a solved problem, but that doesn't mean the overhead or complexity of doing so disappears. Als

by jchw 7y ago

I mean... you do have to parse text formats. HTTP parsing may be a solved problem, but that doesn't mean the overhead or complexity of doing so disappears. Also, TLS is not really ideally lightweight for RPCs, but you should absolutely encrypt your RPC traffic (imo.) So I really think the whole stack is out. (P.S.: If you are wondering what kinds of 'lightweight' replacements for TLS exist, I think my personal favorite attempt is CurveCP, although it is a bit dated nowadays. I wouldn't often recommend people roll their own, but you could certainly do something simple with NaCl/libsodium directly. Maybe QUIC also fits the bill?)

seemslegit 7y ago
There is nothing that says that "RPC" can't do multiple request/response cycles over an existing open (and encrypted) connection rather than initiate a new one for every call, just like HTTP. Or even pipeline them like HTTP/2.0
- jchw 7y ago
  HTTP 2 is more reasonable. But by the time you get to HTTP 3 you're just doing HTTP 2 over QUIC. At which point, why not just send RPC payloads directly over QUIC?
  - seemslegit 7y ago
    Because what I have in place is a good old HTTP 1.1 ?
    - jchw 7y ago
      That's circular. I can't argue anything from this point. "Why should I get an electric car when I have a good old gas car in the driveway?" I don't have an answer. I do have answers for why one is better than the other. APIs work over HTTP in spite of limitations, not because of good synergy. I think gRPC is the most reasonable implementation of such (disclaimer: I work for Google, but not on gRPC and I've used gRPC before I worked at Google) but I still think it is overkill for many people. If you are using HTTP+REST+JSON and it works fine for what you are doing, then fine - there's an ecosystem already built around it. But the kinds of things people do with lighter weight and more efficient RPC layers literally aren't doable over standard HTTP/1.1 and REST. It enables stuff you wouldn't think of, when you can measure the absolute overhead in bytes. (As an example, I'm not aware of anyone actually doing this, but it would almost certainly be possible to forward low level signals like USB or perhaps even PCI express packets over a lightweight RPC layer, and get all of the encryption/access control/etc. you already have in your stack.) Answers for why HTTP/1.1 is a poor fit: - Text format requires text parsing. How long do you limit the header lines? What transport compression do you support? Text parsing is inefficient compared to binary formats. - A lot of difficult to understand behavior. When do you send 100 Continue, what do you do when you receive it? What happens when you are on a keep-alive connection and there's no Content-length? (There's a whole flow chart for something simple like this.) etc. - A lot of cruft. Like chunked encoding is weird. Trailers are also weird. What happens when a header is specified twice? Answers for why HTTP/2 is still a poor fit: - What are the headers even for? You now have this entire section of your request that doesn't matter, with its own compression scheme called HPACK. Why? - Server push. It's nice that you have bidirectional streams, but this is clearly designed for browser agents. gRPC repurposes this for bidirectional streaming as it should be, but... - ...Often times, hacks like that lead to the worst problem: You did all of this work to use HTTP as an RPC layer, and you can't even use it in a browser because the sane things you do for your backend might not be compatible. in gRPC there's a special layer for handling this, but it's a lot of additional cruft. HTTP/REST is great because there's a huge ecosystem, but that's not even a solid win due to the complexity. As an example, years ago I ran into huge problems with Amazon ELB because it was buffering my entire request and response payloads, and imposing its own timeouts on top. All documented behavior, but you can't just plug in this HTTP thing and hope for it to work. Basically anything in the middle that also speaks HTTP has to be carefully configured. Again, leading to doubt over the whole point of using a protocol like HTTP. There's rules for what should be GET, PUT, POST, DELETE, and yet those all interact strangely. No payload in GET body, some software gets weird about calls like DELETE, so sometimes you have to support POST for what should be a PUT and so on. And at the end of the day, all you really wanted was RPC payloads in both directions, and you have all of this crap around it, and it's largely just because web browsers exist, but none of this stuff even works well together. It works OK if you don't really care much and just throw a software stack together, but that doesn't mean it will be efficient, doesn't mean you won't run into problems. I definitely prefer to go for simpler, and HTTP is not actually simpler. It just has the benefit of having an existing ecosystem.
      - seemslegit 7y ago
        I don't think we disagree about anything here, if you wanna optimize for maximal machine/network utilization then optimize for that with gRPC or equivalent, if you wanna optimize for lean stack and have to use HTTP anyway because you're on the web then use (RPC over) HTTP - both can be considered more "efficient" depending on the setting and your constraints. But the point was that contrasting web requests with RPC is a mistake of category and has little to do with various IO handling and concurrency models that the author was discussing.
        jchw 7y ago
        Well, the thing is, I do agree with the author, though, on their point of not using web requests for RPCs. I think we must be interpreting the author's text differently.
        seemslegit 7y ago
        Either that or she's lumping together two separate issues, or both.
        lern_too_spel 7y ago
        The author never conflated the design choices of RPC protocol and threading model. Just the opposite, in fact. She listed them separately.
        seemslegit 7y ago
        Except she writes: " Then it does its weird userspace "thread" flip back to the original request's context and reads the response. It chews on this data, because, again, it's terrible JSON stuff and not anything reasonable like a strongly-typed, unambiguously, easily-deserialized message. The clock is ticking and ticking. " If she laments that it is bad design to do the deserialization on the IO thread it is just as true for JSON as it is for protobuf or whatever "true rpc" format she considers worthy.
- dfox 7y ago
  Or even do RPC calls in both directions over one stream socket... essentially all the “inflexible” RPC protocols of 90's can do that (and incidentally the way how this is implemented usually involves nested event loops)...
- dirtydroog 7y ago
  HTTP1.1 only closes the connection if one party sets Keepalive: Close. It was HTTP1.0 that was one-shot. There was also pipelining support added but apparently nobody bothered to support it.
topspin 7y ago
> that doesn't mean the overhead or complexity of doing so disappears No, it doesn't. But there is no evidence this overhead actually mattered here. It usually doesn't because the CPUs easily outrun whatever bandwidth is available which is why JSON over HTTP is fine 99% of the time. There is absolutely nothing in this blog post that shows that's not also the case here. No rationale is provided as to how a strongly typed RPC mechanism would solve any actual problems the services is having. So we're left with guesswork and the authors hang-ups about HTTP vs some as yet unnamed RPC solution. Also, Gunicorn? Thundering herd? These are solved problems. Space the toy proxy and use something real like haproxy. At a minimum. Finally, none of this griping about HTTP vs RPC actually addresses the _actual_ problem: the server can't process requests in a timely manner. That points to some deeper inefficiency or design issue that likely has nothing to do with Python or Gunicorn or Gevent at all. We're not given any insight as to what the hell the server is doing with all that CPU. Or why the client isn't using a protocol intended for long running processes; RPC schemes have timeouts too ya know.... This reads like a poorly informed rant.
- dirtydroog 7y ago
  HTTP is overly verbose, is a pain and slow to parse, and there are various interpretations of what the protocol spec allows and disallows. If machines are communicating why should it be in a human readable format? Binary is far quicker
  - topspin 7y ago
    This is based on a number of obsolete premises. Contemporary HTTP techniques include[1] framed binary wire protocol, redundant header elimination, compressed header values and efficient cryptography. HTTP/2 + TLS 1.3+ are extremely efficient together and are difficult to improve upon. When compatibility, implementation quality and ubiquity[2] are considered they are effectively impossible to improve upon. Except... If, in the unlikely case that your particular bit of brilliance is indeed hampered by the vestigial amount of overhead still present in contemporary HTTP, then you might do as Google and several other operations have done and dispense with traditional techniques (including TCP) altogether via QUIC. Just because what you see in the Network tab of your browser's 'Developer tools' window looks like something from 2005 doesn't mean that's what is actually on the wire. It mostly isn't any longer as a share of global traffic. But again, all of that is irrelevant; the post provided no evidence that replacing HTTP with some dubiously unnamed form of RPC would solve any actual problems. HTTP was tossed in the rant basket with a bunch of other things, few of which appeared relevant to the actual failure modes cited. [1] https://www.oreilly.com/library/view/high-performance-browser/9781449344757/ch12.html https://www.oreilly.com/library/view/high-performance-browse... [2] https://github.com/http2/http2-spec/wiki/Implementations https://github.com/http2/http2-spec/wiki/Implementations

by jchw 7y ago

seemslegit 7y ago

There is nothing that says that "RPC" can't do multiple request/response cycles over an existing open (and encrypted) connection rather than initiate a new one for every call, just like HTTP. Or even pipeline them like HTTP/2.0

jchw 7y ago
HTTP 2 is more reasonable. But by the time you get to HTTP 3 you're just doing HTTP 2 over QUIC. At which point, why not just send RPC payloads directly over QUIC?
- seemslegit 7y ago
  Because what I have in place is a good old HTTP 1.1 ?
  - jchw 7y ago
    That's circular. I can't argue anything from this point. "Why should I get an electric car when I have a good old gas car in the driveway?" I don't have an answer. I do have answers for why one is better than the other. APIs work over HTTP in spite of limitations, not because of good synergy. I think gRPC is the most reasonable implementation of such (disclaimer: I work for Google, but not on gRPC and I've used gRPC before I worked at Google) but I still think it is overkill for many people. If you are using HTTP+REST+JSON and it works fine for what you are doing, then fine - there's an ecosystem already built around it. But the kinds of things people do with lighter weight and more efficient RPC layers literally aren't doable over standard HTTP/1.1 and REST. It enables stuff you wouldn't think of, when you can measure the absolute overhead in bytes. (As an example, I'm not aware of anyone actually doing this, but it would almost certainly be possible to forward low level signals like USB or perhaps even PCI express packets over a lightweight RPC layer, and get all of the encryption/access control/etc. you already have in your stack.) Answers for why HTTP/1.1 is a poor fit: - Text format requires text parsing. How long do you limit the header lines? What transport compression do you support? Text parsing is inefficient compared to binary formats. - A lot of difficult to understand behavior. When do you send 100 Continue, what do you do when you receive it? What happens when you are on a keep-alive connection and there's no Content-length? (There's a whole flow chart for something simple like this.) etc. - A lot of cruft. Like chunked encoding is weird. Trailers are also weird. What happens when a header is specified twice? Answers for why HTTP/2 is still a poor fit: - What are the headers even for? You now have this entire section of your request that doesn't matter, with its own compression scheme called HPACK. Why? - Server push. It's nice that you have bidirectional streams, but this is clearly designed for browser agents. gRPC repurposes this for bidirectional streaming as it should be, but... - ...Often times, hacks like that lead to the worst problem: You did all of this work to use HTTP as an RPC layer, and you can't even use it in a browser because the sane things you do for your backend might not be compatible. in gRPC there's a special layer for handling this, but it's a lot of additional cruft. HTTP/REST is great because there's a huge ecosystem, but that's not even a solid win due to the complexity. As an example, years ago I ran into huge problems with Amazon ELB because it was buffering my entire request and response payloads, and imposing its own timeouts on top. All documented behavior, but you can't just plug in this HTTP thing and hope for it to work. Basically anything in the middle that also speaks HTTP has to be carefully configured. Again, leading to doubt over the whole point of using a protocol like HTTP. There's rules for what should be GET, PUT, POST, DELETE, and yet those all interact strangely. No payload in GET body, some software gets weird about calls like DELETE, so sometimes you have to support POST for what should be a PUT and so on. And at the end of the day, all you really wanted was RPC payloads in both directions, and you have all of this crap around it, and it's largely just because web browsers exist, but none of this stuff even works well together. It works OK if you don't really care much and just throw a software stack together, but that doesn't mean it will be efficient, doesn't mean you won't run into problems. I definitely prefer to go for simpler, and HTTP is not actually simpler. It just has the benefit of having an existing ecosystem.
    - seemslegit 7y ago
      I don't think we disagree about anything here, if you wanna optimize for maximal machine/network utilization then optimize for that with gRPC or equivalent, if you wanna optimize for lean stack and have to use HTTP anyway because you're on the web then use (RPC over) HTTP - both can be considered more "efficient" depending on the setting and your constraints. But the point was that contrasting web requests with RPC is a mistake of category and has little to do with various IO handling and concurrency models that the author was discussing.
      - jchw 7y ago
        Well, the thing is, I do agree with the author, though, on their point of not using web requests for RPCs. I think we must be interpreting the author's text differently.
        seemslegit 7y ago
        Either that or she's lumping together two separate issues, or both.
      - lern_too_spel 7y ago
        The author never conflated the design choices of RPC protocol and threading model. Just the opposite, in fact. She listed them separately.
        seemslegit 7y ago
        Except she writes: " Then it does its weird userspace "thread" flip back to the original request's context and reads the response. It chews on this data, because, again, it's terrible JSON stuff and not anything reasonable like a strongly-typed, unambiguously, easily-deserialized message. The clock is ticking and ticking. " If she laments that it is bad design to do the deserialization on the IO thread it is just as true for JSON as it is for protobuf or whatever "true rpc" format she considers worthy.
dfox 7y ago
Or even do RPC calls in both directions over one stream socket... essentially all the “inflexible” RPC protocols of 90's can do that (and incidentally the way how this is implemented usually involves nested event loops)...
dirtydroog 7y ago
HTTP1.1 only closes the connection if one party sets Keepalive: Close. It was HTTP1.0 that was one-shot. There was also pipelining support added but apparently nobody bothered to support it.

topspin 7y ago

> that doesn't mean the overhead or complexity of doing so disappears No, it doesn't. But there is no evidence this overhead actually mattered here. It usually doesn't because the CPUs easily outrun whatever bandwidth is available which is why JSON over HTTP is fine 99% of the time. There is absolutely nothing in this blog post that shows that's not also the case here. No rationale is provided as to how a strongly typed RPC mechanism would solve any actual problems the services is having. So we're left with guesswork and the authors hang-ups about HTTP vs some as yet unnamed RPC solution. Also, Gunicorn? Thundering herd? These are solved problems. Space the toy proxy and use something real like haproxy. At a minimum. Finally, none of this griping about HTTP vs RPC actually addresses the _actual_ problem: the server can't process requests in a timely manner. That points to some deeper inefficiency or design issue that likely has nothing to do with Python or Gunicorn or Gevent at all. We're not given any insight as to what the hell the server is doing with all that CPU. Or why the client isn't using a protocol intended for long running processes; RPC schemes have timeouts too ya know.... This reads like a poorly informed rant.

dirtydroog 7y ago
HTTP is overly verbose, is a pain and slow to parse, and there are various interpretations of what the protocol spec allows and disallows. If machines are communicating why should it be in a human readable format? Binary is far quicker
- topspin 7y ago
  This is based on a number of obsolete premises. Contemporary HTTP techniques include[1] framed binary wire protocol, redundant header elimination, compressed header values and efficient cryptography. HTTP/2 + TLS 1.3+ are extremely efficient together and are difficult to improve upon. When compatibility, implementation quality and ubiquity[2] are considered they are effectively impossible to improve upon. Except... If, in the unlikely case that your particular bit of brilliance is indeed hampered by the vestigial amount of overhead still present in contemporary HTTP, then you might do as Google and several other operations have done and dispense with traditional techniques (including TCP) altogether via QUIC. Just because what you see in the Network tab of your browser's 'Developer tools' window looks like something from 2005 doesn't mean that's what is actually on the wire. It mostly isn't any longer as a share of global traffic. But again, all of that is irrelevant; the post provided no evidence that replacing HTTP with some dubiously unnamed form of RPC would solve any actual problems. HTTP was tossed in the rant basket with a bunch of other things, few of which appeared relevant to the actual failure modes cited. [1] https://www.oreilly.com/library/view/high-performance-browser/9781449344757/ch12.html https://www.oreilly.com/library/view/high-performance-browse... [2] https://github.com/http2/http2-spec/wiki/Implementations https://github.com/http2/http2-spec/wiki/Implementations