5 ms·

"I will not use web requests when the situation calls for RPCs" I'm surprised how often devs treat this distinction as architecturally meaningful. Web requests

"I will not use web requests when the situation calls for RPCs" I'm surprised how often devs treat this distinction as architecturally meaningful. Web requests are just RPCs with some of the parameters standardized and multiple surfaces for parameters and return values - query string, headers, body. This is completely orthogonal to the strategy used to schedule IO, concurrency, etc.

jchw 7y ago
I mean... you do have to parse text formats. HTTP parsing may be a solved problem, but that doesn't mean the overhead or complexity of doing so disappears. Also, TLS is not really ideally lightweight for RPCs, but you should absolutely encrypt your RPC traffic (imo.) So I really think the whole stack is out. (P.S.: If you are wondering what kinds of 'lightweight' replacements for TLS exist, I think my personal favorite attempt is CurveCP, although it is a bit dated nowadays. I wouldn't often recommend people roll their own, but you could certainly do something simple with NaCl/libsodium directly. Maybe QUIC also fits the bill?)
- seemslegit 7y ago
  There is nothing that says that "RPC" can't do multiple request/response cycles over an existing open (and encrypted) connection rather than initiate a new one for every call, just like HTTP. Or even pipeline them like HTTP/2.0
  - jchw 7y ago
    HTTP 2 is more reasonable. But by the time you get to HTTP 3 you're just doing HTTP 2 over QUIC. At which point, why not just send RPC payloads directly over QUIC?
    - seemslegit 7y ago
      Because what I have in place is a good old HTTP 1.1 ?
      - jchw 7y ago
        That's circular. I can't argue anything from this point. "Why should I get an electric car when I have a good old gas car in the driveway?" I don't have an answer. I do have answers for why one is better than the other. APIs work over HTTP in spite of limitations, not because of good synergy. I think gRPC is the most reasonable implementation of such (disclaimer: I work for Google, but not on gRPC and I've used gRPC before I worked at Google) but I still think it is overkill for many people. If you are using HTTP+REST+JSON and it works fine for what you are doing, then fine - there's an ecosystem already built around it. But the kinds of things people do with lighter weight and more efficient RPC layers literally aren't doable over standard HTTP/1.1 and REST. It enables stuff you wouldn't think of, when you can measure the absolute overhead in bytes. (As an example, I'm not aware of anyone actually doing this, but it would almost certainly be possible to forward low level signals like USB or perhaps even PCI express packets over a lightweight RPC layer, and get all of the encryption/access control/etc. you already have in your stack.) Answers for why HTTP/1.1 is a poor fit: - Text format requires text parsing. How long do you limit the header lines? What transport compression do you support? Text parsing is inefficient compared to binary formats. - A lot of difficult to understand behavior. When do you send 100 Continue, what do you do when you receive it? What happens when you are on a keep-alive connection and there's no Content-length? (There's a whole flow chart for something simple like this.) etc. - A lot of cruft. Like chunked encoding is weird. Trailers are also weird. What happens when a header is specified twice? Answers for why HTTP/2 is still a poor fit: - What are the headers even for? You now have this entire section of your request that doesn't matter, with its own compression scheme called HPACK. Why? - Server push. It's nice that you have bidirectional streams, but this is clearly designed for browser agents. gRPC repurposes this for bidirectional streaming as it should be, but... - ...Often times, hacks like that lead to the worst problem: You did all of this work to use HTTP as an RPC layer, and you can't even use it in a browser because the sane things you do for your backend might not be compatible. in gRPC there's a special layer for handling this, but it's a lot of additional cruft. HTTP/REST is great because there's a huge ecosystem, but that's not even a solid win due to the complexity. As an example, years ago I ran into huge problems with Amazon ELB because it was buffering my entire request and response payloads, and imposing its own timeouts on top. All documented behavior, but you can't just plug in this HTTP thing and hope for it to work. Basically anything in the middle that also speaks HTTP has to be carefully configured. Again, leading to doubt over the whole point of using a protocol like HTTP. There's rules for what should be GET, PUT, POST, DELETE, and yet those all interact strangely. No payload in GET body, some software gets weird about calls like DELETE, so sometimes you have to support POST for what should be a PUT and so on. And at the end of the day, all you really wanted was RPC payloads in both directions, and you have all of this crap around it, and it's largely just because web browsers exist, but none of this stuff even works well together. It works OK if you don't really care much and just throw a software stack together, but that doesn't mean it will be efficient, doesn't mean you won't run into problems. I definitely prefer to go for simpler, and HTTP is not actually simpler. It just has the benefit of having an existing ecosystem.
  - dfox 7y ago
    Or even do RPC calls in both directions over one stream socket... essentially all the “inflexible” RPC protocols of 90's can do that (and incidentally the way how this is implemented usually involves nested event loops)...
  - dirtydroog 7y ago
    HTTP1.1 only closes the connection if one party sets Keepalive: Close. It was HTTP1.0 that was one-shot. There was also pipelining support added but apparently nobody bothered to support it.
- topspin 7y ago
  > that doesn't mean the overhead or complexity of doing so disappears No, it doesn't. But there is no evidence this overhead actually mattered here. It usually doesn't because the CPUs easily outrun whatever bandwidth is available which is why JSON over HTTP is fine 99% of the time. There is absolutely nothing in this blog post that shows that's not also the case here. No rationale is provided as to how a strongly typed RPC mechanism would solve any actual problems the services is having. So we're left with guesswork and the authors hang-ups about HTTP vs some as yet unnamed RPC solution. Also, Gunicorn? Thundering herd? These are solved problems. Space the toy proxy and use something real like haproxy. At a minimum. Finally, none of this griping about HTTP vs RPC actually addresses the _actual_ problem: the server can't process requests in a timely manner. That points to some deeper inefficiency or design issue that likely has nothing to do with Python or Gunicorn or Gevent at all. We're not given any insight as to what the hell the server is doing with all that CPU. Or why the client isn't using a protocol intended for long running processes; RPC schemes have timeouts too ya know.... This reads like a poorly informed rant.
  - dirtydroog 7y ago
    HTTP is overly verbose, is a pain and slow to parse, and there are various interpretations of what the protocol spec allows and disallows. If machines are communicating why should it be in a human readable format? Binary is far quicker
    - topspin 7y ago
      This is based on a number of obsolete premises. Contemporary HTTP techniques include[1] framed binary wire protocol, redundant header elimination, compressed header values and efficient cryptography. HTTP/2 + TLS 1.3+ are extremely efficient together and are difficult to improve upon. When compatibility, implementation quality and ubiquity[2] are considered they are effectively impossible to improve upon. Except... If, in the unlikely case that your particular bit of brilliance is indeed hampered by the vestigial amount of overhead still present in contemporary HTTP, then you might do as Google and several other operations have done and dispense with traditional techniques (including TCP) altogether via QUIC. Just because what you see in the Network tab of your browser's 'Developer tools' window looks like something from 2005 doesn't mean that's what is actually on the wire. It mostly isn't any longer as a share of global traffic. But again, all of that is irrelevant; the post provided no evidence that replacing HTTP with some dubiously unnamed form of RPC would solve any actual problems. HTTP was tossed in the rant basket with a bunch of other things, few of which appeared relevant to the actual failure modes cited. [1] https://www.oreilly.com/library/view/high-performance-browser/9781449344757/ch12.html https://www.oreilly.com/library/view/high-performance-browse... [2] https://github.com/http2/http2-spec/wiki/Implementations https://github.com/http2/http2-spec/wiki/Implementations
WookieRushing 7y ago
RPC systems come in many different levels like programming languages. While there are low level RPC systems which are just a simple layer around web requests, there are others that do a lot more. They can do retries, host selection, and stateful operations like the article mentions. These systems tend to take care of a bunch of easy to mess up logic which tends to be accumulate around any system that wants to send something and not mess up. Any sufficiently old web request system tends to look identical to a high level RPC system designed badly. So choosing an RPC system should give you all the features you’d eventually end up building around using web requests without spending your time rewriting it.
- seemslegit 7y ago
  See above
wrs 7y ago
I think what’s she’s getting at is that RPC usually comes bundled with some kind of strictly typed serialization format and standard infrastructure (service discovery, dispatch, error handling, etc.). A lot of web frameworks just take one request and hand it to a function, and the rest, including decoding the loose JSON that might be in the body, is up to you.
- seemslegit 7y ago
  Sure, but this is porcelain and has nothing to do with threading vs forking vs greenlets etc.
keyanp 7y ago
Thanks for posting this. I had the same reaction. I think making this distinction ends up muddling what an author means when they refer to RPC and ends up overloading the term.
kragen 7y ago
REST constrains your architecture in ways that RPC doesn't. Of course you can use HTTP without using REST, but then you're paying a lot of the costs of REST without getting its benefits, and a simpler RPC protocol might be better.
- seemslegit 7y ago
  Contrasting REST with RPC makes more sense, while REST apis map quite naturally to RPC as the other way around is more convoluted. But a dedicatd RPC stack such as grpc or thrift constraints my architecture in other ways, that json-over-http doesn't. And what are those unavoidable costs of REST that are not just the wire overhead of HTTP ?
  - kragen 7y ago
    Proxies, built-in retry semantics in many clients, code complexity for dealing with the mandatory flexibility in HTTP, the presumption that GET is safe for, say, prefetching or spidering, etc. I think REST is often worth the cost (I might be the only person on this thread who has written an HTTP server in assembly), but if you're not using REST, using HTTP will tend to cause you a lot of headaches you could have avoided, for little or no benefit. “But firewalls!”
    - bostik 7y ago
      I would think this is orthogonal to REST-vs-RPC: > the presumption that GET is safe for [reads] GET requests are supposed to be idempotent and side-effect free. Sadly, too often, they are not. Internal state gets mutated, a query parameter gets stored and/or processed in a way that affects future reads, and so on. But then again, nothing prevents you from writing RPC based service with those semantics either. It just might be that people who develop and maintain RPC services are better aware of the consequences of their actions. REST is conceptually easy, has human-readable on-the-wire payloads and operates on synchronous semantics.[×] The bar for entry is simply lower, allowing even an inexperienced developer to get going with very little friction. The very fact that I can do this: import json, requests dada = json.loads(whatever) res = requests.post(URL, data=dada) ... means it's incredibly easy to get off the ground and shuffle data between services. Conversely, it's also easy to send the wrong data, in a wrong way. ×: webhooks are effectively callbacks, but instead of setting the code entry point for async return in your call, you expose a webhook route and treat it as any other request.

jchw 7y ago

I mean... you do have to parse text formats. HTTP parsing may be a solved problem, but that doesn't mean the overhead or complexity of doing so disappears. Also, TLS is not really ideally lightweight for RPCs, but you should absolutely encrypt your RPC traffic (imo.) So I really think the whole stack is out. (P.S.: If you are wondering what kinds of 'lightweight' replacements for TLS exist, I think my personal favorite attempt is CurveCP, although it is a bit dated nowadays. I wouldn't often recommend people roll their own, but you could certainly do something simple with NaCl/libsodium directly. Maybe QUIC also fits the bill?)

seemslegit 7y ago
There is nothing that says that "RPC" can't do multiple request/response cycles over an existing open (and encrypted) connection rather than initiate a new one for every call, just like HTTP. Or even pipeline them like HTTP/2.0
- jchw 7y ago
  HTTP 2 is more reasonable. But by the time you get to HTTP 3 you're just doing HTTP 2 over QUIC. At which point, why not just send RPC payloads directly over QUIC?
  - seemslegit 7y ago
    Because what I have in place is a good old HTTP 1.1 ?
    - jchw 7y ago
      That's circular. I can't argue anything from this point. "Why should I get an electric car when I have a good old gas car in the driveway?" I don't have an answer. I do have answers for why one is better than the other. APIs work over HTTP in spite of limitations, not because of good synergy. I think gRPC is the most reasonable implementation of such (disclaimer: I work for Google, but not on gRPC and I've used gRPC before I worked at Google) but I still think it is overkill for many people. If you are using HTTP+REST+JSON and it works fine for what you are doing, then fine - there's an ecosystem already built around it. But the kinds of things people do with lighter weight and more efficient RPC layers literally aren't doable over standard HTTP/1.1 and REST. It enables stuff you wouldn't think of, when you can measure the absolute overhead in bytes. (As an example, I'm not aware of anyone actually doing this, but it would almost certainly be possible to forward low level signals like USB or perhaps even PCI express packets over a lightweight RPC layer, and get all of the encryption/access control/etc. you already have in your stack.) Answers for why HTTP/1.1 is a poor fit: - Text format requires text parsing. How long do you limit the header lines? What transport compression do you support? Text parsing is inefficient compared to binary formats. - A lot of difficult to understand behavior. When do you send 100 Continue, what do you do when you receive it? What happens when you are on a keep-alive connection and there's no Content-length? (There's a whole flow chart for something simple like this.) etc. - A lot of cruft. Like chunked encoding is weird. Trailers are also weird. What happens when a header is specified twice? Answers for why HTTP/2 is still a poor fit: - What are the headers even for? You now have this entire section of your request that doesn't matter, with its own compression scheme called HPACK. Why? - Server push. It's nice that you have bidirectional streams, but this is clearly designed for browser agents. gRPC repurposes this for bidirectional streaming as it should be, but... - ...Often times, hacks like that lead to the worst problem: You did all of this work to use HTTP as an RPC layer, and you can't even use it in a browser because the sane things you do for your backend might not be compatible. in gRPC there's a special layer for handling this, but it's a lot of additional cruft. HTTP/REST is great because there's a huge ecosystem, but that's not even a solid win due to the complexity. As an example, years ago I ran into huge problems with Amazon ELB because it was buffering my entire request and response payloads, and imposing its own timeouts on top. All documented behavior, but you can't just plug in this HTTP thing and hope for it to work. Basically anything in the middle that also speaks HTTP has to be carefully configured. Again, leading to doubt over the whole point of using a protocol like HTTP. There's rules for what should be GET, PUT, POST, DELETE, and yet those all interact strangely. No payload in GET body, some software gets weird about calls like DELETE, so sometimes you have to support POST for what should be a PUT and so on. And at the end of the day, all you really wanted was RPC payloads in both directions, and you have all of this crap around it, and it's largely just because web browsers exist, but none of this stuff even works well together. It works OK if you don't really care much and just throw a software stack together, but that doesn't mean it will be efficient, doesn't mean you won't run into problems. I definitely prefer to go for simpler, and HTTP is not actually simpler. It just has the benefit of having an existing ecosystem.
- dfox 7y ago
  Or even do RPC calls in both directions over one stream socket... essentially all the “inflexible” RPC protocols of 90's can do that (and incidentally the way how this is implemented usually involves nested event loops)...
- dirtydroog 7y ago
  HTTP1.1 only closes the connection if one party sets Keepalive: Close. It was HTTP1.0 that was one-shot. There was also pipelining support added but apparently nobody bothered to support it.
topspin 7y ago
> that doesn't mean the overhead or complexity of doing so disappears No, it doesn't. But there is no evidence this overhead actually mattered here. It usually doesn't because the CPUs easily outrun whatever bandwidth is available which is why JSON over HTTP is fine 99% of the time. There is absolutely nothing in this blog post that shows that's not also the case here. No rationale is provided as to how a strongly typed RPC mechanism would solve any actual problems the services is having. So we're left with guesswork and the authors hang-ups about HTTP vs some as yet unnamed RPC solution. Also, Gunicorn? Thundering herd? These are solved problems. Space the toy proxy and use something real like haproxy. At a minimum. Finally, none of this griping about HTTP vs RPC actually addresses the _actual_ problem: the server can't process requests in a timely manner. That points to some deeper inefficiency or design issue that likely has nothing to do with Python or Gunicorn or Gevent at all. We're not given any insight as to what the hell the server is doing with all that CPU. Or why the client isn't using a protocol intended for long running processes; RPC schemes have timeouts too ya know.... This reads like a poorly informed rant.
- dirtydroog 7y ago
  HTTP is overly verbose, is a pain and slow to parse, and there are various interpretations of what the protocol spec allows and disallows. If machines are communicating why should it be in a human readable format? Binary is far quicker
  - topspin 7y ago
    This is based on a number of obsolete premises. Contemporary HTTP techniques include[1] framed binary wire protocol, redundant header elimination, compressed header values and efficient cryptography. HTTP/2 + TLS 1.3+ are extremely efficient together and are difficult to improve upon. When compatibility, implementation quality and ubiquity[2] are considered they are effectively impossible to improve upon. Except... If, in the unlikely case that your particular bit of brilliance is indeed hampered by the vestigial amount of overhead still present in contemporary HTTP, then you might do as Google and several other operations have done and dispense with traditional techniques (including TCP) altogether via QUIC. Just because what you see in the Network tab of your browser's 'Developer tools' window looks like something from 2005 doesn't mean that's what is actually on the wire. It mostly isn't any longer as a share of global traffic. But again, all of that is irrelevant; the post provided no evidence that replacing HTTP with some dubiously unnamed form of RPC would solve any actual problems. HTTP was tossed in the rant basket with a bunch of other things, few of which appeared relevant to the actual failure modes cited. [1] https://www.oreilly.com/library/view/high-performance-browser/9781449344757/ch12.html https://www.oreilly.com/library/view/high-performance-browse... [2] https://github.com/http2/http2-spec/wiki/Implementations https://github.com/http2/http2-spec/wiki/Implementations

WookieRushing 7y ago

RPC systems come in many different levels like programming languages. While there are low level RPC systems which are just a simple layer around web requests, there are others that do a lot more. They can do retries, host selection, and stateful operations like the article mentions. These systems tend to take care of a bunch of easy to mess up logic which tends to be accumulate around any system that wants to send something and not mess up. Any sufficiently old web request system tends to look identical to a high level RPC system designed badly. So choosing an RPC system should give you all the features you’d eventually end up building around using web requests without spending your time rewriting it.

seemslegit 7y ago
See above

wrs 7y ago

I think what’s she’s getting at is that RPC usually comes bundled with some kind of strictly typed serialization format and standard infrastructure (service discovery, dispatch, error handling, etc.). A lot of web frameworks just take one request and hand it to a function, and the rest, including decoding the loose JSON that might be in the body, is up to you.

seemslegit 7y ago
Sure, but this is porcelain and has nothing to do with threading vs forking vs greenlets etc.

keyanp 7y ago

Thanks for posting this. I had the same reaction. I think making this distinction ends up muddling what an author means when they refer to RPC and ends up overloading the term.

kragen 7y ago

REST constrains your architecture in ways that RPC doesn't. Of course you can use HTTP without using REST, but then you're paying a lot of the costs of REST without getting its benefits, and a simpler RPC protocol might be better.

seemslegit 7y ago
Contrasting REST with RPC makes more sense, while REST apis map quite naturally to RPC as the other way around is more convoluted. But a dedicatd RPC stack such as grpc or thrift constraints my architecture in other ways, that json-over-http doesn't. And what are those unavoidable costs of REST that are not just the wire overhead of HTTP ?
- kragen 7y ago
  Proxies, built-in retry semantics in many clients, code complexity for dealing with the mandatory flexibility in HTTP, the presumption that GET is safe for, say, prefetching or spidering, etc. I think REST is often worth the cost (I might be the only person on this thread who has written an HTTP server in assembly), but if you're not using REST, using HTTP will tend to cause you a lot of headaches you could have avoided, for little or no benefit. “But firewalls!”
  - bostik 7y ago
    I would think this is orthogonal to REST-vs-RPC: > the presumption that GET is safe for [reads] GET requests are supposed to be idempotent and side-effect free. Sadly, too often, they are not. Internal state gets mutated, a query parameter gets stored and/or processed in a way that affects future reads, and so on. But then again, nothing prevents you from writing RPC based service with those semantics either. It just might be that people who develop and maintain RPC services are better aware of the consequences of their actions. REST is conceptually easy, has human-readable on-the-wire payloads and operates on synchronous semantics.[×] The bar for entry is simply lower, allowing even an inexperienced developer to get going with very little friction. The very fact that I can do this: import json, requests dada = json.loads(whatever) res = requests.post(URL, data=dada) ... means it's incredibly easy to get off the ground and shuffle data between services. Conversely, it's also easy to send the wrong data, in a wrong way. ×: webhooks are effectively callbacks, but instead of setting the code entry point for async return in your call, you expose a webhook route and treat it as any other request.