3 ms·

> gRPC uses HTTP/2 ... while on a typical JSON API which probably uses http/1.1, you can't. What are you actually talking about? An HTTP API by itself doesn't

by brabel 6y ago

> gRPC uses HTTP/2 ... while on a typical JSON API which probably uses http/1.1, you can't. What are you actually talking about? An HTTP API by itself doesn't actually care about which http version is used, that's something only the HTTP server that serves the API should care about (nowadays, pretty much any http server in any language supports HTTP/2 and 1.1). > Bi-directional streaming e.g. Kubernetes controllers use the Watch API which notifies the object changes/add/deletes in etcd to the control plane. Even HTTP/1.1 has mechanisms for that, like long polling and chunk encoding (which allows infinite streams to keep a dual channel of communication open where each chunk can be treated as a message - with "headers" and all). > Client libraries are automatically generated, and not error prone. This is true, but we've had similar technology since the SOAP days... you could easily do the same with a XML-based HTTP API. By the way, most of your points are against JSON, not a HTTP API per se, which usually would allow a number of formats , including XML and JSON at least. > Builtin authentication primitives... HTTP also has that when you include cookies and something like OAuth/OpenID. > gRPC has support for interceptors which are like middleware functions This kind of thing is better done by using the specific platform you're running on (depends on the language) so I don't see something like this as something desirable on the specific RPC framework you're using. With HTTP, caching, logging etc are trivial to do and one of the strongest advantages of using HTTP in the first place, so I think you're confusingly making a point for HTTP here, unless your focus is on the RPC side of things? In which case, you would have to make a case on why RPC is a better fit than HTTP for etcd, which you haven't. So all in all, I found your points utterly unconvincing, but presented with so much conviction that you're actually right that I could not resist to respond (even if I don't want at all to get into a pointless discussion on the merits of HTTP VS RPC or JSON VS Protobuffers)!

sercand 6y ago
By using gRPC, we get much more without thinking too much. Also, bi-directional streaming is instantaneous and it is not like afterthought long polling. For example, whole etcd API is located at https://github.com/etcd-io/etcd/blob/4c6881ffe4b3bae257c07205c2334f84c0f4577b/etcdserver/etcdserverpb/rpc.proto https://github.com/etcd-io/etcd/blob/4c6881ffe4b3bae257c0720... It is pretty straightforward and not too hard to understand the API. Designing APIs with protobuf makes things convenient and it brings lots of already written tools with it. Since gRPC built on top of HTTP/2, we thought gRPC as easiest and performant way of writing a HTTP API with good defaults.
- cmrdporcupine 6y ago
  "we get much more without thinking too much" is precisely one of the reasons for the original push to REST (and against RPC at that time SOAP and CORBA and RMI etc.) was made in our industry 10-15 years ago. By falsely representing a _remote_ resource as if it was a _local_ one we open a whole can of worms; not just performance, but resilience, infrastructure issues, etc. Transferring documents over HTTP with a common language of HTTP's verbs was supposed to get programmers to model their applications as the sets of resources that they are, discourage them from making excessive round trips, make debugging easier, and make use of standard HTTP load balancing hardware. Yes it's easy to make that RPC call. But should you? I've been off doing other things in the intervening period, but while I had my back turned the industry seems to have turned its back on REST and gone whole hog on RPC. Again. At first I was thinking this was just internally here at Google, where protobufs and gRPC reign supreme. But it seems to have taken hold everywhere. What did I miss? Why have we swung this way. Again. Is the pendulum going to go back?
  - apta 6y ago
    > By falsely representing a _remote_ resource as if it was a _local_ one we open a whole can of worms I've heard this argument before, but how does gRPC itself cause this issue to manifest? I'm curious to hear what your opinions are on a better alternative, and how not to represent a remote resource as a local one.
    - cmrdporcupine 6y ago
      You could use gRPC responsibly, yes. But the fact that it presents the remote resource in an API which resembles a local object means that programmers often get lazy in the manner of which they think about these things. The REST semantic is supposed to make this more explicit. A remote object is not an object in your program or even your computer. It's something you're taking from something that is computationally miles and miles away. Compared to the microseconds it takes to dispatch a local call, it's an eternity away, even on a local network of the highest speed. Accumulate those latencies over thousands of dispatches, and trouble can ensue. I am reminded of an observation from when I first joined Google, coming out of their acquisition of the scrappy awesome ads company I worked at (Admeld). We had a little service that kept track of ad impression caps / budgeting. We didn't want to serve an ad a single time more than the customer wanted us to, etc. Serving many thousands of ads per second, a process distributed across multiple machines in multiple data centres, this is a bit of a tricky shared state problem. The people who came before me had designed a rather clever solution which used a form of backoff to trickle down the number of ads served as they got closer and closer to budget cap, and to synchronize this state across clusters (this was before there were Rafty services to make this kind of thing easier, BTW). We had a bit of a show and tell with Google when we first joined. I wanted to know how they were handling this problem, since their scale was many times ours, so I asked the question and got a puzzled/annoyed look: "Oh we just make an RPC call to our budgeting server." Summary: if you're at Google you don't have to worry as much about these problems. You still do, but there's an insane amount of infrastructure and horsepower and an army of SREs to help make it happen. So, yeah, my point is -- just because Google does something or has invented something doesn't mean it's the best way to do it, especially in a smaller more cost conscious organization.

by brabel 6y ago

sercand 6y ago

By using gRPC, we get much more without thinking too much. Also, bi-directional streaming is instantaneous and it is not like afterthought long polling. For example, whole etcd API is located at https://github.com/etcd-io/etcd/blob/4c6881ffe4b3bae257c07205c2334f84c0f4577b/etcdserver/etcdserverpb/rpc.proto https://github.com/etcd-io/etcd/blob/4c6881ffe4b3bae257c0720... It is pretty straightforward and not too hard to understand the API. Designing APIs with protobuf makes things convenient and it brings lots of already written tools with it. Since gRPC built on top of HTTP/2, we thought gRPC as easiest and performant way of writing a HTTP API with good defaults.

cmrdporcupine 6y ago
"we get much more without thinking too much" is precisely one of the reasons for the original push to REST (and against RPC at that time SOAP and CORBA and RMI etc.) was made in our industry 10-15 years ago. By falsely representing a _remote_ resource as if it was a _local_ one we open a whole can of worms; not just performance, but resilience, infrastructure issues, etc. Transferring documents over HTTP with a common language of HTTP's verbs was supposed to get programmers to model their applications as the sets of resources that they are, discourage them from making excessive round trips, make debugging easier, and make use of standard HTTP load balancing hardware. Yes it's easy to make that RPC call. But should you? I've been off doing other things in the intervening period, but while I had my back turned the industry seems to have turned its back on REST and gone whole hog on RPC. Again. At first I was thinking this was just internally here at Google, where protobufs and gRPC reign supreme. But it seems to have taken hold everywhere. What did I miss? Why have we swung this way. Again. Is the pendulum going to go back?
- apta 6y ago
  > By falsely representing a _remote_ resource as if it was a _local_ one we open a whole can of worms I've heard this argument before, but how does gRPC itself cause this issue to manifest? I'm curious to hear what your opinions are on a better alternative, and how not to represent a remote resource as a local one.
  - cmrdporcupine 6y ago
    You could use gRPC responsibly, yes. But the fact that it presents the remote resource in an API which resembles a local object means that programmers often get lazy in the manner of which they think about these things. The REST semantic is supposed to make this more explicit. A remote object is not an object in your program or even your computer. It's something you're taking from something that is computationally miles and miles away. Compared to the microseconds it takes to dispatch a local call, it's an eternity away, even on a local network of the highest speed. Accumulate those latencies over thousands of dispatches, and trouble can ensue. I am reminded of an observation from when I first joined Google, coming out of their acquisition of the scrappy awesome ads company I worked at (Admeld). We had a little service that kept track of ad impression caps / budgeting. We didn't want to serve an ad a single time more than the customer wanted us to, etc. Serving many thousands of ads per second, a process distributed across multiple machines in multiple data centres, this is a bit of a tricky shared state problem. The people who came before me had designed a rather clever solution which used a form of backoff to trickle down the number of ads served as they got closer and closer to budget cap, and to synchronize this state across clusters (this was before there were Rafty services to make this kind of thing easier, BTW). We had a bit of a show and tell with Google when we first joined. I wanted to know how they were handling this problem, since their scale was many times ours, so I asked the question and got a puzzled/annoyed look: "Oh we just make an RPC call to our budgeting server." Summary: if you're at Google you don't have to worry as much about these problems. You still do, but there's an insane amount of infrastructure and horsepower and an army of SREs to help make it happen. So, yeah, my point is -- just because Google does something or has invented something doesn't mean it's the best way to do it, especially in a smaller more cost conscious organization.