17 ms·

gRPC in Production

186 points by loppers92 9y ago

nemothekid 9y ago
Thrift is Facebook's version of gRPC right? If so, I don't quite understand the comparison, and how gRPC succeeds where thrift "fails". Wouldn't all language implementations of gRPC have to be well documented, reliable, highly performant and easy to install?
- azurezyq 9y ago
  on the contrary, I always found thrift lack good documentation.
  - vikiomega9 9y ago
    That's the sense I got too but beyond the initial hurdles it's good enough.
- atombender 9y ago
  I guess Google wanted to start with a clean slate based on the design principles already established internally (a system called Stubby). gRPC was designed from the start for HTTP/2, which comes with some benefits: It's able to work wherever HTTP works (load balancers and proxies), can multiplex calls over a single stream (Thrift on the JVM, where it's most popular, uses a thread per socket), supports cancelation and streaming and so on. gRPC is arguably more opinionated than Thrift here; Thrift is both a serialization format and an RPC mechanism, and for Thrift RPC you can choose between different transports and framings, of which HTTP is just one. gRPC is HTTP-only, and is (at least nominally) serialization-format-agnostic; you could, in principle, use Thrift over gRPC instead of Protocol Buffers, for example. So in this sense, gRPC is more pure and generic than Thrift. (I'm sure Thrift fans might disagree here.) I haven't actually used Thrift, so maybe someone else can chime in about other reasons gRPC is preferable.
- wrsh07 9y ago
  My understanding is that Thrift is Facebook's analog to Google's Stubby. Or gRPC [which is the next generation of Stubby]. It's not implementing the gRPC interface. I don't think gRPC was open-sourced [2015] early enough for Thrift [2007] to be built to its API. Disclosure: Google employee who uses Stubby. [but is not on the team]
  - puzzle 9y ago
    Thrift was written at Facebook by an ex Google intern trying to recreate something close to protobuf+Stubby.
  - deleted 9y ago
    [deleted]
- fizx 9y ago
  Thrift is comparable to GRPC, and more user-friendly in practice than GRPC. GRPC is more flexible, rigorous (anal). I disagree that Thrift failed, but GRPC has more momentum right now IMO. Source (GRPC at current startup, multiple years of thrift)
  - cookiecaper 9y ago
    Thrift is popular and it "supports" more targets, but I found a lot of bugs in some of its lesser-known implementations. This may not be an issue if you're using a widely-used Thrift platform, but if you're choosing Thrift because it supports the target you want while another RPC doesn't, do watch out for weird bugs and issues. I haven't tried to use Protobufs or gRPC in a serious way so I can't say if it's better or worse, but I would hope it supports fewer targets because it takes stability and QC more seriously.
brango 9y ago
The only thing holding back gRPC is JS web support. If it had that it'd be time to drop swagger completely. As it is you need to go protobuf -> swagger -> js lib, but it's cumbersome and doesn't work 100% (e.g adding auth keys, etc.). Will there be any progress on JS web, or can it just not be done at all with HTTP1? Even a subset of features for basic GET/POST would be fine...
- buckhx 9y ago
  We've been using grpc-web from Improbable successfully for internal services. I'm working on a more production scale application soon that I'm planning on leveraging grpc-web for. The biggest missing feature is client side streaming support which is hamstrung the new wgfetch/streams API, but I feel confident in finding solutions for situations that generally require client side streaming. https://github.com/improbable-eng/grpc-web https://github.com/improbable-eng/grpc-web
  - brango 9y ago
    I hadn't seen that. Thanks!
  - simonhorlick 9y ago
    We've just gone live with a grpc-web based application. It's still early days and there are some rough edges, but it's an absolute joy to use and the amount of time we've saved is crazy.
- cshenton 9y ago
  There's also the fact that, for high level languages like python, the gRPC server has lower throughput than available python http servers. Even with the overhead of parsing http and whatever message format you're sending over it.
  - weberc2 9y ago
    This is surprising given that there Python implementation is apparently C. Anyway I can't imagine this is the bottleneck in any nontrivial Python app.
_ydfu 9y ago
Another downside in gRPC is that (de)serialization can be relatively slow. One of my pet projects is a tagging server that takes a list of tags and returns a list of objects that match, retrieved from RocksDB. I tested gRPC (proto and FlatBuffers) and Cap'n'Proto. For all of them, the process looked like: Insert: Object comes in (built in RPC system for both), object is assigned ID (which is added before serialization), object is written to RocksDB, object is serialized and indexed. Query: List of strings come in, strings are looked up in tag index, intersection of results is done, objects are retrieved from RocksDB, objects are deserialized, objects are added to list, objects are returned to client. FlatBuffers was unfortunately a no-go since it doesn't permit modification of fields that haven't yet been set and it didn't have a sane way of making a copy (which would also be slow). Protocol Buffers worked but the pure Python driver is incredibly slow and the C++ driver for Python was non-default and marked "experimental". I eventually adapted to it but it was still quite slow even on the server side. From memory a response with C++ client and server took ~9ms, 4-5ms of which were spent on serialization/deserialization. Cap'n'Proto eventually won for me. The Python driver is unstable and memory usage soars like an eagle until Linux shoots it down with an OOM but the server was much faster. Typical response times were closer to 3ms, most of which was spent in RocksDB or the actual index lookup. A downside though was that Cap'n'Proto has its own built in and odd library for async stuff and doesn't really support threading.
- deleted 9y ago
  [deleted]
- shaklee3 9y ago
  I looked at capn proto as well, but ultimately decided against it because the common view online was grpc was much better at rpc. Maybe it's gotten better since then, but capnproto at the time was really comparable to protobufs and not grpc.
  - maccard 9y ago
    Last time I looked, capn porto was a no-go on Windows for RPC.
stevvooe 9y ago
Decent overview, but, remember that when evaluating something, projects change over time. Even better, you can be the change you want to see! Thus far, the grpc project has been fairly responsive in making solid changes, either through PRs or filing issues. ;) Regarding the complaint about errors, there is already protocol support for structured error handling: https://godoc.org/google.golang.org/grpc/status https://godoc.org/google.golang.org/grpc/status. https://github.com/grpc/grpc-go/pull/1358 https://github.com/grpc/grpc-go/pull/1358 should make this easier to use. In practice, the provided error code set is very good, so give them a try before making things more complex. Worst case, you can just stuff things in a header or trailer.
misterbowfinger 9y ago
> Inefficient (textual representations aren’t optimal for networks) REST APIs don't _have_ to be text-based, AFAIK. Why not just send/receive binary?
- ehsankia 9y ago
  Sure, but then you'd have to write a serializer/deserializer for your messages. You could also use an existing one like Bson but then why not just use Protobuf?
  - cshenton 9y ago
    Because bson, msgpack etc are self describing? It means a way lower barrier to entry.
    - throwaway91111 9y ago
      I was under the impression it was trivial to store the message description in the message itself with protobuf.
    - adrianmonk 9y ago
      I'm not sure I see how self-describing is a lower barrier to entry. It seems like generally the main reason why you'd want a self-describing format is if you're writing client (and server) language bindings by hand. If you have a tool to auto-generate those language bindings, you can skip that step, and there's no need to make a step easier if it's done for you automatically. I can see where self-describing is better when it comes to side issues like debugging, exploration, or the hassle of configuring your build to generate code from the IDL files, though. But if I had to choose only one, those are lower priority to me.
    - duality 9y ago
      But sadly it also means redundancy with every message sent, which must carrying this self-description.
- morecoffee 9y ago
tnolet 9y ago
The point about operations is a very, very valid constraint of REST. Easy and common stuff like, "run this thing in the background", "send of this one, ephemeral, message" are very unnatural. Maybe a hybrid / bastard child of REST and gRPC would be a good marriage of resource and operations modelling.
- QuercusMax 9y ago
  The Google Cloud standard for async operations via the google.longrunning.Operation service: https://github.com/googleapis/googleapis/blob/master/google/longrunning/operations.proto https://github.com/googleapis/googleapis/blob/master/google/.... This should be usable either via REST/JSON or gRPC. It's designed to be generic for any kind of async operation, and to be "mixed in" with your APIs. There are utilities for waiting for an operation to finish (via polling); in theory it should be possible to use some type of server-push to avoid polling, but I'm not sure if anybody's doing this. (I'm an Alphabet employee who's working on APIs delivered via gRPC / REST/JSON.)
tnolet 9y ago
The point about operations is a very, very valid constraint of REST. Easy and common stuff like, "run this thing in the background", "send of this one, ephemeral, message" are very unnatural. Maybe a hybrid / bastard child of REST and gRPC would be a good marriage of resource and operations modelling.
joneholland 9y ago
You can add Expedia to the list of production grpc users. Our entire hotel pricing backend is grpc microservices written in Scala. Some of these services take > 100k TPS. We have found grpc to be extremely scaleable. If that sounds interesting, I'm hiring engineers, shoot me a note at joholland at Expedia.com
- morecoffee 9y ago
  Do you use the Java generated stubs or generate your own? There are several people using gRPC with Scala that could benefit from more idiomatic stubs.
  - joneholland 9y ago
    We use the java stubs. We briefly looked at some scala proto generators, but most times you dump the proto class into a rich scala type right away anyways.

186 points by loppers92 9y ago

Thrift is Facebook's version of gRPC right? If so, I don't quite understand the comparison, and how gRPC succeeds where thrift "fails". Wouldn't all language implementations of gRPC have to be well documented, reliable, highly performant and easy to install?

azurezyq 9y ago
on the contrary, I always found thrift lack good documentation.
- vikiomega9 9y ago
  That's the sense I got too but beyond the initial hurdles it's good enough.
atombender 9y ago
I guess Google wanted to start with a clean slate based on the design principles already established internally (a system called Stubby). gRPC was designed from the start for HTTP/2, which comes with some benefits: It's able to work wherever HTTP works (load balancers and proxies), can multiplex calls over a single stream (Thrift on the JVM, where it's most popular, uses a thread per socket), supports cancelation and streaming and so on. gRPC is arguably more opinionated than Thrift here; Thrift is both a serialization format and an RPC mechanism, and for Thrift RPC you can choose between different transports and framings, of which HTTP is just one. gRPC is HTTP-only, and is (at least nominally) serialization-format-agnostic; you could, in principle, use Thrift over gRPC instead of Protocol Buffers, for example. So in this sense, gRPC is more pure and generic than Thrift. (I'm sure Thrift fans might disagree here.) I haven't actually used Thrift, so maybe someone else can chime in about other reasons gRPC is preferable.
wrsh07 9y ago
My understanding is that Thrift is Facebook's analog to Google's Stubby. Or gRPC [which is the next generation of Stubby]. It's not implementing the gRPC interface. I don't think gRPC was open-sourced [2015] early enough for Thrift [2007] to be built to its API. Disclosure: Google employee who uses Stubby. [but is not on the team]
- puzzle 9y ago
  Thrift was written at Facebook by an ex Google intern trying to recreate something close to protobuf+Stubby.
- deleted 9y ago
  [deleted]
fizx 9y ago
Thrift is comparable to GRPC, and more user-friendly in practice than GRPC. GRPC is more flexible, rigorous (anal). I disagree that Thrift failed, but GRPC has more momentum right now IMO. Source (GRPC at current startup, multiple years of thrift)
- cookiecaper 9y ago
  Thrift is popular and it "supports" more targets, but I found a lot of bugs in some of its lesser-known implementations. This may not be an issue if you're using a widely-used Thrift platform, but if you're choosing Thrift because it supports the target you want while another RPC doesn't, do watch out for weird bugs and issues. I haven't tried to use Protobufs or gRPC in a serious way so I can't say if it's better or worse, but I would hope it supports fewer targets because it takes stability and QC more seriously.

brango 9y ago

The only thing holding back gRPC is JS web support. If it had that it'd be time to drop swagger completely. As it is you need to go protobuf -> swagger -> js lib, but it's cumbersome and doesn't work 100% (e.g adding auth keys, etc.). Will there be any progress on JS web, or can it just not be done at all with HTTP1? Even a subset of features for basic GET/POST would be fine...

buckhx 9y ago
We've been using grpc-web from Improbable successfully for internal services. I'm working on a more production scale application soon that I'm planning on leveraging grpc-web for. The biggest missing feature is client side streaming support which is hamstrung the new wgfetch/streams API, but I feel confident in finding solutions for situations that generally require client side streaming. https://github.com/improbable-eng/grpc-web https://github.com/improbable-eng/grpc-web
- brango 9y ago
  I hadn't seen that. Thanks!
- simonhorlick 9y ago
  We've just gone live with a grpc-web based application. It's still early days and there are some rough edges, but it's an absolute joy to use and the amount of time we've saved is crazy.
cshenton 9y ago
There's also the fact that, for high level languages like python, the gRPC server has lower throughput than available python http servers. Even with the overhead of parsing http and whatever message format you're sending over it.
- weberc2 9y ago
  This is surprising given that there Python implementation is apparently C. Anyway I can't imagine this is the bottleneck in any nontrivial Python app.

_ydfu 9y ago

Another downside in gRPC is that (de)serialization can be relatively slow. One of my pet projects is a tagging server that takes a list of tags and returns a list of objects that match, retrieved from RocksDB. I tested gRPC (proto and FlatBuffers) and Cap'n'Proto. For all of them, the process looked like: Insert: Object comes in (built in RPC system for both), object is assigned ID (which is added before serialization), object is written to RocksDB, object is serialized and indexed. Query: List of strings come in, strings are looked up in tag index, intersection of results is done, objects are retrieved from RocksDB, objects are deserialized, objects are added to list, objects are returned to client. FlatBuffers was unfortunately a no-go since it doesn't permit modification of fields that haven't yet been set and it didn't have a sane way of making a copy (which would also be slow). Protocol Buffers worked but the pure Python driver is incredibly slow and the C++ driver for Python was non-default and marked "experimental". I eventually adapted to it but it was still quite slow even on the server side. From memory a response with C++ client and server took ~9ms, 4-5ms of which were spent on serialization/deserialization. Cap'n'Proto eventually won for me. The Python driver is unstable and memory usage soars like an eagle until Linux shoots it down with an OOM but the server was much faster. Typical response times were closer to 3ms, most of which was spent in RocksDB or the actual index lookup. A downside though was that Cap'n'Proto has its own built in and odd library for async stuff and doesn't really support threading.

deleted 9y ago
[deleted]
shaklee3 9y ago
I looked at capn proto as well, but ultimately decided against it because the common view online was grpc was much better at rpc. Maybe it's gotten better since then, but capnproto at the time was really comparable to protobufs and not grpc.
- maccard 9y ago
  Last time I looked, capn porto was a no-go on Windows for RPC.

stevvooe 9y ago

Decent overview, but, remember that when evaluating something, projects change over time. Even better, you can be the change you want to see! Thus far, the grpc project has been fairly responsive in making solid changes, either through PRs or filing issues. ;) Regarding the complaint about errors, there is already protocol support for structured error handling: https://godoc.org/google.golang.org/grpc/status https://godoc.org/google.golang.org/grpc/status. https://github.com/grpc/grpc-go/pull/1358 https://github.com/grpc/grpc-go/pull/1358 should make this easier to use. In practice, the provided error code set is very good, so give them a try before making things more complex. Worst case, you can just stuff things in a header or trailer.

misterbowfinger 9y ago

> Inefficient (textual representations aren’t optimal for networks) REST APIs don't _have_ to be text-based, AFAIK. Why not just send/receive binary?

ehsankia 9y ago
Sure, but then you'd have to write a serializer/deserializer for your messages. You could also use an existing one like Bson but then why not just use Protobuf?
- cshenton 9y ago
  Because bson, msgpack etc are self describing? It means a way lower barrier to entry.
  - throwaway91111 9y ago
    I was under the impression it was trivial to store the message description in the message itself with protobuf.
  - adrianmonk 9y ago
    I'm not sure I see how self-describing is a lower barrier to entry. It seems like generally the main reason why you'd want a self-describing format is if you're writing client (and server) language bindings by hand. If you have a tool to auto-generate those language bindings, you can skip that step, and there's no need to make a step easier if it's done for you automatically. I can see where self-describing is better when it comes to side issues like debugging, exploration, or the hassle of configuring your build to generate code from the IDL files, though. But if I had to choose only one, those are lower priority to me.
  - duality 9y ago
    But sadly it also means redundancy with every message sent, which must carrying this self-description.
morecoffee 9y ago

tnolet 9y ago

The point about operations is a very, very valid constraint of REST. Easy and common stuff like, "run this thing in the background", "send of this one, ephemeral, message" are very unnatural. Maybe a hybrid / bastard child of REST and gRPC would be a good marriage of resource and operations modelling.

QuercusMax 9y ago
The Google Cloud standard for async operations via the google.longrunning.Operation service: https://github.com/googleapis/googleapis/blob/master/google/longrunning/operations.proto https://github.com/googleapis/googleapis/blob/master/google/.... This should be usable either via REST/JSON or gRPC. It's designed to be generic for any kind of async operation, and to be "mixed in" with your APIs. There are utilities for waiting for an operation to finish (via polling); in theory it should be possible to use some type of server-push to avoid polling, but I'm not sure if anybody's doing this. (I'm an Alphabet employee who's working on APIs delivered via gRPC / REST/JSON.)

tnolet 9y ago

joneholland 9y ago

You can add Expedia to the list of production grpc users. Our entire hotel pricing backend is grpc microservices written in Scala. Some of these services take > 100k TPS. We have found grpc to be extremely scaleable. If that sounds interesting, I'm hiring engineers, shoot me a note at joholland at Expedia.com

morecoffee 9y ago
Do you use the Java generated stubs or generate your own? There are several people using gRPC with Scala that could benefit from more idiomatic stubs.
- joneholland 9y ago
  We use the java stubs. We briefly looked at some scala proto generators, but most times you dump the proto class into a rich scala type right away anyways.