14 ms·

Application Load Balancers enables gRPC workloads with end to end HTTP/2 support

185 points by weitzj 6y ago

lvice 6y ago
I find interesting the discussion regarding gRPC support for Azure App Service, and the amount of moving parts involved to achieve such support... https://github.com/dotnet/aspnetcore/issues/9020#issuecomment-662583528 https://github.com/dotnet/aspnetcore/issues/9020#issuecommen...
- muststopmyths 6y ago
  Really illustrates the dumbassery of sticking a (relatively) fast-moving application-layer protocol into the kernel. Now you can't update the Web Server without updating the operating system. Might have been handy to beat benchmarks back in the day when people liked to whip them out for comparison, but IIS is under 10% according to Netcraft now. Time to fold up the tent and go home. I suppose .Net Core is sticking with Http.sys to avoid implementing their own web server, but is tying yourself to the Windows shipping cycle worth it ?
  - ComputerGuru 6y ago
    > Really illustrates the dumbassery of sticking a (relatively) fast-moving application-layer protocol into the kernel. Really? There are lots of problems with doing HTTP in the kernel, but “fast moving” is a new one. HTTP was stuck in permafrost from 1995 to 2015. If it weren’t for Google, neither of HTTP 2 or 3 would have ever happened.
  - halter73 6y ago
    The default ASP.NET Core server is Kestrel which runs in-process and is cross platform. Kestrel has officially supported gRPC for over a year now[1]. That linked GitHub comment is specifically about IIS gRPC support which relies on HTTP.sys (which runs in kernel mode and is tied to the Windows shipping cycle as noted). [1]: https://docs.microsoft.com/en-us/aspnet/core/grpc/?view=aspnetcore-3.1 https://docs.microsoft.com/en-us/aspnet/core/grpc/?view=aspn...
  - tybit 6y ago
    They’ve already folded up shop and solved these problems with Kestrel. Them continuing to offer support for laggards using IIS shouldn’t be criticised though.
k__ 6y ago
Half-OT: What's the main use-case for gRPC? I had the impression RPC was seen as a mistake. Sure, gRPC also uses a binary protocol, but that doesn't seem like a USP of gRPC. Why didn't they went fron non-RPC binary? Serious question! It sounds a bit counterinuitive to me at the first glance.
- snipewheelcelly 6y ago
  it's use case is companies that want to use SOAP but don't want to say they use SOAP
  - outworlder 6y ago
    How is it related to SOAP in any way?
    - bubersson 6y ago
      I'm not OP, but the main parallel is a well defined schema of communication between the services using different underlying technologies. SOAP is in my experience really hard to use and get right, compared to protobufs that bring well understandable set of primitives and intuitive support in many languages. gRPC is a solid carrier for protobufs. Yes, gRPC has many cons (e.g. with undefined/nil values, etc.), but overall it has worked great for our usecases.
      - thinkharderdev 6y ago
        Yeah, one way I've describe gRPC to colleagues (which may help or hurt depending on the perspective) is that is "SOAP, but without all the lunacy"
- q3k 6y ago
  It's a generic RPC protocol based on a well-enough-typed serialization format (protobuf) that is battle-tested. You'd use it where you'd use REST/API/JSONRPC/... Compared to plain JSON/REST RPC, it has all the advantages of protobuf over JSON (ie. strong typing, client/server code generation, API evolution, etc), but also provides some niceties at the transport layer: bidirectional streaming, high quality TCP connection multiplexing, ...
diogenesjunior 6y ago
Can somebody please explain to me why we would use gRPC?
- deleted 6y ago
  [deleted]
- jruroc 6y ago
  Serialization/deserialization speed and reducing transfer size are good reasons for large throughput service-to-service communication. Also a decent ecosystem around code generation from .proto files and gateways to still support some level of JSON-based calls.
- weitzj 6y ago
  It makes it really nice to define APIs (like with Openapi of swagger). There is a bunch of code generators out there to produce code for your definitions to have a native swift , objective , Java, Go api stubs for either clients or servers. It is a joy to work with in cross functional teams and define your APIs whilst taking into account what Api versioning would mean, how to define enums, how to rename field names whilst being compatible with the transport protocol and other things. Also if you were to route a payload from service A via B to C and each service is deployed independently and gets new Api changes, gRPC supports you in how to handle ther Szenarios. Sure enough openapi can do all of this I guess but grpc definitions in Protobuf or Google artman are just way quicker to understand and work with. (At least for me)
  - k__ 6y ago
    I see, cool. So the spec already includes versioning?
    - q3k 6y ago
      No, gRPC/protobuf instead provides you with ways to evolve your schema easily in the IDL and the result on the wire, without breaking either side. You can rename fields (but keep the tag number and therefore wire format compatibility), add fields (which will be ignored by the other side), remove fields (as all are explicitly optional so every consumer explicitly checks for their presence anyway), ignore unset fields (as the wire encoding is to a certain-degree self-describing), etc.
tchalla 6y ago
AWS, ALB, gRPC - it seems one needs to swallow a thesaurus to communicate these days.
- wongarsu 6y ago
  AWS is suffering from a TLA problem. gRPC on the other hand is a decent name, at least you can guess at a glance that is a RPC protocol. Meanwhile you just have to know that ALB is a type of ELB.
  - ForHackernews 6y ago
    ...only if you know what "RPC" means!
    - lights0123 6y ago
      Which is at least a very common thing across many fields, not a proprietary Amazon technology.
weitzj 6y ago
So the only remaining question is: When does AWS roll out quic support in ALBs?
- calcifer 6y ago
  If it's anything like HTTP/2, in 4-5 years.
- mrkurt 6y ago
  (Disclaimer: I work on Fly.io, this post is bias) It will probably be a while. We've been evaluating Quic and the ecosystem just isn't quite ready. We opted to release UDP support instead, so apps that want Quic can do it, but we can avoid adding much extra plumbing in front of the simple HTTP apps. Given how much AWS is investing in Rust, they'll probably ship first class support for Quic when Hyper does (same as us!): https://github.com/hyperium/hyper/issues/2078 https://github.com/hyperium/hyper/issues/2078
benreesman 6y ago
I started a recent project with gRPC but wound up moving to fbthrift after having a bad time with the C++ async server story. Overall I’d like to be using gRPC because the fbthrift documentation is weak, but thread-per-request is a non-starter for some use cases. From the gRPC source it looks like they’ve got plans to do something about it but it seems a ways off.
- jeffbee 6y ago
  What specifically was your problem with grpc c++ async? I'm using async C++ with one CQ per core and it seems to more or less work.
  - benreesman 6y ago
    It was not obvious to me how to support a large number of methods without a bunch of error-prone boilerplate. It seemed very low-level, which is fine if you have a small number of interactions but it started getting out of hand quickly and fbthrift does this neatly out of the box.
- apta 6y ago
  Are you bound to C++ for the implementation?
  - benreesman 6y ago
    Debatable, latency matters. It’s possible that Rust or a well-tuned JVM could be an alternative, but C++ is a sure thing and schedule matters too.
    - apta 6y ago
      What's the domain? Financial, or something else out of curiosity?
      - benreesman 6y ago
        Financial.
sneak 6y ago
This thread seems as good a place as any to ask: Does anyone have experience (good, bad, otherwise) using the gRPC JSON transcoding option for real-world stuff? I'm debating using it (still need REST clients sometimes) but I'm not sure how hacky it is.
- booi 6y ago
  It probably varies by the language and library but for java it has been flawless. I wouldn’t expect many issues for any major language.
- et1337 6y ago
  We use it. It's pretty good. It has a lot of places you can hook in extra functionality. You get most of the HTTP error status codes for free, but we also have a filter that looks at outgoing protobuf messages for a certain field that indicates the messages is a response to a create request, and that allows us to return an HTTP 202 instead of 200. We were even able to do Amazon-style request signing. One thing about request signing is that if you use protobuf key-value maps, the order is not deterministic on the wire. This broke our signing. Key-value maps are kind of a protobuf hack anyway, so we ended up using an array of structs. When it came time to add the JSON gateway, we found it pretty easy to write custom JSON serialization/deserialization code to convert the structs to a JSON map. This is all in Go by the way.
- weitzj 6y ago
  The grpc Gateway in Go worked quite well for us. I have not tried the native Envoy decoding functionality, yet. Also you should look at Google Artman on github/googleapis as sometimes it felt that defining the REST mappings in Protobuf were lacking some features. Using google artman you kind of mix/match Protobuf with yaml definitions of your service. We never had to use it, though. It just depends on where you want to put your authentication information. As of today I would probably change my mind and make it explicitly in payload, I.e. Protobuf message and not fiddle with headers any more.
- mariojv 6y ago
  This is only somewhat related, but I've used Go's protojson lib for pretty printing protobuf encoded data: https://godoc.org/google.golang.org/protobuf/encoding/protojson#Format https://godoc.org/google.golang.org/protobuf/encoding/protoj... They say not to rely on the output being stable, so I would recommend guaranteeing a stable translation yourself for a REST client. You can achieve this by translating from the JSON to your proto or grpc service structure yourself.
psnosignaluk 6y ago
I'm just going to jump in with an utterly pointless "woohoo!" As a gRPC shop, this is going to open up a lot of options for both our own infra and make it easier to support clients on AWS. Now if only Azure would make it easy to implement solutions that leverage gRPC...
malkia 6y ago
Used stubby at Google (mainly java), and was intimidated first, then saw the light - when almost everything uses the same way of talking, not only you get C++, java, python, go and other languages speaking freely to each other, but other extra benefits - for example each RPC can carry as a "tag" (key/value?) the user/group it came from, and this can be used for budgeting: For example - your internal backend A, calls service B, which then calls some other service C - so it's easy to log that A called B, but the fact that C was called because of what A asked is not, though if propagated (through the tag) then C can report - well I was called by B, but that was on on A to pay. Then dapper, their distributed tracing was helpful in the few times I had to deal with oncall (actually them asking me to do it). And in general, it felt like you don't have to write any low-level sockets code (which I love)
- jeffbee 6y ago
  Unfortch, gRPC brings none of these things. If you want delegated caller, originator, trace ID, or any other kind of baggage propagated down your RPC call graph, you are doing it yourself with metadata injectors and extractors at every service boundary.
  - gen220 6y ago
    Depending on your perspective, though, this can be seen as a positive thing: gRPC is extensible enough that all of this can be built on top. I'm sure that in 10 years, there will be more concepts like "trace IDs" that we will consider minimally necessary for distributed service architectures, that don't exist today. FWIW, writing the libs to do the metadata injection/extraction is pretty straightforward and transparent to application developers if they're done right.
    - jeffbee 6y ago
      There's always two sides to extensibility. One the one hand, you have the opportunity to do it your way. On the other, you have to do it. The extreme of extensibility is always an empty file. You get to pick the language and the architecture and everything!
fbru02 6y ago
When should I use XMPP and and RPC ?
forty 6y ago
I don't know gRPC: why does it need special load balancer support?
- wmf 6y ago
  gRPC requires HTTP/2 while many load balancers only support HTTP/1 on the backend.
  - weitzj 6y ago
    Before ALb, you could setup gRPC workloads with a layer 4 LB like Elb or Nlb but would have to roll your own TLS termination in a self hosted reverse proxy with gRPC support behind the LB. The downsides were: You can’t rely on ACM for certificate renewal The LAyer 4 NLB is “too dumb” to balance the traffic. You have a long running http/2 connection and maybe all go to reverse proxy instance A whilst the reverse proxy B replica is idle. It’s worse than it sounds. For us it worked. And maybe with The TLS support of NLBs and the feature that the NLB can set the ALPN header to h2, you actually might be able to use ACM with NLB for gRPC. But now with an ALB you get all of these features and can even load balance per request method (since it is layer 7) So for example you offer a unified Api and one method of this Api has a disproportional amount of traffic, you can do something about this already at the ALB
ainiriand 6y ago
I'm in the process process of advocating gRPC to my company that is starting to lay down the foundations to scale up. This presentation comes in handy.
bwarren2 6y ago
Does anyone have a favorite intro/guide/book on gRPC? I have been wanting to learn for a while.
- weitzj 6y ago
  grpc.io is great to start learning. Also the blog posts on grpc.io are interesting, but I find them harder to discover whilst reading the documentation. But here they are: https://grpc.io/blog/ https://grpc.io/blog/ Grasping the concept of a context/deadlines is quite helpful: https://grpc.io/blog/deadlines/ https://grpc.io/blog/deadlines/ You could also find related information in the Google SRE Handbook (Service Level Objectives): https://landing.google.com/sre/sre-book/chapters/service-level-objectives/ https://landing.google.com/sre/sre-book/chapters/service-lev... If you are familiar with Go, the article about "Context" might also be helpful: https://blog.golang.org/context https://blog.golang.org/context But in any case, gRPC is language agnostic and has nothing to do with Go. To get an idea how to create an api-repository with protobuf defintions to be shared by multiple services/clients, one can look at: https://github.com/googleapis/googleapis https://github.com/googleapis/googleapis
  - bwarren2 6y ago
    Thank you!!
  - Saser 6y ago
    In addition to these, I think that Google's API Design Guide (https://cloud.google.com/apis/design https://cloud.google.com/apis/design) and their AIPs (https://aip.dev https://aip.dev) are good references for learning about how their style of APIs, called resource-oriented APIs, can be designed. There is a linter that can check whether an API follows the AIPs (I know, these acronyms are easy to mix up), available at https://linter.aip.dev https://linter.aip.dev. I am building a side project following the AIPs and have found them to be very helpful. Disclaimer: I work at Google, although I would have recommended these resources anyway.
    - weitzj 6y ago
corytheboyd 6y ago
Maybe I’m crazy, but here is something I have been toying with recently. I have defined services in protobuff and generated static typescript definitions for the services and associated messages. I then implemented my own flavor of RPC over a WebSocket connection, where RPC calls are implemented as two calls— a “start” call from client to server, and a “result” call from server to client. It’s interesting and I don’t know if I would go this far down to the “metal” if you will on a team, but for my own project it’s been interesting.
- foota 6y ago
  I'd be curious, how do you handle associating rpc responses from the server with the request call site? Some sort of id?
  - corytheboyd 6y ago
    Yeah, request IDs that are used to invoke deferred callback functions

185 points by weitzj 6y ago

I find interesting the discussion regarding gRPC support for Azure App Service, and the amount of moving parts involved to achieve such support... https://github.com/dotnet/aspnetcore/issues/9020#issuecomment-662583528 https://github.com/dotnet/aspnetcore/issues/9020#issuecommen...

muststopmyths 6y ago
Really illustrates the dumbassery of sticking a (relatively) fast-moving application-layer protocol into the kernel. Now you can't update the Web Server without updating the operating system. Might have been handy to beat benchmarks back in the day when people liked to whip them out for comparison, but IIS is under 10% according to Netcraft now. Time to fold up the tent and go home. I suppose .Net Core is sticking with Http.sys to avoid implementing their own web server, but is tying yourself to the Windows shipping cycle worth it ?
- ComputerGuru 6y ago
  > Really illustrates the dumbassery of sticking a (relatively) fast-moving application-layer protocol into the kernel. Really? There are lots of problems with doing HTTP in the kernel, but “fast moving” is a new one. HTTP was stuck in permafrost from 1995 to 2015. If it weren’t for Google, neither of HTTP 2 or 3 would have ever happened.
- halter73 6y ago
  The default ASP.NET Core server is Kestrel which runs in-process and is cross platform. Kestrel has officially supported gRPC for over a year now[1]. That linked GitHub comment is specifically about IIS gRPC support which relies on HTTP.sys (which runs in kernel mode and is tied to the Windows shipping cycle as noted). [1]: https://docs.microsoft.com/en-us/aspnet/core/grpc/?view=aspnetcore-3.1 https://docs.microsoft.com/en-us/aspnet/core/grpc/?view=aspn...
- tybit 6y ago
  They’ve already folded up shop and solved these problems with Kestrel. Them continuing to offer support for laggards using IIS shouldn’t be criticised though.

k__ 6y ago

Half-OT: What's the main use-case for gRPC? I had the impression RPC was seen as a mistake. Sure, gRPC also uses a binary protocol, but that doesn't seem like a USP of gRPC. Why didn't they went fron non-RPC binary? Serious question! It sounds a bit counterinuitive to me at the first glance.

snipewheelcelly 6y ago
it's use case is companies that want to use SOAP but don't want to say they use SOAP
- outworlder 6y ago
  How is it related to SOAP in any way?
  - bubersson 6y ago
    I'm not OP, but the main parallel is a well defined schema of communication between the services using different underlying technologies. SOAP is in my experience really hard to use and get right, compared to protobufs that bring well understandable set of primitives and intuitive support in many languages. gRPC is a solid carrier for protobufs. Yes, gRPC has many cons (e.g. with undefined/nil values, etc.), but overall it has worked great for our usecases.
    - thinkharderdev 6y ago
      Yeah, one way I've describe gRPC to colleagues (which may help or hurt depending on the perspective) is that is "SOAP, but without all the lunacy"
q3k 6y ago
It's a generic RPC protocol based on a well-enough-typed serialization format (protobuf) that is battle-tested. You'd use it where you'd use REST/API/JSONRPC/... Compared to plain JSON/REST RPC, it has all the advantages of protobuf over JSON (ie. strong typing, client/server code generation, API evolution, etc), but also provides some niceties at the transport layer: bidirectional streaming, high quality TCP connection multiplexing, ...

diogenesjunior 6y ago

Can somebody please explain to me why we would use gRPC?

deleted 6y ago
[deleted]
jruroc 6y ago
Serialization/deserialization speed and reducing transfer size are good reasons for large throughput service-to-service communication. Also a decent ecosystem around code generation from .proto files and gateways to still support some level of JSON-based calls.
weitzj 6y ago
It makes it really nice to define APIs (like with Openapi of swagger). There is a bunch of code generators out there to produce code for your definitions to have a native swift , objective , Java, Go api stubs for either clients or servers. It is a joy to work with in cross functional teams and define your APIs whilst taking into account what Api versioning would mean, how to define enums, how to rename field names whilst being compatible with the transport protocol and other things. Also if you were to route a payload from service A via B to C and each service is deployed independently and gets new Api changes, gRPC supports you in how to handle ther Szenarios. Sure enough openapi can do all of this I guess but grpc definitions in Protobuf or Google artman are just way quicker to understand and work with. (At least for me)
- k__ 6y ago
  I see, cool. So the spec already includes versioning?
  - q3k 6y ago
    No, gRPC/protobuf instead provides you with ways to evolve your schema easily in the IDL and the result on the wire, without breaking either side. You can rename fields (but keep the tag number and therefore wire format compatibility), add fields (which will be ignored by the other side), remove fields (as all are explicitly optional so every consumer explicitly checks for their presence anyway), ignore unset fields (as the wire encoding is to a certain-degree self-describing), etc.

tchalla 6y ago

AWS, ALB, gRPC - it seems one needs to swallow a thesaurus to communicate these days.

wongarsu 6y ago
AWS is suffering from a TLA problem. gRPC on the other hand is a decent name, at least you can guess at a glance that is a RPC protocol. Meanwhile you just have to know that ALB is a type of ELB.
- ForHackernews 6y ago
  ...only if you know what "RPC" means!
  - lights0123 6y ago
    Which is at least a very common thing across many fields, not a proprietary Amazon technology.

weitzj 6y ago

So the only remaining question is: When does AWS roll out quic support in ALBs?

calcifer 6y ago
If it's anything like HTTP/2, in 4-5 years.
mrkurt 6y ago
(Disclaimer: I work on Fly.io, this post is bias) It will probably be a while. We've been evaluating Quic and the ecosystem just isn't quite ready. We opted to release UDP support instead, so apps that want Quic can do it, but we can avoid adding much extra plumbing in front of the simple HTTP apps. Given how much AWS is investing in Rust, they'll probably ship first class support for Quic when Hyper does (same as us!): https://github.com/hyperium/hyper/issues/2078 https://github.com/hyperium/hyper/issues/2078

benreesman 6y ago

I started a recent project with gRPC but wound up moving to fbthrift after having a bad time with the C++ async server story. Overall I’d like to be using gRPC because the fbthrift documentation is weak, but thread-per-request is a non-starter for some use cases. From the gRPC source it looks like they’ve got plans to do something about it but it seems a ways off.

jeffbee 6y ago
What specifically was your problem with grpc c++ async? I'm using async C++ with one CQ per core and it seems to more or less work.
- benreesman 6y ago
  It was not obvious to me how to support a large number of methods without a bunch of error-prone boilerplate. It seemed very low-level, which is fine if you have a small number of interactions but it started getting out of hand quickly and fbthrift does this neatly out of the box.
apta 6y ago
Are you bound to C++ for the implementation?
- benreesman 6y ago
  Debatable, latency matters. It’s possible that Rust or a well-tuned JVM could be an alternative, but C++ is a sure thing and schedule matters too.
  - apta 6y ago
    What's the domain? Financial, or something else out of curiosity?
    - benreesman 6y ago
      Financial.

sneak 6y ago

This thread seems as good a place as any to ask: Does anyone have experience (good, bad, otherwise) using the gRPC JSON transcoding option for real-world stuff? I'm debating using it (still need REST clients sometimes) but I'm not sure how hacky it is.

booi 6y ago
It probably varies by the language and library but for java it has been flawless. I wouldn’t expect many issues for any major language.
et1337 6y ago
We use it. It's pretty good. It has a lot of places you can hook in extra functionality. You get most of the HTTP error status codes for free, but we also have a filter that looks at outgoing protobuf messages for a certain field that indicates the messages is a response to a create request, and that allows us to return an HTTP 202 instead of 200. We were even able to do Amazon-style request signing. One thing about request signing is that if you use protobuf key-value maps, the order is not deterministic on the wire. This broke our signing. Key-value maps are kind of a protobuf hack anyway, so we ended up using an array of structs. When it came time to add the JSON gateway, we found it pretty easy to write custom JSON serialization/deserialization code to convert the structs to a JSON map. This is all in Go by the way.
weitzj 6y ago
The grpc Gateway in Go worked quite well for us. I have not tried the native Envoy decoding functionality, yet. Also you should look at Google Artman on github/googleapis as sometimes it felt that defining the REST mappings in Protobuf were lacking some features. Using google artman you kind of mix/match Protobuf with yaml definitions of your service. We never had to use it, though. It just depends on where you want to put your authentication information. As of today I would probably change my mind and make it explicitly in payload, I.e. Protobuf message and not fiddle with headers any more.
mariojv 6y ago
This is only somewhat related, but I've used Go's protojson lib for pretty printing protobuf encoded data: https://godoc.org/google.golang.org/protobuf/encoding/protojson#Format https://godoc.org/google.golang.org/protobuf/encoding/protoj... They say not to rely on the output being stable, so I would recommend guaranteeing a stable translation yourself for a REST client. You can achieve this by translating from the JSON to your proto or grpc service structure yourself.

psnosignaluk 6y ago

I'm just going to jump in with an utterly pointless "woohoo!" As a gRPC shop, this is going to open up a lot of options for both our own infra and make it easier to support clients on AWS. Now if only Azure would make it easy to implement solutions that leverage gRPC...

malkia 6y ago

Used stubby at Google (mainly java), and was intimidated first, then saw the light - when almost everything uses the same way of talking, not only you get C++, java, python, go and other languages speaking freely to each other, but other extra benefits - for example each RPC can carry as a "tag" (key/value?) the user/group it came from, and this can be used for budgeting: For example - your internal backend A, calls service B, which then calls some other service C - so it's easy to log that A called B, but the fact that C was called because of what A asked is not, though if propagated (through the tag) then C can report - well I was called by B, but that was on on A to pay. Then dapper, their distributed tracing was helpful in the few times I had to deal with oncall (actually them asking me to do it). And in general, it felt like you don't have to write any low-level sockets code (which I love)

jeffbee 6y ago
Unfortch, gRPC brings none of these things. If you want delegated caller, originator, trace ID, or any other kind of baggage propagated down your RPC call graph, you are doing it yourself with metadata injectors and extractors at every service boundary.
- gen220 6y ago
  Depending on your perspective, though, this can be seen as a positive thing: gRPC is extensible enough that all of this can be built on top. I'm sure that in 10 years, there will be more concepts like "trace IDs" that we will consider minimally necessary for distributed service architectures, that don't exist today. FWIW, writing the libs to do the metadata injection/extraction is pretty straightforward and transparent to application developers if they're done right.
  - jeffbee 6y ago
    There's always two sides to extensibility. One the one hand, you have the opportunity to do it your way. On the other, you have to do it. The extreme of extensibility is always an empty file. You get to pick the language and the architecture and everything!

fbru02 6y ago

When should I use XMPP and and RPC ?

forty 6y ago

I don't know gRPC: why does it need special load balancer support?

wmf 6y ago
gRPC requires HTTP/2 while many load balancers only support HTTP/1 on the backend.
- weitzj 6y ago
  Before ALb, you could setup gRPC workloads with a layer 4 LB like Elb or Nlb but would have to roll your own TLS termination in a self hosted reverse proxy with gRPC support behind the LB. The downsides were: You can’t rely on ACM for certificate renewal The LAyer 4 NLB is “too dumb” to balance the traffic. You have a long running http/2 connection and maybe all go to reverse proxy instance A whilst the reverse proxy B replica is idle. It’s worse than it sounds. For us it worked. And maybe with The TLS support of NLBs and the feature that the NLB can set the ALPN header to h2, you actually might be able to use ACM with NLB for gRPC. But now with an ALB you get all of these features and can even load balance per request method (since it is layer 7) So for example you offer a unified Api and one method of this Api has a disproportional amount of traffic, you can do something about this already at the ALB

ainiriand 6y ago

I'm in the process process of advocating gRPC to my company that is starting to lay down the foundations to scale up. This presentation comes in handy.

bwarren2 6y ago

Does anyone have a favorite intro/guide/book on gRPC? I have been wanting to learn for a while.

weitzj 6y ago
grpc.io is great to start learning. Also the blog posts on grpc.io are interesting, but I find them harder to discover whilst reading the documentation. But here they are: https://grpc.io/blog/ https://grpc.io/blog/ Grasping the concept of a context/deadlines is quite helpful: https://grpc.io/blog/deadlines/ https://grpc.io/blog/deadlines/ You could also find related information in the Google SRE Handbook (Service Level Objectives): https://landing.google.com/sre/sre-book/chapters/service-level-objectives/ https://landing.google.com/sre/sre-book/chapters/service-lev... If you are familiar with Go, the article about "Context" might also be helpful: https://blog.golang.org/context https://blog.golang.org/context But in any case, gRPC is language agnostic and has nothing to do with Go. To get an idea how to create an api-repository with protobuf defintions to be shared by multiple services/clients, one can look at: https://github.com/googleapis/googleapis https://github.com/googleapis/googleapis
- bwarren2 6y ago
  Thank you!!
- Saser 6y ago
  In addition to these, I think that Google's API Design Guide (https://cloud.google.com/apis/design https://cloud.google.com/apis/design) and their AIPs (https://aip.dev https://aip.dev) are good references for learning about how their style of APIs, called resource-oriented APIs, can be designed. There is a linter that can check whether an API follows the AIPs (I know, these acronyms are easy to mix up), available at https://linter.aip.dev https://linter.aip.dev. I am building a side project following the AIPs and have found them to be very helpful. Disclaimer: I work at Google, although I would have recommended these resources anyway.
  - weitzj 6y ago

corytheboyd 6y ago

Maybe I’m crazy, but here is something I have been toying with recently. I have defined services in protobuff and generated static typescript definitions for the services and associated messages. I then implemented my own flavor of RPC over a WebSocket connection, where RPC calls are implemented as two calls— a “start” call from client to server, and a “result” call from server to client. It’s interesting and I don’t know if I would go this far down to the “metal” if you will on a team, but for my own project it’s been interesting.

foota 6y ago
I'd be curious, how do you handle associating rpc responses from the server with the request call site? Some sort of id?
- corytheboyd 6y ago
  Yeah, request IDs that are used to invoke deferred callback functions